Edited from my
comment
on a
LinkedIn post
by Roger Dooley, who I don't know.
Roger wrote about an
article in "Nature"
that describes
Centaur
, an LLM that predicts human behavior with uncanny accuracy. It reminded me of another project that's similar, but in
an opposite way.
A team of researchers created an interesting experiment that tries to measure one aspect of creativity. From
www.datcreativity.com/about:
🙶The Divergent Association Task is a quick measure of verbal creativity and divergent thinking, the ability to
generate diverse solutions to open-ended problems.🙷
So it's sort of like the project Roger described, but in reverse: Instead of asking an LLM to predict human decisions,
in a sense it analyzes our ability to be unpredictable. It asks for 10 common nouns, chooses the first 7 that it deems
🙶acceptable,🙷 and estimates how different they are from each other (emphasis added):
🙶The average score is 78, and most people score between 74 and 82. The lowest score was 24 and the highest was 96 in
our published sample. Although the scores can theoretically range from 0 to 200, in practice they range from 6 to
around 110 after millions of responses online.🙷
For fun, I thought it might be interesting to compare a human's results to a few LLMs'. Guess who won? 😀️
🤖 Gemini scored 67.98, higher than 5.90% of the people who completed the task.
🤖 ChatGPT scored 79.1, higher than 54.23% of the people who completed the task.
🧑 I scored 86.8, higher than 91.52% of the people who completed the task.
Given this highly scientific (ha!) sample size of 1, it seems we humans are still better than LLMs at being human. If you choose to
take the test, I'd love to know your score—please post it in the comments!