How AI Models Measure Up in Creativity:

In AI - Artificial Intelligence, Digital marketing by Julio Ahumada

Insights from Recent Research on AI and creativity.

A massive study involving 100,000 humans and large language models (LLMs) sheds light on how artificial intelligence performs in creative tasks compared to human capabilities. The study, as outlined by Canadian data scientist and multidisciplinary creator Briana Brownell, aimed to explore AI's creative potential using a well-known measure of creativity, the Divergent Association Task (DAT). The findings reveal important lessons for getting more creative outputs from AI models.

Understanding the Divergent Association Task (DAT)

The DAT is a tool used to assess divergent thinking, a key element of creativity defined as the ability to generate varied and novel solutions to open-ended problems. Divergent thinking is often tested through tasks like brainstorming multiple uses for a common object or coming up with creative ideas for marketing a product. The DAT specifically challenges individuals or AI models to list 10 nouns that are "semantically distant" from one another, meaning the words must differ significantly in meaning.

In the study, 100,000 human participants and nine different AI models (each tested 500 times) were tasked with generating these words. The researchers then measured the semantic distance between the words to evaluate creativity.

AI vs. Human Performance

Surprisingly, the AI models outperformed humans in this specific task, with OpenAI’s GPT-4 emerging as the top performer. It surpassed human performance on the DAT, while another model, GeminiPro, matched human abilities. Notably, GPT-4 Turbo performed worse than GPT-3, its predecessor.

However, while LLMs excelled in producing semantically distant words, humans showed greater diversity in their responses. The study revealed that certain models, like GPT-4 and Claude 3, tended to repeat specific favorite words across multiple attempts, such as "microscope," "elephant," and "volcano." On the other hand, lesser-known models like RedPajama and Pythia displayed a wider range of results, though their performance was inconsistent.

Here’s an example of the highest-scoring human response on the DAT, which achieved a score of 95.7

research on AI and creativity
  • Javelin
  • Haemoglobin
  • Citrus
  • Gangrene
  • Upstairs
  • Microphone
  • Numbat
  • Tarantula
  • Question
  • Paraglider

Temperature and Creativity

AI Research shows CHATGPT Outperforms humans in creativity

One of the key findings of the research was how adjusting the "temperature" parameter of an LLM can enhance its creativity. In AI models, temperature controls the randomness of responses. A higher temperature (closer to 1) encourages more varied outputs, while a lower temperature (closer to 0) produces more predictable results.

When the researchers increased the temperature to 1.5, the AI models generated more creative and diverse responses. In fact, at this temperature, the models achieved higher average creativity scores than 72% of the human participants. This suggests that creativity in AI can be fine-tuned by adjusting its temperature settings.

Image

Prompting Strategies Matter

Another fascinating aspect of the study was the impact of different prompting strategies on the AI's performance. When the researchers prompted GPT-4 to vary its word choices based on etymology, the model’s creativity scores improved slightly. However, not all strategies led to better results. For instance, when the model was instructed to generate words using pairs of opposites (e.g., "freedom" and "slavery"), it performed worse than random word generation, highlighting the importance of choosing the right strategy.

Creativity Beyond the DAT

To further explore AI’s creative potential, the researchers tested the top three models (GPT-3, Vicuna, and GPT-4) on additional tasks such as writing haikus, movie synopsis, and flash fiction. Here, humans emerged victorious, although GPT-4 still outperformed the other models. Once again, increasing the temperature helped AI models perform better in creative writing tasks, particularly in generating flash fiction and synopsis.

Key Takeaways

Here are some practical insights from the research that can help you harness AI for creative tasks:

  1. Choose the most creative models: Larger frontier models, like GPT-4, tend to perform better in creativity tests. However, smaller models like Vicuna can also deliver strong results in certain cases.
  2. Test creativity yourself: The Divergent Association Test can be a useful tool for evaluating the creativity of different AI models. Conducting multiple tests can provide a more accurate assessment.
  3. Tweak the temperature: Adjusting the model's temperature can significantly impact creativity. While some platforms hide this feature, you can often influence it through specific prompts or by using advanced interfaces.
  4. Good strategies lead to better results: Prompting with a well-thought-out strategy can enhance creativity, while a poor strategy can worsen outcomes. Experiment with different approaches to find what works best for your needs.
  5. Combine multiple models: Using responses from different models increases your chances of generating highly creative outputs. Mixing human input with AI-generated ideas can yield even better results.

Conclusion

This research highlights the growing potential of AI in creative domains, with models like GPT-4 capable of rivaling—and in some cases surpassing—human performance on certain tasks. By understanding how to optimize LLMs through temperature settings, strategic prompting, and model selection, users can tap into the full creative potential of AI.

The article highlights how AI, especially models like GPT-4, can match and sometimes surpass human creativity in specific tasks. While AI excels in structured creativity tests and can produce diverse outputs with fine-tuning, human creativity still stands out in its unpredictability and nuanced understanding of context. AI models offer incredible efficiency and innovation but still rely on human input for truly groundbreaking ideas. It’s a tool that amplifies creativity, not replaces it.

I find working with AI  fascinating! There is no possible way I could be as productive without it. The article suggests that while AI models like GPT-4 can outperform humans on specific creativity tasks, it’s important to recognize the context. These tests, like the Divergent Association Task (DAT), measure certain aspects of creativity—mainly divergent thinking—but creativity in humans involves more than just generating unique words. It’s about emotional nuance, life experience, and abstract problem-solving, which AI still lacks.

What’s compelling here is how AI can be fine-tuned for better creative output, but it also shows that AI’s "creativity" is highly dependent on input (prompts, temperature, etc.). Humans are still better at generating diverse, original ideas in a broader sense, but AI can be a powerful tool to augment and even enhance the creative process. The future of creativity may lie in the collaboration between human ingenuity and AI’s computational capabilities.

It’s not so much about AI “surpassing” human creativity—it’s more that it’s learning to mimic aspects of it in unique ways. We may be able to leverage this for greater innovation if we play to each other’s strengths.

For more article on AI and the power it has to increase our productivity. https://dnadigitalmarketing.com/incorporating-data-analytics-into-your-business/  https://dnadigitalmarketing.com/ai-tools-in-digital-marketing/

 

If you found this interesting, subscribe to our blog.