AI and Creativity

A recent article (paywalled) summarised the July 2024 paper "Generative AI enhances individual creativity but reduces the collective diversity of novel content" by Doshi and Hauser (publicly accessible).  The researchers wanted to quantify how Large Language Models (specifically GPT-4) could improve the creativity of short stories using two metrics: novelty (originality) and usefulness (potential for becoming a publishable work).  They also assessed qualitative characteristics including writing style, how enjoyable, and how funny the stories were.  

The study group of 293 people were assessed for creativity using the Divergent Association Task (DAT).   They were randomly assigned to one of three groups: no AI support, able to use one AI-generated idea, and able to use up to five AI-generated ideas.  They were then asked to write an eight-sentence short story for a youth audience on one of three topics. 

The stories were scored by 600 evaluators.  Access to AI increased novelty and usefulness, with those having access to five AI-generated ideas getting the biggest benefit.  AI also improved the emotional characteristics except for how funny the stories were. However, only writers who initially had low creativity (DAT) scores benefited from AI: access to AI made little difference to those assessed as creative.  DAT scores did not appear to influence how many AI-generated ideas the writers used.  As other studies have shown, LLMs help "level up" proficiency.  

The study assessed how similar the stories were within the three groups: human only, access to one AI-generated idea, and access to five AI-generated ideas.  Using one or more AI-generated ideas reduced the diversity of the stories and increased the degree to which the stories "stuck" to the AI idea.  The authors suggested that as more AI-inspired content begins to appear in the training data for AI models, "the collective novelty of stories may be reduced further."  

The writers were also asked to self-assess the novelty, usefulness, and emotional characteristics of their own stories.  I was surprised that having access to AI-generated ideas did not result in any statistically significant improvements.  The implication is that AI can benefit writers but since they do not perceive the benefits, they may not be motivated to further develop their writing skills.  I periodically need to write small python programs, but not often enough to become proficient.  I have successfully used AI to generate blocks of code but struggled to make even minor changes to the code.  I can improve my python skills or become increasingly dependent on AI.  This study suggests the same may be true for writing.  Writing well is hard but worthwhile, summed up by Mike Reed in https://www.raconteur.net/technology/trained-ai-to-write-like-me:

Real writing isn’t about transcribing fully formed ideas. It’s a living process through which you generate, explore and refine those ideas. It’s a way to find new ideas and connections, explore counterpoints, to spot gaps.