Anthropic analyzed 1 million anonymized conversations between users and Claude 3.5 Sonnet. The study found that most people used the model for software development and also revealed malfunctions and jailbreaks.
What’s new: Anthropic built a tool, Clio, to better understand how users interact with its large language models. The system mined anonymized usage data for insights to improve performance and security.
How it works: Clio uses Claude 3.5 Sonnet itself to automatically extract summaries of users’ conversations with the model. Then it clusters related topics. To preserve privacy, it anonymizes and aggregates the data, revealing only information about clusters.
- Clio extracts information from conversations such as the number of turns, the language spoken, and a summary of what was said.
- It embeds the summaries and clusters them according to similarity. This process creates thousands of clusters.
- Given example summaries for each cluster, Clio generates a short description of the type of information in the cluster.
- It repeats the process to create a hierarchy, clustering the descriptions of clusters, generating new descriptions, and so on. For example, clusters with the descriptions “tying knots” and “watering plants” are themselves clustered among “daily life skills.”
Results: Clio uncovered common, uncommon, and disallowed uses of Claude 3.5 Sonnet. It also detected erroneous behavior on the part of the system itself.
- The largest single category was software development. Coding accounted for 15 percent to 25 percent of Claude conversations. Web and mobile app development represented over 10 percent of total conversations, AI and machine learning applications 6 percent, DevOps and cloud infrastructure about 4 percent, and data analysis 3.5 percent.
- Business-related uses came next. Text generation and communication accounted for roughly 9 percent of total conversations, while academic research and writing was over 7 percent. Business strategy and operations accounted for nearly 6 percent.
- Niche uses included serving as dungeon master in the game Dungeons & Dragons, interpreting dreams, solving crossword puzzles, analyzing soccer matches, and preparing for disasters.
- Clio spotted large-scale violations of the company’s usage policy. For instance, a large number of users devised prompts that evaded the safety classifier to use Claude for sexually explicit role-playing.
- It also highlighted flaws in Anthropic’s safety classifier. For instance, it found clusters of conversations that were flagged when they shouldn’t have been or not flagged when they should have been.
Why it matters: Traditional approaches to understanding how people use AI, such as surveys, can yield inaccurate results, since people often don’t report their own actions accurately. Clio offers a method for analyzing real-world usage, much like Google Trends monitors search behavior, without compromising privacy. This sort of approach can help AI builders discover niche use cases, identify flaws, and tailor training and testing data to best serve users.
We’re thinking: We’re all for automated dungeon masters, but we’re glad to see that AI-assisted coding tops the list of real-world uses of Claude!