Skip interactive to article
Explanation Open separately

An Atlas of AI Tells

Created September 20, 2026

2026   ·   ai   language

What is this?

This is a map of AI-generated texts, which are arranged and labeled according to a number of indicators that I identified using an AI-based data-mining procedure. You can browse through the map and hover over the texts to see which formulations, individual words, or constructs are particularly indicative of text generated by various AI models, and familiarize yourself with the signs of AI writing.

Why did I make this?

It’s important for me to know whether the text I’m reading is AI-generated. As with human-authored texts, knowing the author drastically changes how I understand it and what I am willing to learn from it.

Luckily, tools to determine whether a text was generated with AI, like Pangram, or the recent open-source MELD (Li et al., 2026) have improved quite a bit. I would not use them in scenarios in which I get sued if I’m wrong, but I think they do provide pretty good evidence of AI-generated text, even though getting around them is not impossible with some dedication. But right now, the good detectors are quite expensive or cumbersome to run, so how to decide whether to put something into them? There are a number of lists going around outlining tells of AI writing. This project is an attempt to find a uniform process to mine such tells in an automated way and build an interface that allows people to educate themselves about them in an interactive way.

How do we get these indicators?

The whole process is inspired by this recent paper, which introduces RLIE, by Yang, Xu, Hu, and Yue (2026). The basic idea behind that paper is that you can use the capabilities and contextual information of AI models to build good text classifiers.

To do so, you put the AI into a loop, in which it proposes textual features that might be useful for classification. Then, another instance of the AI model grades texts according to those proposed features, and the resulting annotations are used to fit a regularized logistic regression model, which tries to predict the outcome class and weights badly performing features down. These get then dropped, and samples of the texts on which the model performed badly get fed back to the AI model. Based on analyzing them, it proposes new features and the cycle repeats until performance stops improving.

For this project, I’ve adapted the process in several ways. One of them is that I’ve replaced the AI-model-based grading. Instead of coming up with rules that are then identified via an LLM, the language model immediately suggests rules that are formalized linguistic patterns or general linguistic attributes. This is inspired by the GRASP algorithm, which uses greedy search for linguistic pattern matching (Shnarch et al., 2017; Lertvittayakumjorn et al., 2022). These rules might be quite simple, for example that a specific word or lemma appears (e.g. “genuinely”) within the text. But they can also be quite complex, e.g. having a participle-ing form and a subordinate clause introduced with “while”, alongside constraints for the right order in the sentence. This captures a while construct very common to AI writing, e.g.: “…the algorithmic moment vindicates Adorno’s structural claim while breaking his specific model in ways that matter.” (Opus dataset)

This rule-based approach has the advantage that it is computationally inexpensive and very fast to check the rules against the whole training set, so that the LLM doesn’t have to repeatedly “hand-label” thousands of texts. This also allows me to use a quite strong LLM, OpenAI’s Astra in high reasoning mode, for 98 calls, for the development of the underlying rules.

I’ve also set it up so that only positive rules for the identification of AI are admissible (the model is only detecting AI), not negative rules that are sensitive to human writing. This also changes the way the elastic logistic regression has to be fitted.

The whole system is then trained on a data mix I’ve put together, which contains both human and AI writing. In particular, I have included the evaluation data from MELD, which consists of a number of pieces of human writing that are rewritten by AI, the Human-AI Parallel corpus (Reinhart et al., 2025), which has an LLM continue human-written text and compares it to the actual human continuation of the text, as well as a number of smaller chat datasets that I have patched together from Hugging Face that capture more recent and stronger models, including GPT-5 variants and Claude Opus, for a total of 25,038 documents (64 % train, 16 % validation, 20 % evaluation).

Below is a diagram of the data composition:

Composition of 25,038 training and development documents. HAP-E and MELD supply most of the data, alongside smaller GPT-5 Chat, Opus, ShareChat, and Fable datasets. Column widths show source sizes; shading distinguishes AI and human text.
Source datasets used for training and development. Click the figure to view it at full size.

I must note here that the classifier created by the system is very likely worse than any of the transformer-based classification AI detection systems that are currently on the market. On a held-out eval set, it reaches about 79.4% accuracy, classifying four in five documents correctly, and an AUC of 0.86, scoring an AI document above a human one 86% of the time.

This unreliable performance is not surprising, as it is dependent on only 100 simple rules instead of a holistic assessment of the whole text and the relationship of all kinds of words within it. This project consequently isn’t an attempt to build an AI detector, but one to come up with simple heuristics to help us become more aware of AI-generated content around us.

To make it easier to explore those, I split the documents into short passages, and mapped them out with UMAP based on the rules that were triggered by them. So you can explore the as-clauses at the top of the map, “merely”, “palpable”, “It’s not just X, it’s Y” towards the left-hand side, the em-dashes at the bottom, and “ensure” and “ensuring” on the right. You can also click on individual points to explore them in more depth, highlight only specific rules on the map, and color it by model or data source.

What else have I tried?

Before settling on this RLIE-based pipeline, I tried a number of other approaches to get at these rules. First, I tried to use various AI interpretability tools on the MELD (transformer-based) classifier. I suspect this failed because the underlying transformer model seems to evaluate the text in such a heuristic way that it’s hard to decompose the effects of individual attributes of the texts out of the classifier. MELD does label individual tokens, but the individual labels don’t seem to correspond to recognizable patterns, as far as I could tell. I also tried running GRASP directly on the human/AI sample. This wasn’t successful, I think, because the greedy search of GRASP had trouble getting to more complicated rules. Finally, I tried using Monte Carlo tree search as an alternative to GRASP in the mining of rules, but found that it often leads to very repetitive rules, with only minor variations of the same rule reoccurring. Although the classification performance was comparable to the current model, the resulting rules were hard to read and didn’t seem very useful. I think the LLM-generated rules from the RLIE-inspired approach are often easier to understand because they are already generated via a process that makes sense semantically.

Disclosure of AI use: As weekend entertainment, this project is largely coded using AI (OpenAI’s Astra model and Anthropic’s Claude Opus 5). The explanatory text was artisanally hand-typed by myself, and edited with AI (fixed typos, formatted citations, etc.).

References

Datasets