{"id":23419,"date":"2024-03-13T08:03:16","date_gmt":"2024-03-13T06:03:16","guid":{"rendered":"https:\/\/www.wjst.de\/blog\/?p=23419"},"modified":"2024-03-13T08:04:47","modified_gmt":"2024-03-13T06:04:47","slug":"how-does-ai-recognize-ai-text","status":"publish","type":"post","link":"https:\/\/www.wjst.de\/blog\/sciencesurf\/2024\/03\/how-does-ai-recognize-ai-text\/","title":{"rendered":"How does AI recognize AI text"},"content":{"rendered":"

The Semrush blog<\/a> has a nice summary<\/p>\n

By analyzing two main characteristics of the text:\u00a0perplexity\u00a0and\u00a0burstiness<\/a>. In other words, how predictable or unpredictable it sounds to the reader, as well as how varied or uniform the sentences are.<\/p><\/blockquote>\n

Perplexity is<\/a><\/p>\n

a statistical measure of how confidently a language model predicts a text sample. In other words, it quantifies how \u201csurprised\u201d the model is when it sees new data. The lower the perplexity, the better the model predicts the text.<\/p><\/blockquote>\n

Burstiness is<\/a><\/p>\n

is the intermittent increases and decreases in activity or frequency of an event. One of measures of burstiness is the Fano factor \u2014a ratio between the variance and mean of counts. In natural language processing, burstiness has a slightly more specific definition… A word is more likely to occur again in a document if it has already appeared in the document. Importantly, the burstiness of a word and its semantic content are positively correlated; words that are more informative are also more bursty.<\/p><\/blockquote>\n

Or lets call it entropy?<\/a>\u00a0So we now have some criteria<\/p>\n