{"id":25641,"date":"2025-10-07T14:44:00","date_gmt":"2025-10-07T12:44:00","guid":{"rendered":"https:\/\/www.wjst.de\/blog\/?p=25641"},"modified":"2025-11-21T07:29:33","modified_gmt":"2025-11-21T05:29:33","slug":"is-there-a-seahorse-emoji","status":"publish","type":"post","link":"https:\/\/www.wjst.de\/blog\/sciencesurf\/2025\/10\/is-there-a-seahorse-emoji\/","title":{"rendered":"Is there a seahorse emoji?"},"content":{"rendered":"

Ask this question chatGPT<\/a> – and it starts rattling like a shaken pinball machine…<\/p>\n

\"\"<\/a>
first discovered by https:\/\/x.com\/voooooogel\/status\/1964465679647887838<\/figcaption><\/figure>\n

Here is an \u00a0explanation of that phenomenon by Brian Huang<\/a><\/p>\n

if the model wants to output the word “hello”, it needs to construct a residual similar to the vector for the “hello” output token that the lm_head can turn into the hello token id. and if the model wants to output a seahorse emoji, it needs to construct a residual similar to the vector for the seahorse emoji output token(s) – which in theory could be any arbitrary value, but in practice is seahorse + emoji in word2vec style.
\nThe only problem is the seahorse emoji doesn’t exist! So when this seahorse + emoji residual hits the lm_head, it does its dot product over all the vectors, and the sampler picks the closest token – a fish emoji.<\/p><\/blockquote>\n

For an even longer version see [here<\/a>].<\/p>\n

Bonus 1 \u00a0– here is a my seahorse image taken at the Mus\u00e9e oc\u00e9anographique de Monaco last week from its wonderful collection. Let\u2018s forget the virtual world and preserve the real one.<\/p>\n

\"\"<\/a>
image taken at the Mus\u00e9e oc\u00e9anographique de Monaco (c) 2025.<\/figcaption><\/figure>\n

Bonus 2 – the answer to a long-standing question: The origin of male seahorses\u2019 brood pouch<\/a>!<\/p>\n\n

 <\/p>\n

\n CC-BY-NC Science Surf , accessed 26.04.2026<\/span>\n <\/div>","protected":false},"excerpt":{"rendered":"

Ask this question chatGPT – and it starts rattling like a shaken pinball machine… Here is an \u00a0explanation of that phenomenon by Brian Huang if the model wants to output the word “hello”, it needs to construct a residual similar to the vector for the “hello” output token that the lm_head can turn into the … Continue reading Is there a seahorse emoji?<\/span> →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[3358],"class_list":["post-25641","post","type-post","status-publish","format-standard","hentry","category-joke-fun","tag-ai"],"_links":{"self":[{"href":"https:\/\/www.wjst.de\/blog\/wp-json\/wp\/v2\/posts\/25641","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.wjst.de\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.wjst.de\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.wjst.de\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.wjst.de\/blog\/wp-json\/wp\/v2\/comments?post=25641"}],"version-history":[{"count":8,"href":"https:\/\/www.wjst.de\/blog\/wp-json\/wp\/v2\/posts\/25641\/revisions"}],"predecessor-version":[{"id":26010,"href":"https:\/\/www.wjst.de\/blog\/wp-json\/wp\/v2\/posts\/25641\/revisions\/26010"}],"wp:attachment":[{"href":"https:\/\/www.wjst.de\/blog\/wp-json\/wp\/v2\/media?parent=25641"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.wjst.de\/blog\/wp-json\/wp\/v2\/categories?post=25641"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.wjst.de\/blog\/wp-json\/wp\/v2\/tags?post=25641"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}