Category Archives: Software

Are we really thinking at 10 bits/s?

There is a funny paper at arXiv, that is now published in Neurology. It claims to have found a

neural conundrum behind the slowness of human behavior. The information throughput of a human being is about 10 bits/s. In comparison, our sensory systems gather data at ~10^9 bits/s. The stark contrast between these numbers remains unexplained and touches on fundamental aspects of brain function: What neural substrate sets this speed limit on the pace of our existence? Why does the brain need billions of neurons to process 10 bits/s? Why can we only think about one thing at a time?

If there are really two brains, an “outer” brain with fast high-dimensional sensory and motor signals and an “inner” brain that does are the processing?  My inner brain says this is a huge speculation.

CC-BY-NC

How to run LLaMA on your local PDFs

I needed this urgently for indexing PDFs as Spotlight on the Mac is highly erratic after all this years.

Anything LLM seemed the most promising approach with an easy to use GUI and being well documented. But indexing failed after several hours, so I went on with LM Studio. Also this installation turned out to be more complicated than expected due to library “dependency hell” and version mismatch spiralling…

  1. Download and install LM Studio
  2. From inside LM Studio download your preferred model
  3. Index your PDFs in batches of 1,000 using the Python script below
  4. Combine indices and run queries against the full index

30.000 PDFs result in a 4G index while the system is unfortunately not very responsive (yet)

 

Continue reading How to run LLaMA on your local PDFs

CC-BY-NC

Fighting AI with AI

Here is our newest paper – a nice collaboration with Andrea Taloni et al.  along with a nice commentary – to recognize surgisphere-like fraud

Recently, it was proved that the large language model Generative Pre-trained Transformer 4 (GPT-4; OpenAI) can fabricate synthetic medical datasets designed to support false scientific evidence. To uncover statistical patterns that may suggest fabrication in datasets produced by large language models and to improve these synthetic datasets by attempting to remove detectable marks of nonauthenticity, investigating the limits of generative artificial intelligence.

[…] synthetic datasets were produced for 3 fictional clinical studies designed to compare the outcomes of 2 alternative treatments for specific ocular diseases. Synthetic datasets were produced using the default GPT-4o model and a custom GPT. Data fabrication was conducted in November 2024. Prompts were submitted to GPT-4o to produce 12 “unrefined” datasets, which underwent forensic examination. Based on the outcomes of this analysis, the custom GPT Synthetic Data Creator was built with detailed instructions to generate 12 “refined” datasets designed to evade authenticity checks. Then, forensic analysis was repeated on these enhanced datasets.  […]

Sufficiently sophisticated custom GPTs can perform complex statistical tasks and may be abused to fabricate synthetic datasets that can pass forensic analysis as authentic.

 

CC-BY-NC

How to recognize an AI image

Lensrental has some great advice

Quantity Based: One of the continual problems the AI art generation faces is in quantity, though it is continually improving. For instance, in the past, AI art would struggle with getting the correct number of fingers correct, or perhaps the correct placement of knuckles and joints in the fingers.

General Softness & Low Resolution: AI art takes immense computing power to generate, and it still hasn’t streamlined this problem. So often, AI art is limited in resolution and detail.

Repetition: To further expand on the tip above, AI art often uses repetition to help speed up the generation process. So you may see something copied several times over the same image.

Asymmetry: Asymmetry exists in all facets of life,  [… if you] photograph the building so that it looks symmetrical across the plane. AI doesn’t understand these rules and often creates subtle symmetry shifts in its images.

TBC

CC-BY-NC

A useless study of AI accuracy

https://www.jaci-inpractice.org/article/S2213-2198(25)00280-6/pdf

Although promised, the  supplement with the questions is missing and there is no information in the methods how the authors prompted – so basically useless as methods cannot be replicated.

And at the end, chatGPT is only a mirror of how good or bad these experts informed the public…

CC-BY-NC

Is it a crime to use AI for peer review?

I consult the almighty chatGPT frequently for additional information as this saves me hours of wading through my own database, Pubmed, Scholar and Goggle Hits.

But I have my own opinion, I never cut & paste as this is always running at risk (1) to plagiarize unknowingly and (2) to produce nonsense.

Miryam Naddaf has an article about this

In a survey of nearly 5,000 researchers, some 19% said they had already tried using LLMs to ‘increase the speed and ease’ of their review. But the survey, by publisher Wiley, headquartered in Hoboken, New Jersey, didn’t interrogate the balance between using LLMs to touch up prose, and relying on the AI to generate the review.

And well, maybe I am already sticking to the NEJM that said

Although human expert review should continue to be the foundation of the scientific process, LLM feedback could benefit researchers

CC-BY-NC

Citation quality

There is a super interesting study on citation quality in scientific studies.

In this study, we introduced a computational pipeline to analyze citation fidelity on a large scale and developed an automated measure to evaluate how authors report prior findings. By applying this method to 13 million citation sentence pairs, we uncovered several key insights into the dynamics of scholarly communication and show citation fidelity is higher with closer proximity. Through a quasi-causal experiment, we further establish that the “telephone effect” contributes to information loss: when citing papers exhibit low fidelity to the original claim, future papers that cite both the citing paper and the original show even lower fidelity to the original.

Fig4 in Citation fidelity is negatively correlated with the seniority of first authors

Clearly incorrect or misleading information couldn’t be identified in this study although the definition a cut-off should be possible.

And well, I would be also very interested if there are differences by scientific field.

CC-BY-NC

Elicit

In addition to some programming questions, I have prompted several LLMs for a summary of  my longstanding research question “vitamin D supplementation in the newborn period and later allergy”. Here is the elicit response:

Research on vitamin D supplementation in infancy and its relationship to later allergy development has yielded mixed results. Some studies suggest that higher vitamin D intake during the first year of life may increase the risk of atopic conditions later in childhood (Bäck et al., 2009; Hyppönen et al., 2004). These findings are supported by animal studies and epidemiological surveys linking early vitamin D supplementation to allergy development (Wjst, 2006). However, a randomized controlled trial found that high-dose vitamin D supplementation did not prevent allergic sensitization or diseases in infants compared to standard dosage, except for an increased risk of milk allergy in the high-dose group (Rosendahl et al., 2019). Interestingly, this study also observed that infants with high cord blood vitamin D levels had a higher risk of food allergen sensitization. These conflicting results highlight the complex relationship between vitamin D and allergy development, suggesting that both insufficient and excessive vitamin D levels may potentially influence allergic outcomes.

The limitation introduced by “however” is wrong – the whole story is not really conflicting  if you read my earlier paper on a paradoxical reaction. I have been a reviewer of the Rosendahl paper and asked them to change their misleading spin. It is basically a triple blind study (the authors believed the opposite fact eg that vitamin D supplementation prevents while it induces allergy).  Unfortunately, as Rosendahl et al.  did not respond to my complaint, we can now clearly see how the bias of the authors is being injected into the LLM.

I dont know why there is still the discussion of any genuine understanding of LLMs as  there is none, see Binz et al.

even though LLMs often achieve state-of-the-art performance on existing benchmarks, it remains debated whether this reflects genuine understanding, or whether they are merely acting like stochastic parrots […] When deciding whether to use an LLM, it is important to recognize that LLMs are simply models of word form distributions extracted from text—not models of the information that people might get from reading that text

Anyway, the workflow suggested by inkenai – putting these PDFs now into NotebookML for further analysis – is excellent.

CC-BY-NC

Die elektronische Patientenakte

“das Narrativ der sicheren elektronischen Patientenakte ist nicht mehr zu halten” so der CCC2024.

oder heise.de

Nachdem Sicherheitsforscher auf dem 38. Chaos Communication Congress gravierende Mängel bei der elektronischen Patientenakte (ePA) für gesetzliche Versicherte gefunden haben, fordert der Chef der Bundesärztekammer, Klaus Reinhardt, rasche Nachbesserung. Er könne die ePA 3.0 nach aktuellem Stand nicht empfehlen. Dennoch sei das keine Aufforderung zum Opt-out. Der Verband der Kinder- und Jugendärzt:innen (BVKJ) rät Eltern hingegen, für deren Kinder Widerspruch einzulegen. Das berichten das Ärzteblatt und die Ärztezeitung.

und nochmal heise.de

Ärzte unterliegen der Schweigepflicht und gehören zu den Berufsgeheimnisträgern [2]. Dass ärztliche Unterlagen und Aufzeichnungen über Patienten nicht einfach beschlagnahmt werden können, wird in der Strafprozessordnung (StPO) in § 97 Beschlagnahmeverbote [3] geregelt. Voraussetzung ist, dass sich zu beschlagnahmende Gegenstände “im Gewahrsam der zur Verweigerung des Zeugnisses Berechtigten” befinden. Da sich die elektronische Gesundheitskarte nicht im Gewahrsam des Arztes, sondern im Gewahrsam des Patienten befindet …

kann wohl auch der Staat darauf zugreifen.

CC-BY-NC

AI lobotomizing knowledge

I tried out chatGPT 4o to create the R ggplot2 code for a professional color chart

v1
v20

ChatGPT had serious problems to recognize even the grid fields while it was impossible to get the right colors or any order after more than a dozen attempts (I created the above chart in less than 15m).

At the end, chatGPT ended with something like a bad copy of Gerhard Richters “4900 Colours”…

https://www.hatjecantz.de/products/16130-gerhard-richter

Why was this task so difficult?

Although labeled as generative, AI is not generative in a linguistic sense that

… aims to explain the cognitive basis of language by formulating and testing explicit models of humans’ subconscious grammatical knowledge

I would like to call it better imitating AI. ChatGPT never got the idea of a professional color chart for optimizing color workflow from camera to print).

It was also lacking any aesthetics. Although the Richter squares are arranged randomly, they form a luminous grid pattern with overwhelming kaleidoscopic color fields.

A less academic version – it is the biggest copyright infringement ever since Kim Dotcom.

TBC

CC-BY-NC

Update on Mendelian Randomization

As written before I never published any study that included a Mendelian randomization. The reasons are well known.

A new paper from Bristol discusses  the  recent explosion of low-quality two-sample Mendelian randomization studies and offers a cure.

We advise editors to simply reject papers that only report 2SMR findings, with no additional supporting evidence. For reviewers receiving such papers, we provide a template for rejection.

CC-BY-NC

Audio Video Latency Test

Here is new  video test file build with R.

I changed several things from the last version, basically switching to a new layout and going down from 100fps to 60fps as YT can handle this much better.

 

 

Just in case, somebody wants to modify it, here is the script.

vid <- function(nr){
	nr2 = as.integer(nr*60) # total number 1/2s
	nr3 = -600 + (nr*10) # current ms title
	for (ii in 1:60 ){
		fn = paste0(str_pad(nr*60+ii,5,pad = "0"), ".png")
		png(file =  fn, width = 1600, height = 900, units = 'px') # defaults to 300 ppi
		par(mar=c(0,0,0,0),bg="black")	
		plot(c(0, 1), c(0, 1), ann = F, bty = 'n', type = 'n', xaxt = 'n', yaxt = 'n', asp=1)
		color="red"
		rect(xleft=0.5, xright=(nr3+500)/1000, ybottom=0.94, ytop=0.99, col= color)
		color="lightgrey"
		if( (nr<=58 & ii==30+nr/2) | (nr>=60 & ii==-30+nr/2) ) {
			circlize::draw.sector(0, 360, center = c(0.02, 0.01), rou1 = 0.01, col = color, border = color)		
		}	
		circlize::draw.sector(90, 90-ii*6, center = c(0.5, 0.52), rou1 = 0.4, col = color, border = color)
		if (ii<3 | ii>57) {
			color="white"
			circlize::draw.sector(0, 360, center = c(0.5, 0.52), rou1 = 0.4, col = color, border = color)
		}
		tx=paste(nr3,' ms')
		text(x = 0.5, y = 0.85, tx, cex = 6, col = "white", family="Lato", adj=0.5)
		tx=paste0(nr/2,':',str_pad( round(100*ii/60), 2, pad = "0"))
		text(x = 0.5, y = 0.5, tx, cex = 12, col = "white", family="Lato", adj=0.5)
		tx = "play until beep & flash in sync OR take image of source and processed video"
		text(x = 0.5, y = 0.075, tx, cex = 3, col = "grey", family="Lato", adj=0.5)
		par(bg="white")
		dev.off()
	}	
}
for (i in seq(0,120,2) ) {
	vid(i)
}
fn = paste0(list.files('*.png'))
av::av_encode_video(fn, framerate = 60, output = 'video.mp4')
CC-BY-NC

Too many AI powered scientific search engines

Being a regular Scholar user, I am quite lost now with the many new scientific search engines. They don’ tell us which data they used for training, how they have been trained and how the results have been validated. The field is also highly dynamic when compared to the situation 2 years ago. Is it worth to test them?

https://www.undermind.ai/home/

Continue reading Too many AI powered scientific search engines

CC-BY-NC

Similarity between false memory (of humans) and hallucination( of LLMs)

The common theme seems the low certainty about facts – a historical event that is wrongly memorized by a human or the Large Language Model that wrongly extrapolates from otherwise secure knowledge. But is there even more?

Yann Le Cun is being quoted at IEEE Spectrum

“Large language models have no idea of the underlying reality that language describes,” he said, adding that most human knowledge is nonlinguistic. “Those systems generate text that sounds fine, grammatically, semantically, but they don’t really have some sort of objective other than just satisfying statistical consistency with the prompt.”
Humans operate on a lot of knowledge that is never written down, such as customs, beliefs, or practices within a community that are acquired through observation or experience. And a skilled craftsperson may have tacit knowledge of their craft that is never written down.

I think “hallucination” is way too much an anthropomorphic concept – some LLM output is basically statistical nonsense (although I wouldn’t go as far as  Michael Townsen Hicks…). Reasons for these kind of errors are manifold -reference divergence may be already in the data used for learning – data created by bots, conspiracy followers or even fraud science. The error may also originate from encoding or decoding routines.

I couldn’t find any further analogy with wrong human memory recall except the possibility that also human memory is influenced by  probability as well. Otgar 2022 cites Calado 2020

The issue of whether repeated events can be implanted in memory has recently been addressed by Calado and colleagues (2020). In their experiment, they falsely told adult participants that they lost their cuddling toy several times while control participants were told that they only lost it once. Strikingly, they found that repeated false events were as easily inserted in memory as suggesting that the event happened once. So, this study not only showed that repeated events can be implanted, it raised doubts about the idea that repeated events might be harder to implant than single events

 

CC-BY-NC