There is a funny paper at arXiv, that is now published in Neurology. It claims to have found a
neural conundrum behind the slowness of human behavior. The information throughput of a human being is about 10 bits/s. In comparison, our sensory systems gather data at ~10^9 bits/s. The stark contrast between these numbers remains unexplained and touches on fundamental aspects of brain function: What neural substrate sets this speed limit on the pace of our existence? Why does the brain need billions of neurons to process 10 bits/s? Why can we only think about one thing at a time?
If there are really two brains, an “outer” brain with fast high-dimensional sensory and motor signals and an “inner” brain that does are the processing? My inner brain says this is a huge speculation.
I needed this urgently for indexing PDFs as Spotlight on the Mac is highly erratic after all this years.
Anything LLM seemed the most promising approach with an easy to use GUI and being well documented. But indexing failed after several hours, so I went on with LM Studio. Also this installation turned out to be more complicated than expected due to library “dependency hell” and version mismatch spiralling…
Recently, it was proved that the large language model Generative Pre-trained Transformer 4 (GPT-4; OpenAI) can fabricate synthetic medical datasets designed to support false scientific evidence. To uncover statistical patterns that may suggest fabrication in datasets produced by large language models and to improve these synthetic datasets by attempting to remove detectable marks of nonauthenticity, investigating the limits of generative artificial intelligence.
[…] synthetic datasets were produced for 3 fictional clinical studies designed to compare the outcomes of 2 alternative treatments for specific ocular diseases. Synthetic datasets were produced using the default GPT-4o model and a custom GPT. Data fabrication was conducted in November 2024. Prompts were submitted to GPT-4o to produce 12 “unrefined” datasets, which underwent forensic examination. Based on the outcomes of this analysis, the custom GPT Synthetic Data Creator was built with detailed instructions to generate 12 “refined” datasets designed to evade authenticity checks. Then, forensic analysis was repeated on these enhanced datasets. […]
Sufficiently sophisticated custom GPTs can perform complex statistical tasks and may be abused to fabricate synthetic datasets that can pass forensic analysis as authentic.
Quantity Based: One of the continual problems the AI art generation faces is in quantity, though it is continually improving. For instance, in the past, AI art would struggle with getting the correct number of fingers correct, or perhaps the correct placement of knuckles and joints in the fingers.
General Softness & Low Resolution: AI art takes immense computing power to generate, and it still hasn’t streamlined this problem. So often, AI art is limited in resolution and detail.
Repetition: To further expand on the tip above, AI art often uses repetition to help speed up the generation process. So you may see something copied several times over the same image.
Asymmetry: Asymmetry exists in all facets of life, [… if you] photograph the building so that it looks symmetrical across the plane. AI doesn’t understand these rules and often creates subtle symmetry shifts in its images.
Although promised, the supplement with the questions is missing and there is no information in the methods how the authors prompted – so basically useless as methods cannot be replicated.
And at the end, chatGPT is only a mirror of how good or bad these experts informed the public…
I consult the almighty chatGPT frequently for additional information as this saves me hours of wading through my own database, Pubmed, Scholar and Goggle Hits.
But I have my own opinion, I never cut & paste as this is always running at risk (1) to plagiarize unknowingly and (2) to produce nonsense.
In a survey of nearly 5,000 researchers, some 19% said they had already tried using LLMs to ‘increase the speed and ease’ of their review. But the survey, by publisher Wiley, headquartered in Hoboken, New Jersey, didn’t interrogate the balance between using LLMs to touch up prose, and relying on the AI to generate the review.
And well, maybe I am already sticking to the NEJM that said
Although human expert review should continue to be the foundation of the scientific process, LLM feedback could benefit researchers
In this study, we introduced a computational pipeline to analyze citation fidelity on a large scale and developed an automated measure to evaluate how authors report prior findings. By applying this method to 13 million citation sentence pairs, we uncovered several key insights into the dynamics of scholarly communication and show citation fidelity is higher with closer proximity. Through a quasi-causal experiment, we further establish that the “telephone effect” contributes to information loss: when citing papers exhibit low fidelity to the original claim, future papers that cite both the citing paper and the original show even lower fidelity to the original.
Fig4 in Citation fidelity is negatively correlated with the seniority of first authors
Clearly incorrect or misleading information couldn’t be identified in this study although the definition a cut-off should be possible.
And well, I would be also very interested if there are differences by scientific field.
In addition to some programming questions, I have prompted several LLMs for a summary of my longstanding research question “vitamin D supplementation in the newborn period and later allergy”. Here is the elicit response:
Research on vitamin D supplementation in infancy and its relationship to later allergy development has yielded mixed results. Some studies suggest that higher vitamin D intake during the first year of life may increase the risk of atopic conditions later in childhood (Bäck et al., 2009; Hyppönen et al., 2004). These findings are supported by animal studies and epidemiological surveys linking early vitamin D supplementation to allergy development (Wjst, 2006). However, a randomized controlled trial found that high-dose vitamin D supplementation did not prevent allergic sensitization or diseases in infants compared to standard dosage, except for an increased risk of milk allergy in the high-dose group (Rosendahl et al., 2019). Interestingly, this study also observed that infants with high cord blood vitamin D levels had a higher risk of food allergen sensitization. These conflicting results highlight the complex relationship between vitamin D and allergy development, suggesting that both insufficient and excessive vitamin D levels may potentially influence allergic outcomes.
The limitation introduced by “however” is wrong – the whole story is not really conflicting if you read my earlier paper on a paradoxical reaction. I have been a reviewer of the Rosendahl paper and asked them to change their misleading spin. It is basically a triple blind study (the authors believed the opposite fact eg that vitamin D supplementation prevents while it induces allergy). Unfortunately, as Rosendahl et al. did not respond to my complaint, we can now clearly see how the bias of the authors is being injected into the LLM.
I dont know why there is still the discussion of any genuine understanding of LLMs as there is none, see Binz et al.
even though LLMs often achieve state-of-the-art performance on existing benchmarks, it remains debated whether this reflects genuine understanding, or whether they are merely acting like stochastic parrots […] When deciding whether to use an LLM, it is important to recognize that LLMs are simply models of word form distributions extracted from text—not models of the information that people might get from reading that text
Anyway, the workflow suggested by inkenai – putting these PDFs now into NotebookML for further analysis – is excellent.
Nachdem Sicherheitsforscher auf dem 38. Chaos Communication Congress gravierende Mängel bei der elektronischen Patientenakte (ePA) für gesetzliche Versicherte gefunden haben, fordert der Chef der Bundesärztekammer, Klaus Reinhardt, rasche Nachbesserung. Er könne die ePA 3.0 nach aktuellem Stand nicht empfehlen. Dennoch sei das keine Aufforderung zum Opt-out. Der Verband der Kinder- und Jugendärzt:innen (BVKJ) rät Eltern hingegen, für deren Kinder Widerspruch einzulegen. Das berichten das Ärzteblatt und die Ärztezeitung.
Ärzte unterliegen der Schweigepflicht und gehören zu den Berufsgeheimnisträgern [2]. Dass ärztliche Unterlagen und Aufzeichnungen über Patienten nicht einfach beschlagnahmt werden können, wird in der Strafprozessordnung (StPO) in § 97 Beschlagnahmeverbote [3] geregelt. Voraussetzung ist, dass sich zu beschlagnahmende Gegenstände “im Gewahrsam der zur Verweigerung des Zeugnisses Berechtigten” befinden. Da sich die elektronische Gesundheitskarte nicht im Gewahrsam des Arztes, sondern im Gewahrsam des Patienten befindet …
I tried out chatGPT 4o to create the R ggplot2 code for a professional color chart
v1v20
ChatGPT had serious problems to recognize even the grid fields while it was impossible to get the right colors or any order after more than a dozen attempts (I created the above chart in less than 15m).
At the end, chatGPT ended with something like a bad copy of Gerhard Richters “4900 Colours”…
Although labeled as generative, AI is not generative in a linguistic sense that
… aims to explain the cognitive basis of language by formulating and testing explicit models of humans’ subconscious grammatical knowledge
I would like to call it better imitating AI. ChatGPT never got the idea of a professional color chart for optimizing color workflow from camera to print).
It was also lacking any aesthetics. Although the Richter squares are arranged randomly, they form a luminous grid pattern with overwhelming kaleidoscopic color fields.
A new paper from Bristol discusses the recent explosion of low-quality two-sample Mendelian randomization studies and offers a cure.
We advise editors to simply reject papers that only report 2SMR findings, with no additional supporting evidence. For reviewers receiving such papers, we provide a template for rejection.
I changed several things from the last version, basically switching to a new layout and going down from 100fps to 60fps as YT can handle this much better.
Just in case, somebody wants to modify it, here is the script.
vid <- function(nr){
nr2 = as.integer(nr*60) # total number 1/2s
nr3 = -600 + (nr*10) # current ms title
for (ii in 1:60 ){
fn = paste0(str_pad(nr*60+ii,5,pad = "0"), ".png")
png(file = fn, width = 1600, height = 900, units = 'px') # defaults to 300 ppi
par(mar=c(0,0,0,0),bg="black")
plot(c(0, 1), c(0, 1), ann = F, bty = 'n', type = 'n', xaxt = 'n', yaxt = 'n', asp=1)
color="red"
rect(xleft=0.5, xright=(nr3+500)/1000, ybottom=0.94, ytop=0.99, col= color)
color="lightgrey"
if( (nr<=58 & ii==30+nr/2) | (nr>=60 & ii==-30+nr/2) ) {
circlize::draw.sector(0, 360, center = c(0.02, 0.01), rou1 = 0.01, col = color, border = color)
}
circlize::draw.sector(90, 90-ii*6, center = c(0.5, 0.52), rou1 = 0.4, col = color, border = color)
if (ii<3 | ii>57) {
color="white"
circlize::draw.sector(0, 360, center = c(0.5, 0.52), rou1 = 0.4, col = color, border = color)
}
tx=paste(nr3,' ms')
text(x = 0.5, y = 0.85, tx, cex = 6, col = "white", family="Lato", adj=0.5)
tx=paste0(nr/2,':',str_pad( round(100*ii/60), 2, pad = "0"))
text(x = 0.5, y = 0.5, tx, cex = 12, col = "white", family="Lato", adj=0.5)
tx = "play until beep & flash in sync OR take image of source and processed video"
text(x = 0.5, y = 0.075, tx, cex = 3, col = "grey", family="Lato", adj=0.5)
par(bg="white")
dev.off()
}
}
for (i in seq(0,120,2) ) {
vid(i)
}
fn = paste0(list.files('*.png'))
av::av_encode_video(fn, framerate = 60, output = 'video.mp4')
Being a regular Scholar user, I am quite lost now with the many new scientific search engines. They don’ tell us which data they used for training, how they have been trained and how the results have been validated. The field is also highly dynamic when compared to the situation 2 years ago. Is it worth to test them?
The common theme seems the low certainty about facts – a historical event that is wrongly memorized by a human or the Large Language Model that wrongly extrapolates from otherwise secure knowledge. But is there even more?
“Large language models have no idea of the underlying reality that language describes,” he said, adding that most human knowledge is nonlinguistic. “Those systems generate text that sounds fine, grammatically, semantically, but they don’t really have some sort of objective other than just satisfying statistical consistency with the prompt.”
Humans operate on a lot of knowledge that is never written down, such as customs, beliefs, or practices within a community that are acquired through observation or experience. And a skilled craftsperson may have tacit knowledge of their craft that is never written down.
I think “hallucination” is way too much an anthropomorphic concept – some LLM output is basically statistical nonsense (although I wouldn’t go as far as Michael Townsen Hicks…). Reasons for these kind of errors are manifold -reference divergence may be already in the data used for learning – data created by bots, conspiracy followers or even fraud science. The error may also originate from encoding or decoding routines.
I couldn’t find any further analogy with wrong human memory recall except the possibility that also human memory is influenced by probability as well. Otgar 2022 cites Calado 2020
The issue of whether repeated events can be implanted in memory has recently been addressed by Calado and colleagues (2020). In their experiment, they falsely told adult participants that they lost their cuddling toy several times while control participants were told that they only lost it once. Strikingly, they found that repeated false events were as easily inserted in memory as suggesting that the event happened once. So, this study not only showed that repeated events can be implanted, it raised doubts about the idea that repeated events might be harder to implant than single events
