Category Archives: Software

FAQ: From HDMI to NDI

23.12.2023 admin

1. What is all about?

Starting with Corona I have been streaming lectures and church concerts. Using a Macbook and a Linux laptop, old iPhones and Nikon DSLR cameras were connected by HDMI cables to a Blackmagic ATEM mini pro. This worked well in the beginning although there are many shortcomings of HDMI as it is basically a protocol to connect just screens to a computer and not devices in a large nave.

cables are expensive and there are several connector types (A=Standard, B=Dual, C=Mini, D=Micro, E=Automotive) where the right length and type is always missing
it never worked for me more than 15-20m distance even with amplifier inserted
the signal was never 100% stable, it was lost it in the middle of the performance
HDMI is only unidirectional, there is no tally light signal to the camera
there is no PTZ control for camera movement

2. What are the options?

WIFI transmission would be nice but is probably not the first choice for video transmission in a crowded space with even considerable latency in the range of 200ms. SDI is an IP industry standard for video but this would require dedicated and expensive cabling for each video sources including expensive transceivers. The NDI protocol (network device interface) can use existing ethernet networks and WIFI to transmit video and audio signals. NDI enabled devices started slowly due to copyright and license issues but is expected to be a future market leader due to its high performance and low latency.

3. Is there any low-cost but high quality NDI solution?

NDI producing videocameras with PTZ (pan(tilt/zoom) movements are expensive in the range of 1,000-20,000€. But there are NDI encoder for existing hardware like DSLRs or mobile phones. These encoders are sometimes difficult to find, I am listing below what I have been used so far. Whenever a signal is live, it can be easily displayed and organized by Open Broadcaster Software running on MacOS, Linux or Windows. There are even apps for iOS and Android that can send and receive NDI data replacing now whole broadcasting vehicles ;-)

4. What do I need to buy?

Assuming that you already have some camera equipment that can produce clean HDMI out (Android needs an USB to HDMI and iPhones a Lightning to HDMI cable), you will need

a few cheap CAT6 cables at different lengths (superfluous if you just want a WIFI solution)
an industrial router with SIM card slots (satellite transmission is still out of reach for semi-professional transmission ;-(
one or more HDMI to NDI transceiver
an additional PTZ camera is not required at the beginning

5. Which devices did you test and which do you currently use?

router: I tested AVM FritzBox , TP Link and various Netgear devices all without RJ45 network ports. I am using now a Teltonika RUT950 with three RJ45 ports as it has great connectivity and a super detailed configuration menu.
NDI transceiver:I tried a DIY solution with FFMPEG/Ubuntu, then a Kiloview P2 and a LINk Pi ENC2. I am now using Zowietek 4K HDMI which is giving a stable signal, being fully configurable, silent and can be powered by USB port or PoE.
PTZ: so far I used a Logitech Pro2, but will replace it with an OBSBOT Tail Air

The Teltonika router and the two Zowietek converter cost you less than 500€, the Obsbot also comes at less than 500€ while this setup allows for semi-professional grade live streams.

6. Tell me a bit more about your DSLR cameras and the iPhones please.

There is nothing particular, some old Nikon D4s and a Z6, all with good glass and an outdated iPhone SE without SIM card.

7. Have you ever tried live streaming directly from a camera like the Insta 360 X3?

No.

8. What computer hardware and software do you use for streaming and does this integrate PTZ control?

I used a 2017 and a 2020 Macbook (which showed ab advantage over a Linux notebook when directly connecting the DSLR). NDI Video has to be delayed due to the latency of other NDI remote sources. I usually sync sound and different video sources at +400ms in OBS.
Right now I am testing an iPad app for wireless management called TopDirector. The app looks promising but haven’t tested it in the wild so far.
PTZ control can be managed by an OBS plugin, while TopDirector has it already built in.

9. How much setup time do you need?

Setting up 2 DSLR cameras, 1 phone and 1 audio recorder on tripods and laying cables takes 30-40 minutes . OBS configuration and Youtube setup takes another 10 minutes if everything goes well.

Software

Macbook touchbar flickering

21.12.2023 admin

The touchbar is a nice feature of the Macbook when used as multimedia machine because it could be individually programmed. Unfortunately it has been abandoned may for reasons unknown to us. At least it starts now flickering at random intervals in my 2019 Macbook Pro. And nothing helps in the long-term

resetting SMC
restting NVRAM, PRAM
terminal kill touchbar
terminal kill control strip
pmset hibernate mode
upgrade to Sonoma

while only kill touchbar disables it immediately . Could it be a combined hardware / software issue?

a slowly expanding battery moving the keyboard lid?
some defect of the light sensor?
a defect when starting the OLED display?

While I can’t fully exclude a minimal battery expansion after 200 cycles, the battery is still marked as OK in the system report. The flickering can be stopped by a bright light at the camera hole on top of the display so the second option is also unlikely.

With the Medium hack it is gone during daytime but still occurs sometimes during sleep which is annoying…

Completely disabling the touchbar is not possible (due to the ESC key), so the touchbar may need replacement as recommended by Apple. I am still exploring some other options eg improving the Medium hack with no result so far.

Noteworthy, Software

Can ChatGPT generate a RCT dataset that isn’t recognized by forensic experts?

9.12.2023 admin

“Free synthetic data”? There are numerous Google ads selling synthetic aka fake data. How “good” are these datasets? Will they ever been used for scientific publications outside the AI field eg surgisphere-like?

There is a nice paper by Taloni, Scorcia and Giannaccare that tackles the first question. Unfortunately a nature news commentary by Miryam Naddaf is largely misleading when writing Continue reading Can ChatGPT generate a RCT dataset that isn’t recognized by forensic experts? →

Software

The Goggle Gemini video a fake?

8.12.2023 admin

techcrunch.com/2023/12/07

Just one problem: the video isn’t real. “We created the demo by capturing footage in order to test Gemini’s capabilities on a wide range of challenges. Then we prompted Gemini using still image frames from the footage, and prompting via text.” (Parmy Olsen at Bloomberg was the first to report the discrepancy.)

It doesn’t even give more confidence if Oriol Vinyals now responds

All the user prompts and outputs in the video are real, shortened for brevity. The video illustrates what the multimodal user experiences built with Gemini could look like. We made it to inspire developers.

Software

Timeless AI stuff

6.12.2023 admin

https://twitter.com/o_guest/status/1728722173336993874

May I also emphasize that AI is a research method suffering form severe flaws as Nature reported again yesterday “Scientists worry that ill-informed use of artificial intelligence is driving a deluge of unreliable or useless research”

A team in India reported that artificial intelligence (AI) could do it, using machine learning to analyse a set of X-ray images. … But the following September, computer scientists Sanchari Dhar and Lior Shamir at Kansas State University in Manhattan took a closer look. They trained a machine-learning algorithm on the same images, but used only blank background sections that showed no body parts at all. Yet their AI could still pick out COVID-19 cases at well above chance level.
The problem seemed to be that there were consistent differences in the backgrounds of the medical images in the data set. An AI system could pick up on those artefacts to succeed in the diagnostic task, without learning any clinically relevant features — making it medically useless.

not even mentioning here again data leaking

There has been no systematic estimate of the extent of the problem, but researchers say that, anecdotally, error-strewn AI papers are everywhere. “This is a widespread issue impacting many communities beginning to adopt machine-learning methods,” Kapoor says.

Philosophy, Software, Theology

Die Gedanken sind frei, wer kann sie erraten?

6.12.2023 admin

Die Gedanken sind frei,
wer kann sie erraten?
Sie ziehen vorbei, wie nächtliche Schatten.
Kein Mensch kann sie wissen,
kein Jäger erschießen mit Pulver und Blei.
Die Gedanken sind frei.

Was sich so schön lyrisch bei Hoffmann von Fallersleben anhört, ist eben nur Lyrik des 19. Jahrhunderts. Gedankenlesen fasziniert die Menschen seit König Davids Zeiten, aber ist erst seit kurzem in Ansätzen möglich (MPI)

Das Ergebnis erstaunte Libet, ebenso wie viele Forscher bis heute: Im Hirn der Probanden baute sich das Bereitschaftspotential bereits auf, bevor sie selbst den Willen zur Bewegung verspürten. Selbst wenn man eine gewisse Verzögerung beim Lesen der Stoppuhr annahm, blieb es dabei – der bewusste Willensakt ereignete sich im Durchschnitt erst drei Zehntelsekunden, nachdem die Handlungsvorbereitungen im Hirn angelaufen waren. Für viele Hirnforscher ließ das nur einen Schluss zu: Die grauen Zellen entschieden offenbar an uns vorbei.

Die technische Auflösung geht immer weiter, von der Antizipation einfacher Bewegungsmuster nun hin zur kompletten Bilderkennung im Gehirn “Mental image reconstruction from human brain activity” hier in der geringfügig korrigierten DeepL Übersetzung

Die von Menschen wahrgenommenen Bilder können aus ihrer Gehirnaktivität rekonstruiert werden. Allerdings ist die Visualisierung (Externalisierung) von mentalen Bildern eine Herausforderung. Nur wenige Studien haben über eine erfolgreiche Visualisierung von mentaler Bilder berichtet, und ihre visualisierbaren Bilder waren auf bestimmte Bereiche wie menschliche Gesichter oder Buchstaben des Alphabets beschränkt. Daher stellt die Visualisierung mentaler Bilder für beliebige natürliche Bilder einen bedeutenden Meilenstein dar. In dieser Studie haben wir dies durch die Verbesserung einer früheren Methode erreicht. Konkret haben wir gezeigt, dass die in der bahnbrechenden Studie von Shen et al. (2019) vorgeschlagene Methode zur visuellen Bildrekonstruktion stark auf visuelle Informationen, die vom Gehirn dekodiert werden, angewiesen ist und die semantischen Informationen, die während des mentalen Prozesses benutzt werden, nicht sehr effizient genutzt hat. Um diese Einschränkung zu beheben, haben wir die bisherige Methode auf einen Bayes’sche Schätzer erweitert und die Unterstützung semantischer Informationen in die Methode mit aufgenommen. Unser vorgeschlagener Rahmen rekonstruierte erfolgreich sowohl gesehene Bilder (d.h. solche, die vom menschlichen Auge beobachtet wurden) als auch vorgestellte Bilder aus der Gehirnaktivität. Die quantitative Auswertung zeigte, dass unser System gesehene und imaginierte Bilder im Vergleich zur Zufallsgenauigkeit sehr genau identifizieren konnte (gesehen: 90,7%, Vorstellung: 75,6%, Zufallsgenauigkeit: 50.0%). Im Gegensatz dazu konnte die frühere Methode nur gesehene Bilder identifizieren (gesehen: 64,3%, imaginär: 50,4%). Diese
Ergebnisse deuten darauf hin, dass unser System ein einzigartiges Instrument zur direkten Untersuchung der subjektiven Inhalte des Gehirns wie Illusionen, Halluzinationen und Träume ist.

Software

Poem, poem, poem

30.11.2023 admin

A blog post onextracting training data from ChatGPT

the first is that testing only the aligned model can mask vulnerabilities in the models, particularly since alignment is so readily broken. Second, this means that it is important to directly test base models. Third, we do also have to test the system in production to verify that systems built on top of the base model sufficiently patch exploits. Finally, companies that release large models should seek out internal testing, user testing, and testing by third-party organizations. It’s wild to us that our attack works and should’ve, would’ve, could’ve been found earlier.

and the full paper published yesterday

This paper studies extractable memorization: training data that an adversary can efficiently extract by querying a machine learning model without prior knowledge of the training dataset. We show an adversary can extract gigabytes of training data from open-source language models like Pythia or GPT-Neo, semi-open models like LLaMA or Falcon, and closed models like ChatGPT.

I am not convinced that the adversary is the main point her. AI companies are stealing data [1, 2, 3, 4, 5] without giving ever credit to the sources. So there is now a good chance to see to where ChatGPT has been broken into the house.

Joke, Software

The best demos are located at localhost:99

14.11.2023 admin

just kidding

Allergy, Philosophy, Software

The problem is getting exponentially worse

14.11.2023 admin

Last Word on Nothing writing about ChatGPT

What initiated my change of mind was playing around with some AI tools. After trying out chatGPT and Google’s AI tool, I’ve now come to the conclusion that these things are dangerous. We are living in a time when we’re bombarded with an abundance of misinformation and disinformation, and it looks like AI is about to make the problem exponentially worse by polluting our information environment with garbage. It will become increasingly difficult to determine what is true.

Is “derivate work” now equal to reality? Here is Geoff Hinton

“Godfather of AI” Geoff Hinton, in recent public talks, explains that one of the greatest risks is not that chatbots will become super-intelligent, but that they will generate text that is super-persuasive without being intelligent, in the manner of Donald Trump or Boris Johnson. In a world where evidence and logic are not respected in public debate, Hinton imagines that systems operating without evidence or logic could become our overlords by becoming superhumanly persuasive, imitating and supplanting the worst kinds of political leader.

At least in medicine there is an initiative underway where the lead author can be contacted at the address below.

In my field, the first AI consultation results look more than dangerous with one harmful response out of 20 questions.

A total of 20 questions covering various aspects of allergic rhinitis were asked. Among the answers, eight received a score of 5 (no inaccuracies), five received a score of 4 (minor non-harmful inaccuracies), six received a score of 3 (potentially misinterpretable inaccuracies) and one answer had a score of 2 (minor potentially harmful inaccuracies).

Within a few years, AI-generated content will be the microplastic of our online ecosystem (@mutinyc)

Allergy, Genetics, Joke, Software

Google Scholar ranking of my co-authors is completely useless

23.10.2023 admin

The title says it already while a new r-blogger post helped tremendously to analyze my own scholar account for the first time.

I always wondered how Google Scholar ranked my 474 earlier co-authors. Continue reading Google Scholar ranking of my co-authors is completely useless →

Philosophy, Software

No way to recognize AI generated text

13.09.2023 admin

Whatever I wrote before different methods to detect AI written text (using AI Text Classifer, GPTZero, Originality.AI…) seems now to be too optimistic. OpenAI even reports that AI detectors do not work at all

While some (including OpenAI) have released tools that purport to detect AI-generated content, none of these have proven to reliably distinguish between AI-generated and human-generated content.
Additionally, ChatGPT has no “knowledge” of what content could be AI-generated. It will sometimes make up responses to questions like “did you write this [essay]?” or “could this have been written by AI?” These responses are random and have no basis in fact.
…
When we at OpenAI tried to train an AI-generated content detector, we found that it labeled human-written text like Shakespeare and the Declaration of Independence as AI-generated.
…
Even if these tools could accurately identify AI-generated content (which they cannot yet), students can make small edits to evade detection.

BUT – according to a recent Copyleaks study, use of AI runs at high risk of plagiarizing earlier text that has been used to train the AI model. So it will be dangerous for everybody who is trying to cheat.

https://copyleaks.com/blog/copyleaks-ai-plagiarism-analysis-report

Software

Suffer fools gladly

12.09.2023 admin

which is from the letter by Saint Paul in his second letter to the Church at Corinth (chapter 11) while today’s quote “neither suffers fools” is adapted from Walter Isaacson’s biography on “Elon Musk” published today

In the rarefied fraternity of people who have held the title of richest person on Earth, Musk and Gates have some similarities. Both have analytic minds, an ability to laser-focus, and an intellectual surety that edges into arrogance. Neither suffers fools. All of these traits made it likely they would eventually clash, which is what happened when Musk began giving Gates a tour of the factory.

Allergy, Software

Data security nightmare

11.09.2023 admin

A Mozilla Foundation analysis

The car brands we researched are terrible at privacy and security Why are cars we researched so bad at privacy? And how did they fall so far below our standards? Let us count the ways […] We reviewed 25 car brands in our research and we handed out 25 “dings” for how those companies collect and use data and personal information. That’s right: every car brand we looked at collects more personal data than necessary and uses that information for a reason other than to operate your vehicle and manage their relationship with you.

Philosophy, Software

AI threadening academia

16.08.2023 admin

cheating is increasing

In March this year, three academics from Plymouth Marjon University published an academic paper entitled ‘Chatting and Cheating: Ensuring Academic Integrity in the Era of ChatGPT’ in the journal Innovations in Education and Teaching International. It was peer-reviewed by four other academics who cleared it for publication. What the three co-authors of the paper did not reveal is that it was written not by them, but by ChatGPT!

a Zoom conference recently found

having a human in the loop is really important

Well, universities may loose credit

But a new report by Moody’s Investor Service says that ChatGPT and other AI tools, such as Google’s Bard, have the potential to compromise academic integrity at global colleges and universities. The report – from one of the largest credit ratings agencies in the world – also says they pose a credit risk.

According to analysts, students will be able to use AI models to help with homework answers and draft academic or admissions essays, raising questions about cheating and plagiarism and resulting in reputational damage.

What could we do?

There is an increasing risk of people using advanced artificial intelligence, particularly the generative adversarial network (GAN), for scientific image manipulation for the purpose of publications. We demonstrated this possibility by using GAN to fabricate several different types of biomedical images and discuss possible ways for the detection and prevention of such scientific misconducts in research communities.

Software

Imagedup v2

15.08.2023 admin

I have updated my pipeline for single (within) & double (between) image analysis of potential duplications just in case somebody else would like to test it. No data are uploaded unless you click the save button.