Category Archives: Software

Software

Good scientific graphics

I have lectured about good graphics more than a decade ago – basically introducing Edward Tufte as the “Leonardo da Vinci” of data. With the beginning of 2022 I am now absolutely overwhelmed by the number of excellent tutorials out there. Here is my recommended list

 

CC-BY-NC Science Surf , accessed 19.09.2026
Software

The Python nightmare: How to install mask_rcnn and detectron2

Coming from R the most exciting thing with Python is the incompatibility of the different modules…

While I can recommend the Anaconda Navigator, the following commands work only in the terminal.

Thanks to Nouman945 for help with the original mask_rcnn instructions.

conda create --name matterport python=3.6.13 tensorflow==1.15.0 Keras==2.2.4 h5py==2.8.0 pip
conda activate matterport
git clone https://github.com/matterport/Mask_RCNN.git
cd ./Mask_RCNN-master
python -m pip install -r requirements.txt
git clone https://github.com/philferriere/cocoapi.git
python -m pip install pycocotools
python -m pip install git+https://github.com/philferriere/cocoapi.git#subdirectory=PythonAPI
wget https://github.com/matterport/Mask_RCNN/releases/download/v2.1/mask_rcnn_balloon.h5 > Mask_RCNN/mask_rcnn_coco.h5
conda deactivate

For detectron I follow basically detectron2.readthedocs.io

conda create --name detectron2 python=3.6.13 tensorflow==1.15.0 Keras==2.2.4 h5py==2.8.0 pip
conda activate detectron2
conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 -c pytorch
git clone https://github.com/facebookresearch/detectron2.git
CC=clang CXX=clang++ ARCHFLAGS="-arch x86_64" python -m pip install -e detectron2
conda deactivate

Last but not least the installed apps:

 

CC-BY-NC Science Surf , accessed 19.09.2026
Software

The Apple Keynote Feature we have been waiting for

Apple updated last month Keynote with a feature we have been waiting for years – live video which is super cool to show live internet pages from active polls.

 

insert video only works from the top menu

There is no video source connected at the beginning but can can be added from the right side menu.

Unfortunately the virtual camera of OBS is not recognized but I could easily connect my Blackmagic Atem Mini so I have direct access to any source coming from a second computer running Chrome, Ipad with Sketchbook and of course the Z6 HDMI video signal.

 

adding the ATEM Mini from the right menu

 

 

CC-BY-NC Science Surf , accessed 19.09.2026
Software

A comparison of image duplication software

A quick comparison without pretending any objectivity

Open SourceCommercial / Restricted AccessWebUpper Limit ImagesSpeedSensitivitySpecificityComment
Image Verification AssistantNNY1***
ForensicallyNNY1*****
FotoForensicsNNY1****
ImagedupNNY>500*****proof of principle
ImageTwinNYY10*******
LpixelNYY50????
ProofigNYY50?**/***/**
DropletsYNN1***
ImageJYNN1???
SherloqYNN1NANANAtoolbox
figcheckNNY1???scam?
BioVoxxelYNN1???toolbox
max rating is ***

 

I believe that all services should be in the open domain while bulk or routine data processing could be a business model.

 

 

CC-BY-NC Science Surf , accessed 19.09.2026
Allergy, Software

When a scientific journal is modifying your images

The Journal of Allergy and Clinical Immunology (JACI) published numerous images that have been heavily modified. For Pubpeer examples see 1, 2, 3, 4, 5, 6, 7…

As I learned recently, this happened without the knowledge of the authors. As the duplications are always at the outer edges and originate from the same image, we assume that JACI publishing service inserted some cloned background” to insert”journal style” letter boxes.

Difficult to understand? Here is an example from the first reference.

 

JACI is known to ignore misconduct allegations while they are now slowly responding to post publication peer review.

…the duplications occurred when the publisher’s compositor vendor styled the figure panel label for print. The error did not impact the analysis, results, or conclusions of the article. The original figure appears below. The authors were not responsible for the error. Measures have been taken to prevent reoccurrence of this error for which the publisher accepts full responsibility and apologizes.

So photoshopping is no more allowed at the JACI office in Colorado– but who is accepting this “full responsibility”?

 

29 Aug 2021

@spy_sci pointed out this morning to a paper in “Cell Journal” with a similar phenomenon while also @Cheshire has found multiple times over the past year. @SmutClyde contributed another example from Bioscience Reports.

 

 

CC-BY-NC Science Surf , accessed 19.09.2026
Philosophy, Software

Scientific PDFs

I found only one relevant blog post about PDFs produced by scientific publishers.

Although PDFs are the main formal output nowadays (no more “papers”) there is basically no standardization which meta data should be included in scientifc PDFs. It’s largely due to the software in the production office what’s in the PDF document — a largely underused resource.

This is at least my experience when working on an image duplication pipeline in scientific papers. But test it yourself ….

 

CC-BY-NC Science Surf , accessed 19.09.2026
Philosophy, Software

False dichotomies

“The false dichotomy between private interest and public good” by https://www.nuffieldbioethics.org/blog

'Good' and 'public benefit' are subjective concepts and will vary according to individual perceptions and context. Private and public interest are inevitably intertwined and pitting them against each other creates a false dichotomy. For example, if patients cease to trust their clinicians or more broadly the NHS, public good will suffer. Furthermore, extensive exploration of public attitudes towards sharing medical data has found that people approve in general for their data to be used for medical research and for 'good causes', whether environmental, social or medical, but they do not approve of their data to be used for commercial purposes or for powerful companies to profit at society's expense.

 

CC-BY-NC Science Surf , accessed 19.09.2026
Software

Finally I understand JS (a bit)

It produced a lot of headaches. And yes, it has been described already that

as a lot of articles I read about async in JavaScript, especially those that came out around the time ES2015 (ES6) was new, are just plain wrong. I'm going to clear up the falsehood by presenting a real understanding of the way async works in JavaScript.

Some nice examples are at medium.com and freecodecamp as well as a vid at youtube
.

At the end – I have some working code here, that shouldn’t work according to my understanding of concurrency.

 

CC-BY-NC Science Surf , accessed 19.09.2026
Software

How to monitor any website. At any time interval. With Telegram notification. For Free.

These instructions work for OSX Catalina only and are a bit different than the Linux version shown at urlwatch.

python3 -m pip install pyyaml minidb requests keyring appdirs lxml cssselect html2text
python3 -m pip install urlwatch
EDITOR=nano
urlwatch --edit
# CTRL+X leave the editor without saving

Then edit ~/Library/Application Support/urlwatch/urls.yaml

kind: url
max_tries: 3
name: Camptoi
url: https://www.camptoi.de
filter:
- element-by-id: product-desc-18713727425
- html2text:
    body_width: 0
    ignore_images: false
    ignore_links: false
    inline_links: false
    method: pyhtml2text
    pad_tables: false
    single_line_break: true
    unicode_snob: true

After that we create the Telegram bot by sending

/newbot

to user BotFather and record the <API> from the response.

We also need to send a dummy message to the new bot so we can look up the sender <ID>

https://api.telegram.org/bot<API>/getUpdates

<API> and <ID> are both entered into ~/Library/Application Support/urlwatch/urlwatch.yaml

...
  telegram:
    bot_token: '<API>'
    chat_id: '<ID>'
    enabled: true
---

As we have no cron under MacOS we need a parameter file to run launchd. Should be placed at ~/Library/LaunchAgents/com.urlwatch.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
	<dict>
		<key>Label</key>
		<string>com.urlwatch</string>
		<key>Program</key>
		<string>/Users/wjst/opt/anaconda3/bin/urlwatch</string>
		<key>RunAtLoad</key>
        <true/>
        <key>StartInterval</key>
        <integer>9000</integer>
        <key>KeepAlive</key>
         <true/>       
	</dict>
</plist>

Ultimately, the agent is started (or stopped) using

launchctl load -w  ~/Library/LaunchAgents/com.urlwatch.plist               
launchctl unload ~/Library/LaunchAgents/com.urlwatch.plist

 

CC-BY-NC Science Surf , accessed 19.09.2026
Software

How to scrape a website with R II: WYSIWYG

Part II

Although Rselenium allows a screenshot of the current browser window

library(RSelenium)
remDr <- remoteDriver(
  remoteServerAddr = "localhost",
  port = 4444,
  browserName = "Chrome"
)
remDr$open()
remDr$navigate("http://google.com")
remDr$screenshot(display = T) # remDr$screenshot(file="screen.jpg")

I found it extremely difficult to control a webbrowser running in a Docker container – looking up the DOM tree, injecting javascript etc is a lot of guess work.

So we need also a VNC server in the docker container as found at github.

After starting in the terminal

docker run -d -p 4444:4444 -p 5900:5900 -v /dev/shm:/dev/shm selenium/standalone-chrome:4.0.0-beta-1-prerelease-20210207

we can watch live at vnc://127.0.0.1:5900 what’s going on.

 

CC-BY-NC Science Surf , accessed 19.09.2026
Software

Sometimes it is better to switch bikes not gears

My 3 favorite programs? R, Bookends and Exposure.

The first choice is Rstudio (since 2012, being a long time SAS user). Here is the most recent JJ Allaire keynote

 

I was a bit with late with Bookends in 2018 (being a long time Sente user) but here is an introduction by Jon Ashwell.

 

And I am currently migrating to Exposure (after having used Lightroom for 13 years) by Jeff Butterworth

 

Life changes every day while it is sometimes better to switch bikes not gears.

 

CC-BY-NC Science Surf , accessed 19.09.2026
Software

Breaking up long chains in R’s dplyr/magrittr code and calling sub functions

Having several long and redundant chains in R ‘s magittr code, I have now figured out how to pipe into named and unnamed functions

f1 <- function(x) {
  x %>% count() %>% print()
}
f2 <- function(x) {
  x %>% tibble() %>% print()
}

So we have now two named functions with code blocks that can be inserted in an unnamed function whenever needed

iris %>%
  # do any select, mutate ....
  # before running both functions
  (function(x) {
    x %>% select(Species) %>% f1() %>% print()
    x %>% f2() %>% print()
  })

We can make a variable global from inside a function using <<- assignment but I have not found a way to continue the dplyr chain with any returned variable. So we are trapped inside the function

iris %>%
  (function(x) {
    result <<- x %>% head(1)
  })

 

CC-BY-NC Science Surf , accessed 19.09.2026