Archiv der Kategorie: Software

Software

How to scrape a website with R II: WYSIWYG

Part II

Although Rselenium allows a screenshot of the current browser window

library(RSelenium)
remDr <- remoteDriver(
  remoteServerAddr = "localhost",
  port = 4444,
  browserName = "Chrome"
)
remDr$open()
remDr$navigate("http://google.com")
remDr$screenshot(display = T) # remDr$screenshot(file="screen.jpg")

I found it extremely difficult to control a webbrowser running in a Docker container – looking up the DOM tree, injecting javascript etc is a lot of guess work.

So we need also a VNC server in the docker container as found at github.

After starting in the terminal

docker run -d -p 4444:4444 -p 5900:5900 -v /dev/shm:/dev/shm selenium/standalone-chrome:4.0.0-beta-1-prerelease-20210207

we can watch live at vnc://127.0.0.1:5900 what’s going on.

Related posts: How to scrape a website with R I: Using a browser generated cookie | How to mod your blog to a seminar website | The best videoconferencing system (even running on a Synology box) | How to save a Twitter thread | How to switch divs on or off – a super neat trick |

Software

Breaking up long chains in R’s margrittr code and calling sub functions

Having several long and redundant chains in R ‘s margittr code, I have now figured out how to pipe into named and unnamed functions

f1 <- function(x) {
  x %>% count() %>% print()
}
f2 <- function(x) {
  x %>% tibble() %>% print()
}

So we have now two named functions with code blocks that can be inserted in an unnamed function whenever needed

iris %>%
  # do any select, mutate ....
  # before running both functions
  (function(x) {
    x %>% select(Species) %>% f1() %>% print()
    x %>% f2() %>% print()
  })

Related posts: A new dress code, ahem write code, for conferences | Click chains | The Rosetta stone and the genetic code | CNV (not London) calling | Genetic code and God’s language -cont’d- |

Software

How to save a Twitter thread

This is not a trivial task as most browser produce garbage when printing to pdf. Also screenshots do not help so much with multipage threads. “Unroll” also doesn’t work reliable.

So I have been using the archive.today plugin before producing the pdf,  but this method is time consuming  as it needs a captcha.

Having now Selenium running in a docker container, there is a more convenient solution where  just two lines of R are sufficient

remDr$navigate("https://twitter.com/SZ_Muenchen/status/1346425350394236930")
remDr$screenshot(file = 'screen.png')

Login is also possible as explained at github…

username <- remDr$findElement(using = "name", value = "session[username_or_email]")
username$sendKeysToElement(list("XXX"))
passwd <- remDr$findElement(using = "name", value = "session[password]")
passwd$sendKeysToElement(list("XXX", "\uE007"))

Related posts: Corrupted file list in Next Cloud | A strange survey at Slashdot | How to merge local and remote data in the browser | If it wasn’t actually you | A Mac OSX journalling error can inhibit the boot disk to be mounted |

Software

RKI Daten sind tages- und nicht fallbasiert

Leider kann man mit den RKI Daten nicht direkt in ein Regressionsmodell gehen, weil an vielen Tagen mehr als ein Fall in derselben Kategorie auftritt RKI Daten sind tages- und nicht fallbasiert weiterlesen →

Related posts: Was sie gesagt haben ist so wirr, dass ich nicht mal sagen kann, ob es Unsinn ist oder nicht | Die gefährlichsten Unwahrheiten sind Wahrheiten mäßig entstellt | Aber wir sind doch alles Tiroler | Wann sind Tiere am erfinderischten? | Warum die Teilnehmerzahlen so niedrig sind in deutschen Studien |

Software

How to scrape a website with R I: Using a browser generated cookie

While there are quite some SO examples out there how to manage the login, here are the ncessary steps whenever you need to login in manually and have to start with a browser cookie. First install the “EditThisCookie” plugin in Chrome and export the cookie How to scrape a website with R I: Using a browser generated cookie weiterlesen →

Related posts: How to merge local and remote data in the browser | Avoid browser flickering with dark backgrounds | Playing with your browser | How to mod your blog to a seminar website | New genome browser version |

Software

Better than a streamdeck: the Mac touchbar

I have been thinking to buy an Elgato Streamdeck as we have to lecture now also online during winter 2020.

While the Streamdeck mini would be a nice addition to the Elgato 4 K Camlink (that connects the Nikon) as well as the Epoc Cam software (that connects the Iphone) for easy switching input channels, I ultimately decided against this solution and just reconfigured the Touchbar for OBS using Better Touch Tool.

 

 

Related posts: Peer to peer publication | How to switch divs on or off – a super neat trick | $7.99 to detect a lying scientist? | Update problems of MacOS Catalina | Avoid browser flickering with dark backgrounds |

Software

Live video projection over 30 meter

Here is a working hardware combination after several failed attempts – jerky moves, black screen, lost signal, color shifts.

Nikon Z6 – switch video toggle @back of camera for clean HDMI output, set video menu to 60 frames (to avoid display lag), setting c3 standby on. 1800€ incl FTZ.

Sirui VH-10X Fluid, 100€ used.

subtel external power supply for Z6 (only this one allows closing of battery compartment). 56€.

Rode VideoMicro Compact. 88€.

Elgato Cam Link 4K with short USB C connector (to avoid shear stress). 122€.

Macbook Pro with USB-C Digital-AV-Multiport-Adapter (other display port and hdmi did not show up as external display in OBS). adapter 72€.

PW-HT225PIR HDMI Extender with power adapter (USB power alone not sufficient, HDMI amplifier failes consistently). 30€.

30m Cat6 cable. 30€.

Short throw Optoma UHD42 4K Beamer – UHD. 1200€

Related posts: Any new paper? A video cast how to stay informed | The reminder app is dead, long live the reminder app | Show speaker video at the wall | Link list essential photo & video & audio | Making the invisible motion visible |

Philosophy, Software

Science integrity consultants

Inclusion on the list does not mean to endorse or promote any company or individual

Resis (Enrico Bucci, but watch current Lancet controversy)

Science Integrity Digest (Elisabeth Bik, the most famous image spotter)

Image integrity (Jana Christopher, long-term expert)

Image Data Integrity (Mike Rossner, long-term expert as well)

plus many more individuals highlighted at retractionwatch.org

Related posts: Science of science: What makes up good science | What discriminates science journalists and science bloggers? | Celebrities on science | The price we have to pay for better science | The science and art of scientific presentations |

Software

Laborjournal einzelne Artikel archivieren

Ist ganz einfach. Neuestes Heft herunterladen und splitten

# brew install ghostscript
gs -dBATCH -dNOPAUSE -q -sDEVICE=pdfwrite -dFirstPage=6 -dLastPage=9 -sOUTPUTFILE=Krull.pdf LJ_20_07.pdf
gs -dBATCH -dNOPAUSE -q -sDEVICE=pdfwrite -dFirstPage=14 -dLastPage=17 -sOUTPUTFILE=Antes.pdf LJ_20_07.pdf
gs -dBATCH -dNOPAUSE -q -sDEVICE=pdfwrite -dFirstPage=26 -dLastPage=29 -sOUTPUTFILE=Jaeger.pdf LJ_20_07.pdf
gs -dBATCH -dNOPAUSE -q -sDEVICE=pdfwrite -dFirstPage=34 -dLastPage=37 -sOUTPUTFILE=Jeschke.pdf LJ_20_07.pdf

Software

Unter MacOS Keynote streamen mit Video

  1. OBS installieren mit Virtual Camera Plugin
  2. Powerpoint/Keynote als HTML exportieren und in lokalem Browser aufrufen
  3. OBS Template erstellen mit Text, Kamerainput und Browserwindow
  4. Virtual Camera in OBS starten
  5. In zweitem Browserwindow Konferenz starten und Einstellungen > Sicherheitscheck > Website > Einstellungen > Konferenz > Virtual Camera auswählen

Neu:

  1. OBS installieren
  2. Keynote in Fenster wiedergeben
  3. OBS Template erstellen mit Text, Kamerainput und Fensterinput

 

Related posts: Any new paper? A video cast how to stay informed | Show speaker video at the wall | Link list essential photo & video & audio | Verbergt nicht Eure Feigheit unter dem Mantel der Klugheit | Wer lebensgefährlichen Unsinn über Vitamin D verbreitet, wird mit Gefängnis nicht unter 1 Jahr bestraft |

Software

Graphical display of outbreaks: Transmission trees

Let’s start with some examples from the literature, find out necessary elements, compare different versions and develop a R template for general use.

“Exploding Stars”

CDC Field Epidemiology Handbook, 2019, ISBN 9780190933692 p115

“Tetris”

https://doi.org/10.1016/S0140-6736(20)30154-9 (2020)

“Clean Undeterministic”
“Clean Tree”

https://academic.oup.com/mbe/article/34/4/997/2919386

“Tree Addons”

https://openres.ersjournals.com/content/4/2/00162-2017

“Railways”

https://www.ages.at/service/service-presse/pressemeldungen/epidemiologische-abklaerung-am-beispiel-covid-19/

“Simple ”

https://www.ncbi.nlm.nih.gov/pubmed/32003551

“Vertical”

“star bus”

https://en.wikipedia.org/wiki/Index_case

“Mathematical”

https://www.biorxiv.org/content/10.1101/142570v3.full

“Genetics”

https://wwwnc.cdc.gov/eid/article/26/9/20-1798_article

Related posts: SQLite and a graphical ER diagrammer for OS X | On display – the masterclass | From digicam to web display | Leaflet.js – layer order, layer address and links | How to interpret an odds ratio of less than 1 |

Seiten: 1 2
Software, Video

From R to Python

It’s bit confusing if you are having long-term experience with R but need some OpenCV Python code. What worked for me

  1. download and install Python 3.8.3.
  2. pip install opencv-python
  3. pip install opencv-contrib-python
  4. although Spyder or Jupyter is recommended for data science, I went for PyCharm
  5. install Atom and follow the video instructions

  6. take care, numerous non working introductions out there, stick to recent version

Related posts: python3 cv2 installation | What to do with an old Aldi Medion (Dell Inspiron 1520) laptop? | OSX 10.6, Macports, GD and finally Circos | How doctors think | Upgrade and root Honor 6 |

Software

R leaflet – keep map in frame after closing info box

Although having some experience with leaflet before, it took me 5 hours to find out how to
(1) change the background color and
(2) re-center my map after the popup was panning the map  basically out view. Here is my solution

leaflet(options = leafletOptions(
 zoomControl = FALSE,
 minZoom=6, maxZoom=6,
 centerFixed = TRUE) ) %>%
addPolygons(data = ds,
 fillColor = ~pal(AnzahlFall),
 color="black", weight = 1,
 fillOpacity = 0.7,
 label = ~name_2,
 popup = ~www,
 popupOptions = popupOptions( autopan=FALSE, keepInView = TRUE ) ) %>%
htmlwidgets::onRender("
 function(el, x) {
  var myMap = this;
  myMap.dragging.disable();
  myMap.on('popupclose', function(e) { myMap.panTo( {lon: 10.26, lat: 51.1} ) });
  var e = document.getElementsByClassName('leaflet-container');
  e[0].style.backgroundColor = 'white';	    
}")

Related posts: email@nirvana.info | Raspberry Infrared: Just 50€ to detect a frame motor? | Leaflet.js – layer order, layer address and links | Let your visitors switch background color | Better figures |

Software

python3 cv2 installation

Homebrew is struggling with the older python 2.7 version that is missing ssl support. Python 3.8 did not recognize cv2 but this finally worked

brew install python3
brew postinstall python3
pip3 install virtualenv virtualenvwrapper
virtualenv env_name --python="python3"
/Users/wjst/env_name/bin/pip3 install opencv-python
/Users/wjst/env_name/bin/pip3 install pandas scipy matplotlib Pillow tqdm sklearn imageio
/Users/wjst/env_name/bin/python3 -c "import cv2"

then I could finally run the image forgery detection from Github.

cd /Users/wjst/Desktop/image-copy-move-detection-master
/Users/wjst/env_name/bin/python3
> from copy_move_detection import detect
> ...

Related posts: Install brew under Lion? | OSX 10.6, Macports, GD and finally Circos | Your own network running from a Macbook | How to export Whatsapp messages | In a perfect world |