CosmoPhys

New Location – This blog is no longer maintained here.

The cosmology researcher talks database app is a continuing work-in-progress on a collection that currently lists thousands of online talks by cosmology researchers intended for an audience of other researchers working in the field. The data is gathered from diverse sources (called 'series') that host talks from conferences, seminars, workshops, course lectures, summer schools, colloquia, etc.

This collection was started out of personal interest as a service to the cosmology community because no one site was found that comprehensively compiles such data from a wide range of sources. There are many sources available so this represents only a fraction of the total that's out there; still, it's a starter attempt to extend beyond what's otherwise available in one place.

The initial inspiration for this database effort started with the Cosmology Talks series hosted by Shaun Hotchkiss and the Cosmology from Home multi-year series. It quickly grew from that starting point.

There is an alternate version of the app hosted on Google Cloud Run. This is a more full-featured version though by nature of Cloud Run, the database is static and thus does not have the very latest data updates until the app is rebuilt and uploaded to Cloud Run. New data is published periodically (~ every 2 weeks).

Datasette is the open-source tool used to publish the data to Cloud Run. It was created by Simon Willison (github repo, @simonw on twitter). Many thanks to Simon for making this excellent tool available – it's a great match for this project and also especially for projects utilizing SQLite database files. This app can be self-hosted locally or remotely using Datasette, and it can also be hosted as a Docker container- see the Publishing Data section of the Datasette documentation.

The basic data structure is a talk title, including presenter(s), the series site hosting the talk video (e.g., Cosmology Talks on YouTube), and a talk date in (yyyy-mm-dd format). Sometimes only partial dates are available, e.g., dd or mm-dd is not known, but any date entry should include yyyy as a minimum).

The app's source code and collected data (.csv format) are available in this GitHub repo. The containerized app and its immutable sqlite database are hosted on the Google Cloud Run service. The primary app includes the most current updates not yet uploaded to Cloud Run, and is a simpler, quicker, though less full-featured web app. The data from the primary app that is pending upload to the Cloud Run can be viewed here. It's nice to have a variety of options!

Tips: To extract just the urls from the talks data file: egrep -o 'https?://[^)]+' cosmotalks.csv

To extract just the speakers: cut -c 3- cosmotalks.csv | awk -F' - ' '{print $1}'
Extract just the talkdate: grep -o '[^,]\+$' cosmotalks.csv | tr -d '"'

Tags: #astrophysics #cosmology #H0 #neutrinos #software #talks