First GitHub Scraper

A step-by-step introduction to free, automated web scraping with GitHub’s powerful Actions feature.

You will learn how to:

• Create a GitHub repository to store your code

• Use Python to scrape data from the web

• Configure GitHub Actions to schedule the scrape

• Automatically save the results to the repository

• Send a Slack notification when new data arrive

Table of contents

• 1. Create a repository
• 2. Scrape data locally
• 3. Scrape data using Google Collab
• 4. Run via GitHub Actions
• 5. Saving the data
• 6. Blast the results

About

This guide was prepared for a training session at the National Institute for Computer-Assisted Reporting (NICAR)’s 2022 conference in Atlanta. The authors are Iris Lee, Aadit Tambe and Ben Welsh. The tutorial is published as open-source software.