Hardware and software from IRE

Some unexpected hardware arrived in the mail from Investigative Reporters and Editors over the weekend.

It's a thank you for my volunteer work cleaning up the organization's archives as it prepares to launch a new website.

While the fruits of our labor won't be visible for a bit, one trick we figured out might be more broadly useful to newsroom nerds.

I took on a challenge to transcribe the thousands of recordings of IRE trainings, panels and other events the organization has accumulated over the years. By converting them into text, our hope is that we can create a searchable database where members can mine our communal knowledge.

You can see how I got it done by browsing this new open-source code repository, which applies the techniques I learned to a public domain source: the WNYC Radio archives held by city librarians.

It includes all the code and instructions you need to scrape audio files off the web and feed them to OpenAI's groundbreaking Whisper transcription service. By leaning on GitHub Actions' parallel processing system, my code can chew through 256 files at a time, quickly working through thousands of broadcasts.

And, get this, because of GitHub's generous policy towards open-source projects, the whole thing is free. I can imagine using the same toolkit in the newsroom to transcribe government meetings, political podcasts and any number of other newsworthy sources.

You can learn more about how newsrooms can use GitHub Actions to grind data by checking out Go Big with GitHub Actions, the free textbook that Iris Lee, Dana Chiueh and I released at IRE's most recent data journalism conference. It includes even more examples of how journalists are using cloud computing to gather, refine and analyze data.