Bare Facts First
How Reuters uses Datawrapper to
create, automate and disseminate
hundreds of charts each week
Who we are
I’m Jon McClure.

I've been the EMEA graphics
editor at Reuters in London
since 2020. Before that, I led
“interactive news” teams at
Politico and The Dallas
Morning News.
Who we are
I’m Ben Welsh.

I joined Reuters last year after
15 years leading data projects
at the Los Angeles Times.

I’m helping our team create
more of what we call “news
applications.”
What is Reuters?
❖ Founded in 1851, Reuters is the world’s
  largest international multimedia news
  provider.

❖ Each day 2,500 journalists in nearly 200
  locations around the globe produce news in
  16 languages.

❖ Financial professionals receive exclusive
  news and data from Reuters via the LSEG
  terminal.

❖ More than 2,000 newsrooms republish our
  work via a wire service.

❖ Breaking news, analysis and special reports
  are available for free on reuters.com.

❖ A legally binding vow requires we uphold
  trust principles valuing independence,
  integrity and freedom from bias.
Our mission
❖ Breaking news has been at the core of our
  business from start.

❖ An 1883 memo spelled out the London
  office’s expectations for correspondents.

❖ At that time, stories were sent by telegram.

❖ The editors asked “that the bare fact be first
  telegraphed with the utmost promptitude.”
What you’re in for
❖ Jon on how the Datawrapper charts
  created by Reuters staff are distributed to
  the world

❖ Ben on how we created a software system
  that automates routine Datawrapper
  charts

❖ Your questions
 Before Datawrapper
❖ High-caliber graphics team spread thin
  doing too much low-caliber work on
  request from other desks

❖ Too many good, simple charts not getting
  made because of the backlog

❖ Zero-sum between covering breaking
  news and doing significant enterprise

❖ Frustrated reporters resorted to a dark
  market of increasingly terrible and
  inscrutable charting tools
But before we could plug in Datawrapper, we had
to fit a pipeline that would serve our wire clients
as well as our website:
At Reuters, we publish each chart in many
different “editions” for other media clients to
pick from:

         Self-hosted and embeddable pages



         Static and editable images




         Editable source code




         Light & dark-mode charts
This is not the end. It is not even the
beginning of the end. But, it is perhaps the
end of the beginning, innit.
                                 Using Datawrapper’s
          API                    publishing hooks and API with
                                 the cheap scalability of
                                 serverless computing, we can
                                 process, repack and publish
                                 those different charts editions.




Serverless lambdas process,
repack and publish our charts!
      We use custom fields in the chart editor
      to include essential metadata we need
      to move charts through our publishing
      process.

API
Along the way, we flag new charts in
common Teams channels, which gives
our graphics team ample opportunities
to coach our newsroom.




     ✨Bonus: Need a Teams klaxon? Ours is free to use!
      reuters-graphics.github.io/teams-klaxon/
Within the last 18 months, Reuters
has onboarded more than 800
reporters, editors and producers
across our newsroom, who publish
dozens of charts each day.

That spread lets our graphics team
focus on higher spec graphics for
headline global news.

Reporters can publish more niche
charts on their local beat.

Producers have become frontline
chart makers for breaking news.
With our publishing pipeline in place,
graphics started developing automated
chart builders, linking external APIs with
pre-made chart templates at the speed of
breaking news.

“Tremolo” our USGS-powered earthquake
monitor published the first maps of the
deadly 2023 Morocco quakes within minutes
of USGS releasing the seismic data.
Publishing pipeline and a couple automated
chart builders underbelt, we were ready to
try scaling way up.
Text is automated
When Ben arrived at Reuters, he
learned it already has a text
automation system.

It’s called Lynx Insight. It fits
data to pre-defined templates
using carefully tailored logic.

This story is filed immediately
after new US employment
numbers are released each
month.
Why not graphics?
The graphics department head,
Matt Weber, long wanted to
automate graphics too.

As with Lynx Insight, it could
boost speed, increase volume
and allow highly-skilled talent
to focus on higher-skill work.

So Ben set out to try.
Feeding the funnel
Ben learned how Jon’s
distribution tools work.

His life flashed before his eyes.

Jon’s ingestion system for
Datawrapper was, in the
metaphorical parlance of Ben’s
tronc overlords, a funnel.

Ben just needed to figure out
how to feed it.
Enter the API
APIs allow programmers to
access online services with
code.

Datawrapper’s API allows users
to automatically generate
charts and maps.

All Ben had to do was send
instructions over the web with a
special access token.
Python is cool
Sergio Sánchez Zavala created
an open-source package that
lets you access the API using
the Python programming
language.

If you know Python, it’s easy to
use.

Ben knew Python.
Our advantage
Reuters is the exclusive news
provider for LSEG, a financial
service that competes with the
infamous Bloomberg Terminal.

That means we have access to
a high-quality, rapidly updating
feed of newsmaking numbers.

It’s the top source for Reuters
graphics, but reporters have to
claw out data files and paste
them into Datawrapper.

That’s slow.
Another API
Ben learned that the LSEG
system also offers an API.

He found its open-source
Python package.

He got excited.
The missing link
Ben realized that Python code
could download data from
LSEG’s API and then upload
charts to Datawrapper’s API.

All he needed was an “ETL”
pipeline that connected the two
systems.

So he wrote it.
The pipeline in theory
The pipeline in reality
For example
Here is Python code that —
using a set of custom ETL
helpers — creates a line chart of
the unemployment rate.

It’s just one small snippet within
a large system, but it shows the
basic bits needed for each chart
template.
Actions as platform
The system is run via Actions,
the powerful task runner
included with GitHub.

Yes, Actions is the production
environment.

Yes, it works.

Yes, it’s virtually free.

Yes, you should do this.
Down the funnel
After an Action makes a new
chart via the Datawrapper API,
Jon’s system picks it up and
does the rest.

The templated charts drop into
our graphics pool, just like the
custom charts made by our
staff.
Speed wins
The result is that when a
newsmaking number hits LSEG,
an automated chart can be
ready in minutes.

We’re seeing our charts land
even faster than our text
stories.
Scary scale
The templates can easily be
extended to sibling data sets.

This allows our system to scale
far beyond what any human
could do.

A powerful example is our bar
chart template for corporate
earnings reports.

It can instantly make a chart for
each of the thousands of stocks
tracked by LSEG.
Freedom rings
Ally Levine had to log on early
on Fridays to manually make
that unemployment rate chart.

Now the robot does the work.

Ally gets to sleep in and do
things like this instead.
Putting up numbers
We’ve been up and running for a
little more than six months.

Thus far, we have nearly 60
templates, including a few that
branch out beyond LSEG and
Datawrapper.

All told we’ve pushed through
more than 20,000 charts.
Merging back
Ben teamed up with Sergio to
expand and improve the
Datawrapper package for
Python.

It now covers 100% of API
features.
Python API code in reality
Python API code in theory
You must learn
Ben and Sergio created a
tutorial that shows newbies
how to use the Datawrapper API
with Python.

It was a standing-room only
event at the recent NICAR
conference in Baltimore.

It’s fully scripted and free to all
on Ben’s website.
Thank you!
Slides available at
bit.ly/bare-facts-first