Episode 20: Data journalism (feat. Ben Welsh)

By • Unspun in Pocket Casts

Recording

Listen on the original podcast site

Show the timestamped transcript
  1. Do you seek adventure and want to serve your country?
  2. The Central Intelligence Agency's Directorate of Operations is hiring.
  3. We are looking for individuals with a can-do attitude and a collaborative mindset.
  4. If you are looking for a challenge, apply online at cia.gov/careers.
  5. DreamWorks Waterpark at American Dream, just minutes from Manhattan.
  6. Over 40 rides, record-breaking slides, the world's largest indoor wave pool,
  7. and the steepest indoor drop slide.
  8. Tickets start at $59.
  9. Book your visit today at americandream.com.
  10. Our press secretary gave alternative facts to that.
  11. My goal in this deposition was to be truthful, but not particularly helpful.
  12. Welcome to Unspun, the podcast that makes you better at finding the truth.
  13. The way people get news is changing.
  14. It used to be that there were many reporters who would research stories and write articles,
  15. but now politicians and famous people share information directly with you on social media and the internet.
  16. That means you find out things fast, but it's up to you to make sure the information is actually accurate.
  17. And newsmakers don't always do their part.
  18. The temptation to manipulate information is strong.
  19. They bend the truth to deceive so that they can avoid accountability, so that they can advance their agendas.
  20. When you recognize these agendas, you can sometimes find out what's real.
  21. And we're at a crossroads where anyone can share anything online,
  22. so it's important to sharpen your critical thinking skills.
  23. Finding that deception before it goes viral is pretty much a survival skill now,
  24. and we're going to do it together.
  25. Let's get Unspun.
  26. Hello, everyone, and welcome to this week's Unspun.
  27. In this episode, you'll learn to recognize some ways that newsmakers use distraction to shape the news you see.
  28. Then when interviews and press conferences are not enough,
  29. journalists can use databases to find questions and answers that help hold the powerful to account.
  30. So a lot of data journalist Ben Welsh will join me for the interview.
  31. Sound good? Let's get Unspun.
  32. Imagine this. Your child has maybe been taking it easy and they're not doing so well in school.
  33. Education is important to you, and you know he's pretty capable,
  34. so you expect pretty good marks on the next report card.
  35. But your child knows that the next one you see is going to be really disappointing.
  36. For several days before that report arrives home,
  37. he pays extra attention to his contributions in keeping the home clean.
  38. He cleans his room maybe, he remembers to take his shoes off at the door.
  39. Somehow his dishes manage to make it from the table into the dishwasher without you even having to remind him.
  40. The day comes and you open up that mail from the school.
  41. You look up from that disappointing lineup and you start to say something,
  42. and he interrupts to say, "Hey, have you seen my room? It's very clean."
  43. Your child is hoping you will fall for distraction.
  44. Distraction tactics are powerful.
  45. They can change what people think and they hide important problems.
  46. By understanding these tactics, we can see the difference between real talk and lies.
  47. Distraction works by taking advantage of how our brains work.
  48. It's hard to focus on many things at once.
  49. When something new or exciting pops up, we pay attention to that instead.
  50. And this lets people hide things they don't want us to pay attention to.
  51. The news and social media make this worse.
  52. News needs to be new and exciting to get our attention,
  53. and that means big flashy stories, even if they're not important, they get shown more.
  54. And social media doesn't help, because it learns what we like and it shows us more of that.
  55. Which means we might only see things we already agree with, even if they're not true.
  56. This is bad for everyone.
  57. When we don't talk about real problems, they don't get fixed.
  58. Distraction stops us from making good choices because we don't have all the facts.
  59. We need to be careful and look for what's hidden when things feel like they're too simple,
  60. or even just too exciting.
  61. So let's unpack this in the political world.
  62. We'll listen to a few examples for this week's warm-up.
  63. Have a listen and see if you can identify the distraction.
  64. Around the globe, the president realized that our current approach to testing
  65. was inadequate to meet the needs of the American public.
  66. He asked for an entire overhaul of the testing approach.
  67. He immediately called the private sector laboratories to the White House, as noted,
  68. and charged them with developing a high-throughput quality platform
  69. that can meet the needs of the American public.
  70. We are grateful to LabCorp and Quest for taking up the charge immediately after the meeting,
  71. and within 72 hours bringing additional testing access,
  72. particularly to the outbreak areas of Washington State and California and now across the country.
  73. Ah, 2020, the early days of COVID.
  74. Maybe you remember the uncertainty, the fear,
  75. if you live in the United States, the daily press conferences.
  76. This clip beautifully illustrates a classic logical fallacy, the fallacy of distraction.
  77. So here's how it works.
  78. Instead of addressing the actual issue at hand,
  79. in this case, a lack of COVID testing, a shiny, irrelevant fact shows up.
  80. Hey, companies are helping us.
  81. All the things that are being described here are plans that have just started,
  82. but the situation was not good at the time.
  83. In fact, this is the same speech in which the president declared a national health emergency.
  84. Now, imagine this playing out in the whirlwind of the 24/7 news cycle.
  85. Headlines are changing at lightning speed and there's a constant influx of information,
  86. and it makes it difficult to focus on any one thing for too long.
  87. This environment is fertile ground for the distraction fallacy to thrive.
  88. Before we even have time to process the lack of testing,
  89. we're already swept away by the promise of a great new website from Google.
  90. And then enter social media algorithms designed to feed us more of what we already engage with,
  91. you know, creating those echo chambers and those filter bubbles.
  92. So when a politician or an organization successfully deploys a distraction
  93. that aligns with our existing biases, the algorithm amplifies it.
  94. But, you know, this strategy, this well-timed distraction is nothing new.
  95. So here's another example.
  96. Picture this, it's 1998, the Bill Clinton era,
  97. and the House is getting ready to vote on impeaching the president.
  98. A member of the House of Representatives gets up and says this.
  99. Where do we go from here?
  100. We will go to the floor of the House of Representatives to debate this issue.
  101. It is going to be a nasty debate.
  102. It will be a debate about sex.
  103. It will be a debate about the private affair of the president.
  104. It will be a debate about the extramarital affair.
  105. It will be a debate that the American public may not necessarily want to hear.
  106. But they have been thrust into this debate by members of this House
  107. and the Republican Party who are intent and who will stop at nothing
  108. to bring down the president of the United States of America.
  109. So, you know, at this time, the Monica Lewinsky scandal is exploding.
  110. And impeachment hearings are on everybody's mind.
  111. The actual grounds for the impeachment were perjury.
  112. Clinton, when he was being deposed for a sexual harassment lawsuit,
  113. denied having sex with Monica Lewinsky.
  114. And later, evidence emerged proving that he lied
  115. and this led to those charges of perjury, you know, that lying under oath.
  116. But when Representative Waters says what she says,
  117. she's implying that the Republicans are trying to impeach Clinton
  118. for a consensual affair rather than for the perjury itself.
  119. And this framing, whether it's accurate or not,
  120. shifted the focus away from the legal charge of perjury
  121. and toward a debate about the appropriateness of the relationship.
  122. Distraction achieved.
  123. As a little side note, this also kind of showed up
  124. when President Trump was being tried in New York City
  125. and Stormy Daniels was on the stand.
  126. You heard some people commenting saying that people just wanted the,
  127. you know, salacious details of the encounter
  128. between Stormy Daniels and the president.
  129. There was a second distraction tactic that was related to Clinton.
  130. Here's Senator Dianne Feinstein.
  131. Well, I've read most of the press from other countries.
  132. And as you know, it's universal that no one can understand what is happening
  133. and that most of the leaders of these other countries
  134. really believe that we're tearing at the fabric of the presidency
  135. at a time when the United States is really important in the world
  136. in terms of peace and stability.
  137. The world is a more uncertain place now than it has been.
  138. The world is a more uncertain place now than it has been in the last ten years.
  139. And having a strong American president out there is important.
  140. And all of this goes to diminish that.
  141. So I'm one that believes it should be brought to conclusion as quickly as possible.
  142. And then let's get on with it.
  143. Did you hear the distraction here?
  144. The rest of the world will think less of the United States.
  145. So if you're debating this, what are you not debating?
  146. Whether or not the president committed perjury.
  147. So what's the takeaway?
  148. Well, we live in a world of carefully curated narratives
  149. where bad news is eclipsed by shiny distractions
  150. and it's amplified by the actual technology we use to consume the information.
  151. So whether it's a press conference or a news feed
  152. or even just a casual conversation, remember,
  153. you can always look below the surface.
  154. And one thing that helps journalists look below the surface
  155. is to get the data behind the story so that they can evaluate it themselves.
  156. I need to take a quick break, but when I come back,
  157. Ben Welsh from Reuters and I talk about the world of data journalism.
  158. I'll be right back.
  159. While we're on a break, I wanted to remind you that I have a lot of material
  160. like what we have in warm up in my book, Detecting Deception,
  161. Tools to Fight Fake News.
  162. I break down more than 20 ways that newsmakers use facts and language
  163. to be deceptive, and for each, I give you real world examples to explain
  164. and then some historical examples that let you put yourself
  165. in the role of a journalist trying to report on these newsmakers.
  166. There's even answers in the back.
  167. It's an inexpensive, helpful little book that you might enjoy
  168. but in between unspun episodes.
  169. You can find out more at bit.ly/detectingdeceptionbook, all one word.
  170. That's bit.ly/detectingdeceptionbook, all one word.
  171. And now just a quick word from my sponsors
  172. and then my uninterrupted interview with Ben Welsh.
  173. Welcome back.
  174. All right, so I'm very excited to have you with us this week on the podcast.
  175. And one of the things I first wanted to ask you is it seems like your job
  176. is really different from the job you might like see a journalist on TV doing,
  177. right, you know, like in a movie or something.
  178. So it's different from sitting in a press briefing
  179. and asking questions or talking to someone on the phone.
  180. So first of all, can you just explain what data reporting is?
  181. What is data reporting? Great question.
  182. I think it actually is actually more than one answer.
  183. And the way to summarize it, like a lot of things in life, is it depends.
  184. It kind of depends what you do with it.
  185. I mean, the most famous example of a data journalist people might encounter
  186. on cable news would be John King or Steve Kranacki
  187. or these folks that stand in front of the magic maps on election night
  188. and help to decode what the data is telling us about how voters,
  189. you know, how voters are behaving.
  190. But data journalism is a lot more than that, too.
  191. It's it really is, you know, reporters and editors
  192. who treat structured information, spreadsheets, maps,
  193. big piles of documents as sources in the same way
  194. that we've traditionally all treated people.
  195. We can talk to it on the telephone or shoot questions out of the press
  196. conference as sources, you know, structured information.
  197. Data is is a source that in the same ways.
  198. And we're we're using as data journalists
  199. just kind of different tools to ask questions and try to get answers
  200. out of, you know, the data that surrounds us all every day.
  201. Yeah. So that sounds like it would have a kind of different set of challenges
  202. from doing live interviews.
  203. Would you be willing to give maybe an example of a story
  204. that you've worked on that you're particularly proud of? Hmm.
  205. Well, it's always interesting which one pops into your head.
  206. Well, as a few years ago, I was working as a reporter at the Los Angeles Times
  207. and a thing happened in the news.
  208. A candidate for mayor made an allegation.
  209. He said that, hey, the fire department here in our city is in big trouble
  210. and you better elect me to fix it.
  211. Now, obviously, that's a bit of a caricature on my part,
  212. but he said something along those lines.
  213. He actually said it in more vivid terms, made some pretty strong claims
  214. about what was happening in our local 911 system.
  215. And that kind of raised a question in our public sphere in Los Angeles about,
  216. hey, is the fire department doing a bad job?
  217. What's going on over there? Right.
  218. And that's a question that, you know, with traditional techniques,
  219. you can answer by just interviewing people,
  220. you know, going to a fire station and riding along and seeing what's going on.
  221. And all that is supremely valuable.
  222. But it's also a question that there's a measurable answer to.
  223. In some cases, you know, it's claims were and as a data journalist,
  224. that's often what we're looking to do.
  225. We have we have questions for the world.
  226. But can we can we measure the answer to help us come up with something
  227. that's a little maybe more stronger or broader or more contextualized
  228. than what you could do without it?
  229. And so in that case, the claim really was that the response times,
  230. how long it was taking for the fire department to arrive when you call 911
  231. was getting slower was the claim of the candidate.
  232. And hey, that's measurable.
  233. And so I set out to file a Freedom of Information Act request
  234. seeking not public documents, not emails or memos, but a database.
  235. There's a database at the core of the 911 system.
  236. I learned that by interviewing people as a traditional reporter would.
  237. I filed the request is a longer story, but, you know, I eventually received
  238. a CD in the mail that had the government database of every 911 call
  239. for a long period of time.
  240. And then I set out to to write the computer code necessary
  241. to sort of get my hands on and interview that database
  242. to ask you questions like, hey, how long does it take, say, on average
  243. for a 911 response to arrive?
  244. Hey, is it faster in one part of town versus another?
  245. Hey, is it getting slower?
  246. Is it getting faster over time? Right.
  247. And and I also sought out outside standards for what might be good or bad.
  248. Hey, the fire department has said their goal is to respond in X amount of time.
  249. Are they doing that? Are they not, et cetera, et cetera?
  250. And so those are all kind of questions that were measurable.
  251. And then you write the code that kind of spits out the answer,
  252. you know, not unlike how a social scientist or someone who works
  253. at a university might for an academic study, just maybe in a simpler way.
  254. And then the answers that we got back from those provided the foundation
  255. for a whole series of stories we did about problems in the 911 system,
  256. which led to some pretty significant reforms and how it was managed,
  257. including changes to how calls were handled, the replacement of the fire chief.
  258. Ultimately, and a number of changes.
  259. And one fun fact is all the things that we discover and wrote about
  260. were not what the candidate was complaining about.
  261. His complaints were actually inaccurate.
  262. But by getting the data and figuring it out ourselves with, you know,
  263. with our technical skills and a more scientific approach,
  264. we actually got a lot closer to the truth. Right.
  265. Fair enough.
  266. So one thing that makes me think of is that I know that in journalism,
  267. a lot of reporters have sort of standard coverage areas that they work in.
  268. Right. So that they get to know the sources and they get expertise in that area.
  269. You know, we might call that like beat reporting as a data reporter.
  270. Do you have particular areas of specialization
  271. or is it more that you kind of go out and work with other people?
  272. Some do, some don't.
  273. You know, I think it really, again, come back to my unsatisfying answer.
  274. It depends.
  275. And so there are data journalists who have areas of really strong expertise.
  276. I think elections is an example of that and political science and polling.
  277. There's a lot of great data journalists who that's pretty much all they do.
  278. And they do it very, very well. Right.
  279. And then there's other data journalists who are maybe a little more
  280. technical specialists or sort of, you know,
  281. social scientists for hire in the newsroom, you know,
  282. who will range and partner with reporters who have expertise
  283. in this topic or that topic. Right.
  284. And juggle a lot of different things over time.
  285. I think in the career of a data journalist, you can end up in either of those situations.
  286. And it often depends on the personality of the person.
  287. But I think either model can work really very well.
  288. You do end up developing as this is true of all reporters,
  289. is you do end up kind of developing expertise in the areas you cover over time.
  290. And that isn't always planned out. It just kind of happens.
  291. OK, it sounds like this kind of projects take a lot of time, too.
  292. Would that be fair to say?
  293. They can. Again, it depends.
  294. You know, I think traditionally data analysis and journalism was developed
  295. a lot earlier than I think people realize there.
  296. I mean, there were data stories that were quite scientific in the 19th century,
  297. including one you can read about, about how a young congressman named
  298. Abraham Lincoln was submitting the largest expense reports in Congress
  299. at the time for travel.
  300. Those were, of course, all done with, you know,
  301. very traditional analog computational techniques,
  302. probably pen and paper. And but, you know, even computerized data analysis
  303. is older than people realize it in the 1960s and 70s.
  304. You already had a small group of data journalists doing very ambitious
  305. survey stories and other other works of analysis
  306. using punch cards and early computers.
  307. And those projects created kind of an early investigative tradition
  308. in data journalism, where a lot of the specialists were really focused
  309. on long term investigative pieces that took quite a bit of time to get done. Right.
  310. And so that was kind of the origin of a lot of the computational approaches
  311. to data in news. However, over time, as these tools have gotten easier to use,
  312. as data has become much more commonplace in our society,
  313. you see a lot of data work happening very, very quickly.
  314. And so you can see data stories that happen on a one or two day turnaround.
  315. You know, when there's an unexpected event where data can help cover it.
  316. And then there's also cases now where, you know, data has become a flowing
  317. river of news where every day when a new macro economic number comes across
  318. the wire from the Bureau of Labor Statistics that can feed into
  319. not just traditional stories that a data journalist would report and put out,
  320. but also maybe into software applications you're making to automatically publish new stories.
  321. So you're saying you actually do write software as a part of your job?
  322. I do not all data journalists do, but I definitely do.
  323. An example that I think almost all of your listeners will be familiar with.
  324. There's two. One is election night.
  325. So, you know, the most popular piece of journalism on election night now
  326. and pretty much since about 2008 have been the live updating election maps
  327. that give you the results as they come in.
  328. Almost all of those systems were invented and developed by people who started
  329. their careers as news reporters and editors and kind of drifted into software
  330. development or sought it out because it does take quite a bit of editorial
  331. kind of knowledge to be able to make something like that correctly.
  332. Another example that's more recent is a coronavirus tracking.
  333. So I'm sure almost everyone who's listening looked at, you know, a daily
  334. updating map or chart, at least with some frequency of the latest
  335. information on coronavirus where they live.
  336. Those were also cases where data journalists were kind of filling the void
  337. of where the government information wasn't satisfactory and developing
  338. software to kind of put news out automatically.
  339. Yeah. So you bring up, I think, an important thing here, which is
  340. it sounds like this kind of reporting is pretty resource intensive
  341. in terms of sort of the human resources, the time, the expertise.
  342. People have to build up those kind of things.
  343. So I guess what I'm getting at is what makes it worth it.
  344. So you've brought up kind of one thing, which is that you can kind of fill in
  345. gaps, right, telling stories that are not going to come out otherwise.
  346. Are there other things that kind of make the investment worth it?
  347. You're right. It is higher skill work.
  348. And we've seen this in both data and graphics journalism over the last two
  349. decades is that they've gone from jobs that were often about creating
  350. small accessories to go with stories to making these much more sophisticated
  351. feature stories, large scale investigations and these data applications.
  352. So it's definitely higher skill work.
  353. You hit on one of the major payoffs is like, yeah, it lets you do the big
  354. story you couldn't have got otherwise.
  355. Right. And that's been the reason this field really started to develop
  356. in the first place decades ago.
  357. But you have to remember is that once you automate something
  358. and put it on the Internet, you can have quite a bit of scale. Right.
  359. And so, for instance,
  360. when I was an editor at the Los Angeles Times, we created a system
  361. to automatically draft blog posts every time there was an earthquake.
  362. The government data would come across and we would have within seconds
  363. a post with a map that had the bare facts about the earthquake
  364. that was reviewed by a human editor, of course, but was ready to go out
  365. before everyone else for this, an event that would draw a lot of attention.
  366. Well, yeah, it did take us a few months to invent that application,
  367. finish it, vet it and prepare it.
  368. But once it was running, it was close to zero labor whatsoever. Right.
  369. And it then every time there was an earthquake
  370. yielded a piece of content that was oftentimes of very high value. Right.
  371. And so you have a large investment at the outset. Right.
  372. But once it's operational, it can have quite a bit of long running value
  373. and a really wide amount of scale.
  374. The third thing is is a lot of these data features are actually significantly
  375. sometimes by an order of magnitude or more
  376. more popular than a traditional text and photo story. Right.
  377. The coronavirus trackers, for instance, that were published by news
  378. organizations were in almost every case, the most popular piece of content
  379. ever published in the history of the news organizations that published them.
  380. That's these are like at the Los Angeles Times, our coronavirus tracker
  381. was the most read thing ever published by the L.A. Times in 130 years. Right.
  382. Of thousands of people spending how many work hours creating product.
  383. And so, yes, it is higher skill work.
  384. And yes, it does require more time than writing an individual story.
  385. But it also can be significantly more valuable.
  386. OK, so that makes a good business case, I guess, for my investment.
  387. This is why this is me to my boss saying, hey, let me hire some people.
  388. OK, that's fair enough.
  389. It does bring up another question, though, because I think about, you know,
  390. a larger resource publication or TV station like a CNN or L.A.
  391. Times versus maybe like your local news or your local newspaper.
  392. Do you think that the ability to produce these kinds of stories
  393. sort of creates an information divide between national and local?
  394. It definitely does, yes, because it requires this higher amount of skill
  395. to publish it. It can be hard for a small newsroom to put it together.
  396. So the attempts to solve that in the marketplace have been, of course,
  397. like everything else with local news, chain ownership, you know what I mean?
  398. And so the Gannett Company and these sort of large chains that own dozens
  399. and dozens, if not hundreds of local newsrooms have experimented
  400. with having sort of centralized teams that, you know, provide this type
  401. of work for the entire chain.
  402. You see that also in the television marketplace.
  403. So CBS News here in New York has a growing data team they're hiring
  404. for right now, and they'll perform data analysis and work for all
  405. of the owned and operated CBS stations across the country.
  406. So centralization is definitely one solution.
  407. Obviously, I should pump here a little bit for the wire services.
  408. I work at Reuters News, which is one of the world's largest agency
  409. and wire services, and so subscribers, you know, to our wire service,
  410. any outlet are able to access a lot of these graphics that we make automatically
  411. using some of these things I've talked about.
  412. And our live election results will publish this November
  413. with the national election pool are available to subscribers to our wire.
  414. So if you're if you're a member of Reuters News, you'll be able to take
  415. at a bare minimum the sort of finished maps and tables that we create
  416. and put them on your site or on your TV station.
  417. So those are two ways it works.
  418. There are some nonprofit academic alternatives to this.
  419. I spent a year at Stanford University recently working
  420. on a project called Big Local News, and the intention of this project is
  421. to create sort of a a similar model, a similar
  422. centralized kind of model for providing data services and skills
  423. and support to newsrooms in the nonprofit sector.
  424. So a lot of, you know, a lot of the small newsrooms that are coming up
  425. on the nonprofit side who don't have access to the things
  426. these these larger commercial chains do.
  427. OK, so I want to switch gears a little bit here
  428. and ask you about the particular data sets and things that you access.
  429. And can you talk a little bit about how you try to verify
  430. sort of the reliability of the data that you get access to?
  431. Sure. I mean, it can depend because there's there's different types of data sets,
  432. you know, and there's different ways they're kind of created.
  433. But to use the 911 data set as an example, we started with.
  434. So I have via a Freedom of Information Act quest,
  435. the official database from the primary source. Right.
  436. So in a way, that's the same as interviewing the buyer chief.
  437. There's no there's no intermediary who might introduce error. Right.
  438. Now, there are cases where you might get a data set from an intermediary
  439. and that could introduce some question about the kind of chain of custody.
  440. But in this case, it was what I would call a primary source.
  441. But it's, you know, 15 database tables that are largely undocumented.
  442. And not a lot of people at the agency are all that willing to explain to me
  443. how they fit together and how they work and what the caveats are of.
  444. Oh, gee, you need to filter and exclude all these and oh, gee,
  445. be careful when you merge these two things and all the crazy stuff
  446. that can happen with databases. Right.
  447. And what do these fields and columns even mean?
  448. They have these cryptic names.
  449. And so there often is in a circumstance like that in data journalism,
  450. a bit of forensic analysis you do to just sort of try to gradually
  451. test and deduce what the heck is in there.
  452. That can include interviewing people who have worked on the system,
  453. which I did in that case.
  454. We found an ex employee who had worked on the database
  455. and someone who had done data entry on the database.
  456. And we kind of were able to decode what was in there
  457. and understand better how to interpret it.
  458. Oftentimes, just finding the form that's used to create a database
  459. can really help you understand what's in there
  460. because, you know, that has a little more of a human sort of base to it.
  461. And then to me, there's sort of, I guess, lacking a better term,
  462. I would say kind of external validation or checking, checking, checking my work.
  463. And so in the case of the fire system, I had some questions
  464. I wanted to answer that were really specifically about our story
  465. and like, hey, are the response times meeting this national standard?
  466. Are are things getting faster, things getting slow?
  467. That's what I wanted to know at the end of the day.
  468. But those those didn't help me verify whether I was right or not. Right.
  469. And so what I did is I sought out
  470. data analysis that had been published by the department itself
  471. in its annual reports and presented at hearings before the city council.
  472. So in the past, the fire department had added up.
  473. We did this many total calls over the whole year.
  474. And these sort of general aggregate statistics have been published.
  475. And I sort of hypothesized or thought, hey, if I'm analyzing this database correctly,
  476. right, I should be able to hit those numbers, right?
  477. Using whatever sort of queries or formulas I'm going to use to analyze the data set.
  478. And that gave me some confidence that, hey, I'm able to interpret this data
  479. without making a lot of it, you know, without making a mistake in how I'm working with it.
  480. And then, you know, in a case like this, an investigative report,
  481. all of our findings were presented
  482. to the agency prior to being published.
  483. And so I'm a big fan of when publishing
  484. sort of claims that are that are going to that are going to get out on a limb
  485. a little bit or challenge a source or subject,
  486. is that they have the opportunity to review, comment, critique on the method
  487. and the numbers prior to publication, which I did in that.
  488. There's other stories I've been down that road on it to me.
  489. That's a really important step.
  490. And we did that.
  491. And but, you know, even at the end of the day, there sometimes can be a little
  492. you might not get a hundred percent answer
  493. from your source about whether or not you've done it right,
  494. because you may be breaking new ground in terms of, you know, your findings.
  495. And so a really important piece
  496. that I think is often overlooked in a lot of the data journalism today
  497. that's done very quickly or by people who don't have a journalism background
  498. is shoe leather ground truthing the data, which is basically saying, OK,
  499. so we did this analysis that was like these this this part of the call system
  500. is really slow and it's causing problems.
  501. Well, if that's right, I should be able to sort and list the calls
  502. that are the worst, the outliers that were that are going to be inside
  503. this kind of claim that we're making.
  504. And we should be able to call up the people on the phone or knock on the door
  505. of the people who are listed there and be able to verify that.
  506. Yeah, it was slow. Right.
  507. And so to me, it was taking the data out into the real world. Right.
  508. To verify that, hey, our assumption is that this is pointing to a thing
  509. that happened in the world. Did it? Right.
  510. And that requires getting, you know, unplugging your laptop, closing the lid,
  511. you know, getting out onto the street, knocking on doors, talking to people
  512. and really checking whether things you match up out there in the real world.
  513. And I think that that is an underrated and essential piece
  514. of doing quality data journalism.
  515. It also helps you find the stories and real people who are affected
  516. by the statistical stuff that you're writing about.
  517. And you end up with a much, much better piece of journalism.
  518. And you also can have an even greater degree of confidence
  519. that what you're writing about matters, whether or not it, quote, adds up.
  520. Yeah. So just to go back to a term that you used,
  521. you use the term chain of custody.
  522. You got it directly from the horse's mouth. Right.
  523. So there's cases where this happens frequently with document dumps
  524. and other types of data stories where you get the information
  525. from an intermediary, a leaker, right, or there's been a hack.
  526. And you may remember this from the coverage of the WikiLeaks documents
  527. and things about a decade ago.
  528. Questions were raised about whether the information had been tampered with.
  529. Right. And can we really believe that this email that says is from Hillary Clinton
  530. is really from Hillary Clinton? Right.
  531. And those are cases where you have to make extra steps
  532. to verify that what you have is is is authentic.
  533. But in the case where you file a Freedom of Information Act
  534. directly to a government agency and they give it to you directly,
  535. you don't have that intermediary.
  536. The human equivalent of this, and this is true in data journalism,
  537. almost anything we would talk about in this wonky, techie data way.
  538. There almost always is a shoe leather on the street,
  539. old school reporter version of is, hey, you went to the protest
  540. where things really got crazy and did you see the crazy thing that happened?
  541. Did you talk to the person who the crazy thing happened to
  542. or did you talk to the person who heard from the person
  543. the crazy thing happened to? Right.
  544. And it's the same thing.
  545. And this is true in all different aspects of journalism,
  546. is how close are you really to the to the real thing?
  547. Right. To what actually happened and this and in data that's no different.
  548. How do you balance all of those technical wonky things you just talked about
  549. with the need to tell a really interesting story
  550. maybe to a public who's often number phobic?
  551. I mean, connecting it to real people and real things that happened,
  552. I think is is essential.
  553. I think it is missing from a lot of data reporting.
  554. However, I think there's more appetite for data than I think is appreciated.
  555. You know, I think that properly balanced reported and presented
  556. data stories are really very popular.
  557. And I think that not to not to hit this nail too many times in our talk here today,
  558. but I think that that is underappreciated.
  559. I think there's oftentimes a sense that these are vegetable stories
  560. when in fact, when we look at the look at the numbers and the metrics,
  561. we see that these are actually quite popular stories when done well.
  562. Right. The public wants information that's more trustworthy and reliable
  563. and data and science is a way that we bring that to our journalism.
  564. Right. To me, there's a lot of it comes down to the editing.
  565. And this is true in traditional stories as well.
  566. Is data stories can sometimes lack that ground truth thing
  567. and that connection to what's really out there. Right.
  568. They can sometimes be what in a in a traditional story
  569. would be called a notebook dump.
  570. You know, so, hey, I did a data analysis.
  571. Here's 15 things I found from looking at the data.
  572. And it's just like a bunch of stuff.
  573. And it's not clearly focused enough that it connects directly
  574. with a busy person who who is just trying to go through the headlines and read the news.
  575. And so to me, the best data stories, not always,
  576. but oftentimes are about one finding or one very simple statistic.
  577. And then they don't overload all the numbers and stuff in the way
  578. that there's there's a good reason that academic papers do that. Right.
  579. And I do think a well presented data story will offer, you know, below the fold,
  580. so to speak, in the news or on a link that goes off to a methodology page
  581. or some code and data that you posted online.
  582. It will provide that extra stuff for the reader who wants more of it.
  583. And to verify it's kind of it's it's reproducibility.
  584. But the headline, what we call news, the nut graph,
  585. the central finding of any piece needs to be almost always like one thing
  586. with maybe just one or two numbers that really connect
  587. directly as possible to the reader. Why is this news, you know?
  588. Yeah. And so I guess that would explain then why there's often
  589. a big series around a large data story, right?
  590. Each one is telling a different piece of it.
  591. It can be, yes. Yeah. All right.
  592. What are some of the biggest challenges that you face in doing this kind of work?
  593. Gee, I mean, it does require more technical skills
  594. than I think a lot of people receive in their journalism training.
  595. You know, over time, it's something that any person could learn.
  596. Being a data journalist is not like being an opera singer or the piano player
  597. at the orchestra. It's not something that requires you're born
  598. with some incredible innate talent.
  599. It just requires that you learn
  600. numeracy and kind of set of kind of spreadsheet like skills
  601. for how to get your hooks into clean up, deal with and interview a data source.
  602. And so I think that that is its own little educational ladder
  603. that you have to climb to to do.
  604. But I think it's really attainable to anyone.
  605. But I think starting out, that can be a challenge, you know,
  606. in the same way that starting out at any professional job
  607. requires a little bit of of learning.
  608. The data that is often most newsworthy is often difficult to acquire.
  609. So it's there's cases where, you know, the government has a data set
  610. that you want to do a story about, but they're not giving it to you.
  611. And so you got to figure out how to get it.
  612. And so that kind of struggle to get the data that will make the news
  613. is a challenge. Cleaning data, as anyone who works in what is now
  614. known as data science and a lot of kind of these more practical applications
  615. of statistics will tell you, cleaning the data can end up being
  616. about 95 percent of the work.
  617. So the actual analysis that you do can be quite simple,
  618. especially for a journalism story.
  619. But once you even have acquired the data, just getting it into shape
  620. where it's ready to analyze can be a ton of work
  621. just due to errors in how it's been structured or formatted.
  622. That the need to to kind of clean up values, all the different ways
  623. that you can spell the street names of New York or L.A.
  624. If you're a police officer filling out a citation form, right,
  625. can lead to quite a lot of variations that need to be mopped up.
  626. So cleaning is in and learning the skills of cleaning
  627. and how to do it correctly and all that can be a bit of a challenge.
  628. And it's oftentimes most of the job.
  629. All right. And I appreciate your time and thank you so much for stopping by.
  630. Thanks for getting Unspun with me this week.
  631. Unspun is a production of me, Amanda Sturgill,
  632. and is a proud member of the MSW Media family of podcasts.
  633. Some of your thoughts and ideas about trickery in the news on Gmail
  634. at theunspunpodcast@gmail.com.
  635. I even write back.
  636. And find this episode's show notes and more information
  637. at theunspunpodcast.substack.com.
  638. Want to learn more and get smarter?
  639. Check out my book, Detecting Deception, Tools to Fight Fake News,
  640. which is available on Amazon or your favorite online bookseller.
  641. And until next time, stay sharp, everyone.
  642. Looking for the best of VRBO?
  643. Discover Vacation Rentals of the Year,
  644. top quality homes guests love from across the country.
  645. Our curated list takes the guests work out of planning
  646. so you can spend less time searching and more time packing.
  647. Book your next day now with VRBO.
  648. This is Bethany Frankel from Just Be with Bethany Frankel.
  649. I may be in Florida these days, but at heart, I'm a New Yorker.
  650. We're real, we don't do fake, and that includes what we feed our dogs.
  651. Just Food for Dogs has real kitchens right in New York City,
  652. where you can watch chefs make fresh, human-grade food for your dog,
  653. not mystery kibble, real food.
  654. My dogs, Biggie and Smalls, would lose their minds
  655. in the Just Food for Dogs kitchen.
  656. And honestly, same, do better for your dog.
  657. They are obsessed.
  658. Stop by a Just Food for Dogs kitchen
  659. and claim a free meal at jffd.com/free.
  660. You're welcome.
  661. Tired of sugar-coated news and corporate spin?
  662. Check out the Nicole Sandler Show.
  663. Speaking truth to power with the irreverence it deserves.
  664. Smart, fearless, and just the right amount of snark,
  665. Nicole breaks down the headlines, exposes the hypocrisy,
  666. and talks with the people working for real change.
  667. Weekdays at 3 Eastern noon Pacific on Progressive Voices,
  668. nicolesandler.com, Substack, YouTube, or wherever you get your podcasts.
  669. Don't just get the news, get the truth.
  670. Hello, Martin Sheen here, and I have a question for you.
  671. What do you do on Sunday to relax and unwind?
  672. Sleep in, read the paper, eat stuff you shouldn't,
  673. binge watch the shows on your My Stuff list?
  674. Okay, all good and fine, but I think this routine could be a bit better.
  675. Let me offer something to make your me time really special, my podcast.
  676. And I think once you hear it, you'll realize it is the perfect addition to
  677. your Sunday or whenever you're looking for a chance to unwind.
  678. Tune in as I share the stories of my life along with some poetry and
  679. an occasional personal reflection in a way that will have you saying,
  680. that was time well spent, and I didn't even have to get out of my pajamas.
  681. Season four of the Martin Sheen podcast with yours truly Martin Sheen,
  682. of course, is underway with ten brand new episodes.
  683. I know our lives can be very busy and demanding these days, whose are not.
  684. So join me on a storytelling journey that you can enjoy from the comfort of
  685. your favorite easy chair and a chance to relax and be in the moment.
  686. So check out the Martin Sheen podcast, season four underway right now,
  687. and make it part of your self-care routine.
  688. And as always, thank you for listening.

Downloads

Recording audio · Timestamped transcript