What is a Data Desk?

By • • PyDataLA in USC

Slides

Show the extracted slide text

Slide 1

WHAT IS A
DATA DESK?
A brief tutorial

Slide 2

My name is Ben
 I sometimes go by @palewire

Slide 3

All my slides are belong to
lat.ms/whoisdatadesk

Slide 4

I work at @latimes in #GUNDO

Slide 5

Our team is called the Data Desk
       We turn databases into news

Slide 6

We write in code

Slide 7

Including English

Slide 12

Rosanna Xia
@rosannaxia

Slide 13

http://www.latimes.com/local/lanow/la-me-hollister-ranch-public-access-20181005-htmlst

Slide 14

Steve Lopez
@LATstevelopez

Slide 15

https://en.wikipedia.org/wiki/Williamson_Act

Slide 24

https://nbviewer.jupyter.org/github/datadesk/hollister-ranch-analysis/blob/master/src/parcels.ipynb

Slide 27

https://nbviewer.jupyter.org/github/datadesk/hollister-ranch-analysis/blob/master/src/download.ipynb

Slide 28

https://nbviewer.jupyter.org/github/datadesk/hollister-ranch-analysis/blob/master/notebook.ipynb

Slide 29

http://www.latimes.com/local/california/la-me-lopez-hollister-taxes-20181020-story.html

Slide 30

https://github.com/datadesk/notebooks

Slide 32

Andy Roberson
@andyroberson22

Slide 33

http://www.latimes.com/projects/la-sp-lakers-roster/

Slide 34

Ryan Menezes
@ryanvmenezes

Slide 36

https://beta.observablehq.com/@ryanvmenezes/nba-player-hex-calcs

Slide 42

We turn news into databases

Slide 44

Maloy Moore
@maloym

Slide 46

http://projects.latimes.com/wardead/name/daniel-r-parker/

Slide 47

http://homicide.latimes.com/

Slide 48

Nicole Santa Cruz
@nicolesantacruz

Slide 49

Iris Lee
@irisslee

Slide 50

http://homicide.latimes.com/post/bijan-michael-shoushtari/

Slide 51

http://homicide.latimes.com/post/bijan-michael-shoushtari/

Slide 52

http://homicide.latimes.com/post/autumn-johnson/

Slide 53

http://homicide.latimes.com/post/michael-jackson/

Slide 54

http://homicide.latimes.com/post/michael-jackson/

Slide 55

https://nbviewer.jupyter.org/github/datadesk/street-racing-analysis/blob/master/notebook.ipynb

Slide 56

We turn news into software

Slide 57

http://www.latimes.com/projects/la-pol-ca-california-primary-election-results-2018/

Slide 58

https://python-elections.readthedocs.io

Slide 59

https://elex.readthedocs.io/

Slide 60

http://rrcc.co.la.ca.us/elect/downrslt.html-ssi

Slide 62

Anthony Pesce
@anthonyjpesce

Slide 63

https://github.com/datadesk/la-election-night

Slide 64

https://github.com/datadesk/packages

Slide 65

And vice versa

Slide 66

http://cal-access.ss.ca.gov/

Slide 67

http://www.californiacivicdata.org/

Slide 68

http://www.latimes.com/projects/la-pol-ca-california-governor-2018-money/

Slide 69

http://calaccess.californiacivicdata.org/downloads/latest/

Slide 70

Ryan Menezes
@ryanvmenezes

Slide 71

Maloy Moore
@maloym

Slide 72

Seema Mehta
@LATseema

Slide 73

http://www.latimes.com/politics/la-pol-ca-newsom-cannabis-20170727-story.html

Slide 74

http://www.latimes.com/projects/la-pol-ca-california-governor-2018-money/

Slide 75

http://www.latimes.com/politics/la-pol-ca-antonio-villaraigosa-charter-school-money-governor-20180514-story.html

Slide 76

http://www.latimes.com/politics/la-pol-ca-governors-race-independent-expenditures-20180527-story.html

Slide 77

http://www.latimes.com/projects/la-pol-ca-gavin-newsom-san-francisco-money/

Slide 78

PLUS

Slide 79

BOTS

Slide 80

TOOLS

Slide 81

TOYS

Slide 82

THE
END

Slide 83

Follow us
www.latimes.com

@LATdatadesk


github.com/datadesk

Slide 84

All my slides are belong to
lat.ms/whoisdatadesk

Recording

Show the timestamped transcript
  1. okay welcome to everybody
  2. okay welcome to everybody I'm Nathan Dan
  3. I'm Nathan Dan co-chair and part of the goal of this
  4. co-chair and part of the goal of this conference is to showcase some of the
  5. conference is to showcase some of the great work happening in the Los Angeles
  6. great work happening in the Los Angeles data community I'm really happy to
  7. data community I'm really happy to introduce one of the last two keynotes
  8. introduce one of the last two keynotes who both of them are Los Angeles locals
  9. who both of them are Los Angeles locals doing great great things yet in very
  10. doing great great things yet in very different industries Ben Welsh is the
  11. different industries Ben Welsh is the editor of the Los Angeles Times data
  12. editor of the Los Angeles Times data desk a team of reporters and computer
  13. desk a team of reporters and computer programmers in the newsroom who who work
  14. programmers in the newsroom who who work to collect organize analyze and present
  15. to collect organize analyze and present large amounts of information Ben is also
  16. large amounts of information Ben is also co-founder of the California civic data
  17. co-founder of the California civic data coalition an open-source network of
  18. coalition an open-source network of developers working to open up public
  19. developers working to open up public data Ben is also creator of past pages
  20. data Ben is also creator of past pages an arc idea an archive dedicated to the
  21. an arc idea an archive dedicated to the preservation of online news Ben has
  22. preservation of online news Ben has worked at the Los Angeles Times and
  23. worked at the Los Angeles Times and since 2007 before working at the Times
  24. since 2007 before working at the Times been conducted data analysis for
  25. been conducted data analysis for investigative projects of the Senator
  26. investigative projects of the Senator Center for Public Integrity in
  27. Center for Public Integrity in Washington DC projects he has
  28. Washington DC projects he has contributed to have been awarded
  29. contributed to have been awarded numerous prizes including the Pulitzer I
  30. numerous prizes including the Pulitzer I met Ben through an education program he
  31. met Ben through an education program he was leading through the knight
  32. was leading through the knight foundation python for data journalists
  33. foundation python for data journalists analyzing money in politics the
  34. analyzing money in politics the educational program this program taught
  35. educational program this program taught journalists how to use Python pandas to
  36. journalists how to use Python pandas to Purdue notebooks and related Python
  37. Purdue notebooks and related Python tools to help investigate money in
  38. tools to help investigate money in politics
  39. politics another interesting fact I was just
  40. another interesting fact I was just browsing through github through Ben's
  41. browsing through github through Ben's github and I noticed that he has made it
  42. github and I noticed that he has made it accepted contributions to pandas Jupiter
  43. accepted contributions to pandas Jupiter lab Altair probably several others as a
  44. lab Altair probably several others as a conference chair I'm excited to hear
  45. conference chair I'm excited to hear about some of the tools that he that
  46. about some of the tools that he that this community supports and maintains
  47. this community supports and maintains are being utilized by the LA Times
  48. are being utilized by the LA Times please join please welcome me in joining
  49. please join please welcome me in joining Ben wells
  50. hello how are you guys good good
  51. hello how are you guys good good it took a minute to get set up because I
  52. it took a minute to get set up because I tried to plug my Linux computer into
  53. tried to plug my Linux computer into this fancy system you can guess how well
  54. this fancy system you can guess how well that went
  55. that went we're back on the Apple so this is me
  56. we're back on the Apple so this is me Ben Welsh like you heard this is what my
  57. Ben Welsh like you heard this is what my wife calls my Bible salesman photo which
  58. wife calls my Bible salesman photo which is definitely not a compliment
  59. is definitely not a compliment I go by pale wire online that's my
  60. I go by pale wire online that's my handle on most things that's a long
  61. handle on most things that's a long story I mean all the slides that I'm
  62. story I mean all the slides that I'm gonna walk through here today are
  63. gonna walk through here today are available for you online at this URL
  64. available for you online at this URL Whois data desk la tems Whois data desk
  65. Whois data desk la tems Whois data desk if you go there you can follow along you
  66. if you go there you can follow along you can breeze ahead you can share it with
  67. can breeze ahead you can share it with somebody else and also I'll point out in
  68. somebody else and also I'll point out in the lower corner here on the right will
  69. the lower corner here on the right will be little hyperlinks that will link off
  70. be little hyperlinks that will link off to every example I show cuz I'm gonna go
  71. to every example I show cuz I'm gonna go through a lot of stuff really fast and
  72. through a lot of stuff really fast and just to try to keep it interesting and
  73. just to try to keep it interesting and if there's anything you want to check
  74. if there's anything you want to check out you just stop go to this link and
  75. out you just stop go to this link and just look for the little blue link in
  76. just look for the little blue link in the corner okay you should be able to
  77. the corner okay you should be able to find it so I'm back at my first slide I
  78. find it so I'm back at my first slide I had the old LA Times headquarters
  79. had the old LA Times headquarters downtown that's actually not where we
  80. downtown that's actually not where we work anymore the LA Times has recently
  81. work anymore the LA Times has recently moved to El Segundo which I'm taken to
  82. moved to El Segundo which I'm taken to calling gundo and so if you drive down
  83. calling gundo and so if you drive down take the 105 to LAX you'll see a new
  84. take the 105 to LAX you'll see a new office that's where the database sits
  85. office that's where the database sits every day on the 6th floor they're doing
  86. every day on the 6th floor they're doing our job and what the heck do computer
  87. our job and what the heck do computer programmers do in a newsroom well they
  88. programmers do in a newsroom well they do a couple things and I'm gonna walk
  89. do a couple things and I'm gonna walk you through them alright and give you
  90. you through them alright and give you examples of each the first thing we do
  91. examples of each the first thing we do and we do a lot of this is we take data
  92. and we do a lot of this is we take data and for some reason this is like the
  93. and for some reason this is like the emoji for data no one's ever explained
  94. emoji for data no one's ever explained it to me you know what if you know what
  95. it to me you know what if you know what the heck that is please let me know I'm
  96. the heck that is please let me know I'm told it's a database right we take data
  97. told it's a database right we take data big chunks of information as our raw
  98. big chunks of information as our raw material they're our input right and
  99. material they're our input right and then we process those data and that data
  100. then we process those data and that data and we turn it into stories right and so
  101. and we turn it into stories right and so that's one big thing the data test does
  102. that's one big thing the data test does is we turn databases into news and that
  103. is we turn databases into news and that means we write code so even though I
  104. means we write code so even though I went to journalism school and I got
  105. went to journalism school and I got started in TV news and other people in
  106. started in TV news and other people in our team went to journalism school for
  107. our team went to journalism school for the most part or were librarians there
  108. the most part or were librarians there were other things we all write computer
  109. were other things we all write computer but and that includes Python like this
  110. but and that includes Python like this and a lot of other things but we still
  111. and a lot of other things but we still also write in English so even though
  112. also write in English so even though we're computer programmers that doesn't
  113. we're computer programmers that doesn't mean we're just like nerds in a cubicle
  114. mean we're just like nerds in a cubicle in the corner and we don't participate
  115. in the corner and we don't participate in making and telling stories we also
  116. in making and telling stories we also still do that too so let me tell you a
  117. still do that too so let me tell you a story of how that went but before we go
  118. story of how that went but before we go here's some late-breaking news numbers
  119. here's some late-breaking news numbers in the news there's a big number tonight
  120. in the news there's a big number tonight in the news does anyone know what it is
  121. 1.6 billion is the number in the news
  122. 1.6 billion is the number in the news today and that's because there will be a
  123. today and that's because there will be a Mega Millions jackpot for that amount
  124. Mega Millions jackpot for that amount all right and through this speech I'm
  125. all right and through this speech I'm going to be doing some quick trivia
  126. going to be doing some quick trivia questions some easy some hard and if you
  127. questions some easy some hard and if you shout out the correct answer you will
  128. shout out the correct answer you will win one ticket in tonight's Mega
  129. win one ticket in tonight's Mega Millions lotto for 1.6 billion dollars
  130. Millions lotto for 1.6 billion dollars all right and should you win should you
  131. all right and should you win should you win I don't expect anything all right
  132. win I don't expect anything all right but you should consider maybe writing a
  133. but you should consider maybe writing a check to numb focus if you have 1.6
  134. check to numb focus if you have 1.6 billion you might need the tax shelter
  135. billion you might need the tax shelter ok so let's start with an easy one this
  136. ok so let's start with an easy one this will be the easiest one what's on this
  137. will be the easiest one what's on this map California who was it
  138. map California who was it one ticket come collect all right
  139. there you are sir all right so that's an
  140. there you are sir all right so that's an easy one the Maps gonna zoom in a little
  141. easy one the Maps gonna zoom in a little closer
  142. closer it's now here what's this on this map no
  143. it's now here what's this on this map no nope what Santa Barbara County who said
  144. nope what Santa Barbara County who said it come get your ticket
  145. it come get your ticket okay so California has 58 counties I
  146. okay so California has 58 counties I know you guys are numbers people so
  147. know you guys are numbers people so there's 58 counties one of which is LA
  148. there's 58 counties one of which is LA County - the best County but maybe
  149. County - the best County but maybe second best if we're being generous
  150. second best if we're being generous might be Santa Barbara County which is
  151. might be Santa Barbara County which is just up the coast a little north of
  152. just up the coast a little north of where we are now he says it's pretty
  153. where we are now he says it's pretty cool I can't disagree and then if we
  154. cool I can't disagree and then if we zoom in a little closer here's a tougher
  155. zoom in a little closer here's a tougher question within Santa Barbara County if
  156. question within Santa Barbara County if we zoom in on this area right here we're
  157. we zoom in on this area right here we're gonna see that does anybody know what
  158. gonna see that does anybody know what that is hmm no no
  159. that is hmm no no no no no no
  160. no no no no so there's Gaviota here's your hint
  161. so there's Gaviota here's your hint right Gaviota is over here and you just
  162. right Gaviota is over here and you just keep going what's that it's something
  163. keep going what's that it's something ranch you can't win twice all right so
  164. ranch you can't win twice all right so we're moving on that it is actually
  165. we're moving on that it is actually Hollister ranch does anyone know what
  166. Hollister ranch does anyone know what Hollister ranch is this is a picture of
  167. Hollister ranch is this is a picture of it it's a it's a subdivision in the
  168. it it's a it's a subdivision in the mountains just west of Santa Barbara
  169. mountains just west of Santa Barbara City that is private it has private
  170. City that is private it has private roads and about a hundred and thirty
  171. roads and about a hundred and thirty houses including mansions populated by
  172. houses including mansions populated by people like James Cameron and the
  173. people like James Cameron and the founder of Patagonia and Jackson Browne
  174. founder of Patagonia and Jackson Browne and they have a beautiful subdivision
  175. and they have a beautiful subdivision with many many rich mansions and they
  176. with many many rich mansions and they have actually closed off the roads from
  177. have actually closed off the roads from the public including access to the
  178. the public including access to the public beaches like this beautiful one
  179. public beaches like this beautiful one here and if you read the LA Times you
  180. here and if you read the LA Times you can read stories by my colleague Rosanna
  181. can read stories by my colleague Rosanna who writes about environment and coastal
  182. who writes about environment and coastal access here in California
  183. access here in California and you'd receive stories like this like
  184. and you'd receive stories like this like she did recently and if you were to read
  185. she did recently and if you were to read that you would learn that you know in
  186. that you would learn that you know in California the public has a right to
  187. California the public has a right to access the beach right it's the beaches
  188. access the beach right it's the beaches public property but in this case the
  189. public property but in this case the road that goes to the beach is not right
  190. road that goes to the beach is not right and the residents of Hollister ranch
  191. and the residents of Hollister ranch have spent the last several decades
  192. have spent the last several decades trying to make sure that those roads
  193. trying to make sure that those roads don't get open to the public and the
  194. don't get open to the public and the only way you can really get to the beach
  195. only way you can really get to the beach is on horseback or by kayak all right
  196. is on horseback or by kayak all right and one of those people who's attempted
  197. and one of those people who's attempted to kayak to Hollister ranch on a few
  198. to kayak to Hollister ranch on a few occasions is our city columnist Steve
  199. occasions is our city columnist Steve Lopez who writes a lot about coastal
  200. Lopez who writes a lot about coastal access and he thinks that stuff like
  201. access and he thinks that stuff like that should be opened up and he's
  202. that should be opened up and he's curious to learn more about elite places
  203. curious to learn more about elite places like Hollister ranch and how they're run
  204. like Hollister ranch and how they're run so after Rosanna did this story recently
  205. so after Rosanna did this story recently Steve got a tip and then Steve called me
  206. Steve got a tip and then Steve called me on the phone and he said Ben have you
  207. on the phone and he said Ben have you ever heard of something called the
  208. ever heard of something called the Williamson Act I said Steve I have no
  209. Williamson Act I said Steve I have no goddamn clue what you're talking about
  210. goddamn clue what you're talking about and like any good scientist I went to
  211. and like any good scientist I went to Google and then to Wikipedia right and I
  212. Google and then to Wikipedia right and I looked it up when I read this and it
  213. looked it up when I read this and it says the Williamson Act is a law in the
  214. says the Williamson Act is a law in the state of California that says that
  215. state of California that says that certain agricultural lands that are
  216. certain agricultural lands that are certified by local county authorities
  217. certified by local county authorities so there's 58 different ways of doing it
  218. so there's 58 different ways of doing it right are able to be designated as
  219. right are able to be designated as agricultural something or other on
  220. agricultural something or other on state law and then given guess what a
  221. state law and then given guess what a hefty property tax break right and
  222. hefty property tax break right and Hollister ranch even though it's a
  223. Hollister ranch even though it's a subdivision said his tip it still also
  224. subdivision said his tip it still also classified as an agricultural because
  225. classified as an agricultural because there's a cattle operation run by two or
  226. there's a cattle operation run by two or three residents that are allow their
  227. three residents that are allow their cows to wander around everyone else's
  228. cows to wander around everyone else's property whether they participated in or
  229. property whether they participated in or not
  230. not sounds like a pretty good tip but how do
  231. sounds like a pretty good tip but how do you check that kind of thing out well
  232. you check that kind of thing out well this is where data can help so Steve and
  233. this is where data can help so Steve and I went to the County Assessors website
  234. I went to the County Assessors website and then we could actually called up a
  235. and then we could actually called up a guy who works in the appraisal division
  236. guy who works in the appraisal division there and we said do you guys track any
  237. there and we said do you guys track any of this you know like we got this tip we
  238. of this you know like we got this tip we heard this thing is it really true what
  239. heard this thing is it really true what is it what's about and the guy started
  240. is it what's about and the guy started walking us through the Assessors website
  241. walking us through the Assessors website and on this website there's a database
  242. and on this website there's a database and if you're like in our business on
  243. and if you're like in our business on the outside of these big data systems
  244. the outside of these big data systems unlike say hunter Owens earlier from LA
  245. unlike say hunter Owens earlier from LA City who's on the inside if you're on
  246. City who's on the inside if you're on the outside like us you're almost kind
  247. the outside like us you're almost kind of fumbling around we don't know what
  248. of fumbling around we don't know what got dated the government keeps and what
  249. got dated the government keeps and what it released in there and who has it or
  250. it released in there and who has it or da-da-da-da-da but what we love our web
  251. da-da-da-da-da but what we love our web forms right cuz web forms signify
  252. forms right cuz web forms signify there's a database and we started
  253. there's a database and we started interviewing this guy and we're like
  254. interviewing this guy and we're like well what about like James Cameron's
  255. well what about like James Cameron's house or whoever just like pick one the
  256. house or whoever just like pick one the guy came up with a parcel number which
  257. guy came up with a parcel number which is the unique ID for a property you
  258. is the unique ID for a property you punch it in there and he pointed out
  259. punch it in there and he pointed out this site to us you know and this is
  260. this site to us you know and this is what's called the value notice which is
  261. what's called the value notice which is published online for every property in
  262. published online for every property in Santa Barbara County and if you qualify
  263. Santa Barbara County and if you qualify for the Williamson Act someone someone
  264. for the Williamson Act someone someone perhaps anticipating what Steve Lopez
  265. perhaps anticipating what Steve Lopez would like to do right had created this
  266. would like to do right had created this table that exists on every parcels page
  267. table that exists on every parcels page which says hey under normal property tax
  268. which says hey under normal property tax rules and is the state of California a
  269. rules and is the state of California a property is assessed under what's called
  270. property is assessed under what's called prop 13 right under normal property tax
  271. prop 13 right under normal property tax rules this property would be assessed at
  272. rules this property would be assessed at seven hundred and fifty thousand dollars
  273. seven hundred and fifty thousand dollars and it's property tax would be one point
  274. and it's property tax would be one point one percent of that approximately right
  275. one percent of that approximately right but because it was called agricultural
  276. but because it was called agricultural and sorry this is kind of grainy here
  277. and sorry this is kind of grainy here that number went down from 750,000 to
  278. that number went down from 750,000 to only three hundred and thirty eight
  279. only three hundred and thirty eight thousand whoa that cut this person's
  280. thousand whoa that cut this person's property tax in half right
  281. property tax in half right I wonder if everybody in Hollister ranch
  282. I wonder if everybody in Hollister ranch whether they participate in ranching or
  283. whether they participate in ranching or not get that kind of benefit right well
  284. not get that kind of benefit right well how do we figure it out so that there's
  285. how do we figure it out so that there's Hollister ranch so we go back to this
  286. Hollister ranch so we go back to this map you may have noticed that this map
  287. map you may have noticed that this map didn't look as nice as the previous ones
  288. didn't look as nice as the previous ones because those other ones came from
  289. because those other ones came from Google this one came from Mike huges as
  290. Google this one came from Mike huges as ugly as it is it is a beautiful tool
  291. ugly as it is it is a beautiful tool open-source software that allows you to
  292. open-source software that allows you to take electronic map files called
  293. take electronic map files called shapefile so you guys know all this
  294. shapefile so you guys know all this stuff I normally talk to journalists
  295. stuff I normally talk to journalists I'll just stop with that
  296. I'll just stop with that QGIS this is a file I got from guess
  297. QGIS this is a file I got from guess what the county surveyors website where
  298. what the county surveyors website where I was nosing around and hey there's a
  299. I was nosing around and hey there's a shapefile that has all the subdivisions
  300. shapefile that has all the subdivisions when I looked in the attributes I saw
  301. when I looked in the attributes I saw hey these are marked as Hollister ranch
  302. hey these are marked as Hollister ranch then I knows around a little more and I
  303. then I knows around a little more and I found they have an FTP online oh my gosh
  304. found they have an FTP online oh my gosh and the FTP has every parcel right okay
  305. and the FTP has every parcel right okay this is I get excited guys and then I
  306. this is I get excited guys and then I put that in my queue just so now here's
  307. put that in my queue just so now here's every parcel in Santa Barbara County and
  308. every parcel in Santa Barbara County and there's house to ranch if now if I'm
  309. there's house to ranch if now if I'm going to do this analysis I need to like
  310. going to do this analysis I need to like cookie-cutter these guys out right back
  311. cookie-cutter these guys out right back when I first started doing this stuff I
  312. when I first started doing this stuff I might have tried to use the queue just
  313. might have tried to use the queue just plug in waited 45 minutes it would have
  314. plug in waited 45 minutes it would have crashed in I mean I would have bitched
  315. crashed in I mean I would have bitched and moaned or whatever but now thanks to
  316. and moaned or whatever but now thanks to y'all in my Jupiter notebook I import
  317. y'all in my Jupiter notebook I import geo pandas alright have you guys have
  318. geo pandas alright have you guys have seen geo pandas it's like pandas with
  319. seen geo pandas it's like pandas with the geospatial stuff so I'm able to take
  320. the geospatial stuff so I'm able to take one shape file of all the parcels and
  321. one shape file of all the parcels and another of the subdivisions filter the
  322. another of the subdivisions filter the subdivisions down do a spatial join get
  323. subdivisions down do a spatial join get the subset of 136 or whatever parcels
  324. the subset of 136 or whatever parcels and house tranche
  325. and house tranche sorry but then you may have noticed if
  326. sorry but then you may have noticed if you look carefully at this file that was
  327. you look carefully at this file that was in the FTP it had AB suffix underscored
  328. in the FTP it had AB suffix underscored no owners so I was able to confined all
  329. no owners so I was able to confined all the parcels in Hollister ranch but I
  330. the parcels in Hollister ranch but I didn't know who owned him and if we're
  331. didn't know who owned him and if we're gonna write the story we got to know
  332. gonna write the story we got to know which ones James Cameron and which ones
  333. which ones James Cameron and which ones nada
  334. nada so I emailed the guy names have been
  335. so I emailed the guy names have been redacted to protect the innocent okay so
  336. redacted to protect the innocent okay so I emailed the guy at the county and I
  337. I emailed the guy at the county and I said dude where are the
  338. said dude where are the can I have the owners what's that uh I
  339. can I have the owners what's that uh I should be using the clicker anyway and
  340. should be using the clicker anyway and so here's the email he sent me back and
  341. so here's the email he sent me back and you know this stuff is public record
  342. you know this stuff is public record right the data should be out there but
  343. right the data should be out there but this is the kind of thing that happens
  344. this is the kind of thing that happens in real life when you're like doing what
  345. in real life when you're like doing what we do the guy says well actually Ben we
  346. we do the guy says well actually Ben we cannot send you the ownership records
  347. cannot send you the ownership records and they cannot be published online
  348. and they cannot be published online because there's a state law that says
  349. because there's a state law that says that public officials names and
  350. that public officials names and addresses cannot be published over the
  351. addresses cannot be published over the Internet
  352. Internet everybody else is in trouble and our
  353. everybody else is in trouble and our interpretation of that law
  354. interpretation of that law administratively is that no addresses or
  355. administratively is that no addresses or owners can be published online at all
  356. owners can be published online at all right and this is the sort of thing that
  357. right and this is the sort of thing that just like makes me drives me crazy
  358. just like makes me drives me crazy and as we accumulate as we try to ask
  359. and as we accumulate as we try to ask for data like this from different
  360. for data like this from different government offices what we often find is
  361. government offices what we often find is they often have different
  362. they often have different interpretations of these laws and
  363. interpretations of these laws and depending on who you're talking to
  364. depending on who you're talking to you're going to get different stuff
  365. you're going to get different stuff their office isn't Santa Barbara I'm an
  366. their office isn't Santa Barbara I'm an El Segundo oh my god they want me to
  367. El Segundo oh my god they want me to drive up there pay them 50 bucks to get
  368. drive up there pay them 50 bucks to get a CDR and drive back down and do the
  369. a CDR and drive back down and do the merge right and so unlike all you guys
  370. merge right and so unlike all you guys I'm not very nice so then I just start
  371. I'm not very nice so then I just start banging the table and you can see I was
  372. banging the table and you can see I was like so angry writing this email I
  373. like so angry writing this email I accidentally put a question mark at the
  374. accidentally put a question mark at the end of the first sentence which I didn't
  375. end of the first sentence which I didn't even mean to do but but after I sent it
  376. even mean to do but but after I sent it I was like yeah I did mean that you know
  377. I was like yeah I did mean that you know I mean even though it was totally
  378. I mean even though it was totally accidental there's like a typo blah blah
  379. accidental there's like a typo blah blah blah and so then we begin a process
  380. blah and so then we begin a process where we're banging on the table we
  381. where we're banging on the table we begin calling all the politicians hey
  382. begin calling all the politicians hey give us the owners ultimately I did have
  383. give us the owners ultimately I did have to cut him a check which I charged to
  384. to cut him a check which I charged to our new billionaire owner and they sent
  385. our new billionaire owner and they sent it to me via Dropbox file requests which
  386. it to me via Dropbox file requests which is an awesome feature by the way and
  387. is an awesome feature by the way and then now that I had all the owners I had
  388. then now that I had all the owners I had a complete dataset of all the parcels in
  389. a complete dataset of all the parcels in Hollister ranch and who owns them but I
  390. Hollister ranch and who owns them but I needed those value notices that were in
  391. needed those value notices that were in that weird webpage they didn't have the
  392. that weird webpage they didn't have the database for that so here comes my
  393. database for that so here comes my Jupiter notebook again right and some
  394. Jupiter notebook again right and some like a scraper now web scraping is like
  395. like a scraper now web scraping is like if we had a conference like this for
  396. if we had a conference like this for news nerds like in my robe which we do
  397. news nerds like in my robe which we do web scraping is like the number one most
  398. web scraping is like the number one most popular thing everybody loves this this
  399. popular thing everybody loves this this is of two tool and feature in Python
  400. is of two tool and feature in Python that is absolutely crucial to what we do
  401. that is absolutely crucial to what we do because so much data is on the internet
  402. because so much data is on the internet but not in a format that we can like get
  403. but not in a format that we can like get at and how we want and so tools like
  404. at and how we want and so tools like this like requests and beautifulsoup
  405. this like requests and beautifulsoup which I love and other things are really
  406. which I love and other things are really important and just to prove that I put
  407. important and just to prove that I put this up just to prove I'm not totally a
  408. this up just to prove I'm not totally a bad guy
  409. bad guy I did put a user agent letting them know
  410. I did put a user agent letting them know who
  411. who why was and there's like a little time
  412. why was and there's like a little time where it sleeps ten seconds between
  413. where it sleeps ten seconds between requests I'm not a bad guy right and so
  414. requests I'm not a bad guy right and so then we downloaded that parse the data
  415. then we downloaded that parse the data out merged it and at the end of the day
  416. out merged it and at the end of the day ultimately the data analysis that we did
  417. ultimately the data analysis that we did is is seen like in in maybe 10 or 10 10
  418. is is seen like in in maybe 10 or 10 10 or 15 lines of Python and it's it's math
  419. or 15 lines of Python and it's it's math that's so simple that I'm almost
  420. that's so simple that I'm almost embarrassed to show it to this room you
  421. embarrassed to show it to this room you know what I mean because you guys are
  422. know what I mean because you guys are doing all this really sophisticated
  423. doing all this really sophisticated stuff but at the end of the day after we
  424. stuff but at the end of the day after we had been able to get all the parcels and
  425. had been able to get all the parcels and all the value notices for the hundred
  426. all the value notices for the hundred and thirty whatever parcels it really
  427. and thirty whatever parcels it really was just calculating the difference in
  428. was just calculating the difference in the property tax break right for all
  429. the property tax break right for all those properties summing it up right and
  430. those properties summing it up right and then calculating basically the
  431. then calculating basically the percentage you know and so then roughly
  432. percentage you know and so then roughly speaking the property tax of all these
  433. speaking the property tax of all these millionaires who live in these huge
  434. millionaires who live in these huge coastal elite mansions and deny you
  435. coastal elite mansions and deny you access to the beach is a 50% property
  436. access to the beach is a 50% property tax break which adds up to about two
  437. tax break which adds up to about two million dollars last year and will
  438. million dollars last year and will continue to add up each year and then
  439. continue to add up each year and then that ultimately comes out as this
  440. that ultimately comes out as this because you know you or I could maybe
  441. because you know you or I could maybe read this but it lacks context your
  442. read this but it lacks context your mother probably doesn't read Python code
  443. mother probably doesn't read Python code I know mine doesn't but my mother
  444. I know mine doesn't but my mother definitely reads Steve Lopez and so we
  445. definitely reads Steve Lopez and so we then took that data analysis and put it
  446. then took that data analysis and put it into a relatively traditional form which
  447. into a relatively traditional form which was Steve's column and he sort of
  448. was Steve's column and he sort of unfurled the the analysis and all the
  449. unfurled the the analysis and all the context and this was in last Sunday's
  450. context and this was in last Sunday's newspaper and so that's a way and this
  451. newspaper and so that's a way and this column is an example of how all the open
  452. column is an example of how all the open source tools that you guys work on and
  453. source tools that you guys work on and that we depend on in our business end up
  454. that we depend on in our business end up creating kind of a pretty traditional
  455. creating kind of a pretty traditional kind of like investigative story right
  456. kind of like investigative story right and and but we do other things that
  457. and and but we do other things that aren't so traditional that are that
  458. aren't so traditional that are that follow the same pattern
  459. follow the same pattern Oh for this conference actually here's I
  460. Oh for this conference actually here's I created a new repository which we just
  461. created a new repository which we just open sourced over the weekend which has
  462. open sourced over the weekend which has all of the Jupiter notebooks we've
  463. all of the Jupiter notebooks we've released for stories like this including
  464. released for stories like this including Hollister ranch if you go to data desk
  465. Hollister ranch if you go to data desk slash notebooks on github you could find
  466. slash notebooks on github you could find every single one of them only some of
  467. every single one of them only some of them buy me a lot of them by my
  468. them buy me a lot of them by my colleagues as well but another example
  469. colleagues as well but another example from the past week of us taking data and
  470. from the past week of us taking data and turn it into news involve this guy pop
  471. turn it into news involve this guy pop quiz who is he okay you can't win for
  472. quiz who is he okay you can't win for that
  473. that - easy even I you know I love a better
  474. - easy even I you know I love a better basketball one and so Andy Andy Roberson
  475. basketball one and so Andy Andy Roberson who's on our team she had been charged
  476. who's on our team she had been charged with creating for LA Times comm a Lakers
  477. with creating for LA Times comm a Lakers roster to help our readers get to know
  478. roster to help our readers get to know all the different people on the Lakers
  479. all the different people on the Lakers and had little videos for each of them
  480. and had little videos for each of them and their BIOS and stats and stuff about
  481. and their BIOS and stats and stuff about how tall they are and where their foot
  482. how tall they are and where their foot college they went to and all that stuff
  483. college they went to and all that stuff and she made a really nice page but we
  484. and she made a really nice page but we kind of felt like oh I could use like a
  485. kind of felt like oh I could use like a little more data could use like a little
  486. little more data could use like a little more like oomph going on in the page and
  487. more like oomph going on in the page and so that's where Ryan Manassas came in
  488. so that's where Ryan Manassas came in who's sitting right back over there and
  489. who's sitting right back over there and and he went and developed a data set
  490. and he went and developed a data set that had Ryan just correct me where I
  491. that had Ryan just correct me where I get this wrong basically the location on
  492. get this wrong basically the location on the court where all the players in the
  493. the court where all the players in the NBA shoot from and how well they shoot
  494. NBA shoot from and how well they shoot and how many shots they made from each
  495. and how many shots they made from each of those positions he then fed that into
  496. of those positions he then fed that into a different style of notebook which is
  497. a different style of notebook which is called an observable notebook who here
  498. called an observable notebook who here has seen observable notebooks he got to
  499. has seen observable notebooks he got to check out this website observable hq.com
  500. check out this website observable hq.com it's by Mike Bostock the creator of d3
  501. it's by Mike Bostock the creator of d3 and Jeremy Ashkan ashed creator of a lot
  502. and Jeremy Ashkan ashed creator of a lot of other cool programming stuff and
  503. of other cool programming stuff and they're attempting to create a
  504. they're attempting to create a JavaScript notebook that's basically in
  505. JavaScript notebook that's basically in the cloud that is is already a
  506. the cloud that is is already a competitor to stuff like Jupiter and is
  507. competitor to stuff like Jupiter and is an incredible product for for really
  508. an incredible product for for really rapidly creating interfaces and
  509. rapidly creating interfaces and visualizations from data in like
  510. visualizations from data in like cutting-edge JavaScript I would really
  511. cutting-edge JavaScript I would really recommend checking it out and Ryan was
  512. recommend checking it out and Ryan was able to with just a little bit of code
  513. able to with just a little bit of code in this notebook take that data file we
  514. in this notebook take that data file we looked at before wired into a d3 hex bin
  515. looked at before wired into a d3 hex bin system and create different shot charts
  516. system and create different shot charts for every player in the NBA showing
  517. for every player in the NBA showing where they shoot from and how they made
  518. where they shoot from and how they made it we did a little bit of work to style
  519. it we did a little bit of work to style it out in the Lakers colors week sport
  520. it out in the Lakers colors week sport the SVG from that we move it into the
  521. the SVG from that we move it into the website and then here for each player
  522. website and then here for each player when you click on them you get not just
  523. when you click on them you get not just the bio and the video you get also their
  524. the bio and the video you get also their shot chart right so there's Brendan
  525. shot chart right so there's Brendan Ingram and so this is a case where we're
  526. Ingram and so this is a case where we're using that data to help make something
  527. using that data to help make something that's untraditional it's not a
  528. that's untraditional it's not a newspaper article right it's like a web
  529. newspaper article right it's like a web feature it'll never appear in the print
  530. feature it'll never appear in the print paper and our team tries to really do
  531. paper and our team tries to really do both of those things so here's LeBron
  532. both of those things so here's LeBron Jameses shot chart right
  533. Jameses shot chart right LeBron he's good alright you may have
  534. LeBron he's good alright you may have heard all right next quiz who's shot
  535. heard all right next quiz who's shot chart is this
  536. JaVale McGee who said it there he is
  537. JaVale McGee who said it there he is JaVale looks pretty good so far
  538. JaVale looks pretty good so far this year yeah he I've been impressed
  539. this year yeah he I've been impressed especially defensively offensively he's
  540. especially defensively offensively he's got room to improve all right so we
  541. got room to improve all right so we don't just turn turn data into news we
  542. don't just turn turn data into news we also often do the reverse this is the
  543. also often do the reverse this is the second type of thing we do and so I want
  544. second type of thing we do and so I want to walk you through a couple examples of
  545. to walk you through a couple examples of this and that might sound glib just
  546. this and that might sound glib just flipping it around but I'm serious so
  547. flipping it around but I'm serious so here's an example it's a serious example
  548. here's an example it's a serious example so when I first started at the LA Times
  549. so when I first started at the LA Times in 2007 there was the wars in Iraq and
  550. in 2007 there was the wars in Iraq and Afghanistan were going pretty hot and
  551. Afghanistan were going pretty hot and heavy and the LA Times had already
  552. heavy and the LA Times had already committed at that point to write an
  553. committed at that point to write an obituary of every Californian who died
  554. obituary of every Californian who died in either theater and here's an example
  555. in either theater and here's an example of one and this if you start if you put
  556. of one and this if you start if you put on what I call when I talk to the
  557. on what I call when I talk to the normies your database glasses which I'm
  558. normies your database glasses which I'm sure there's no trouble for you guys and
  559. sure there's no trouble for you guys and you look at content like this from our
  560. you look at content like this from our website you could begin to sort of guess
  561. website you could begin to sort of guess kind of what database fields and columns
  562. kind of what database fields and columns are behind a system like this right
  563. are behind a system like this right a new story like this will have a
  564. a new story like this will have a headline right that'll be one column
  565. headline right that'll be one column they'll be like that little sub headline
  566. they'll be like that little sub headline sometimes called the deck head in
  567. sometimes called the deck head in newspaper slang there's the publication
  568. newspaper slang there's the publication date that'll be a column that'll be
  569. date that'll be a column that'll be date-time format right and then what the
  570. date-time format right and then what the journalists think is the most important
  571. journalists think is the most important column the byline all right with their
  572. column the byline all right with their name and then in almost all news
  573. name and then in almost all news databases after stuff like that there's
  574. databases after stuff like that there's just another one that's just the big
  575. just another one that's just the big blob of text right which is all the
  576. blob of text right which is all the English that we put into it just as one
  577. English that we put into it just as one big crazy blob of text and when I had
  578. big crazy blob of text and when I had arrived the LA Times had already written
  579. arrived the LA Times had already written hundreds of obituaries just like that of
  580. hundreds of obituaries just like that of each person and I was looking for a data
  581. each person and I was looking for a data project to hook to start off with and I
  582. project to hook to start off with and I was looking for people to hook up with
  583. was looking for people to hook up with and we decided what if we were to build
  584. and we decided what if we were to build the database of these obituaries rather
  585. the database of these obituaries rather than just publish them as big blobs of
  586. than just publish them as big blobs of text and forget about them and if you
  587. text and forget about them and if you keep those database glasses on and you
  588. keep those database glasses on and you begin to read a story like this you can
  589. begin to read a story like this you can see the other columns that would be very
  590. see the other columns that would be very easy to fill in for many of these like
  591. easy to fill in for many of these like if we just look at the lead here we can
  592. if we just look at the lead here we can see we have the person's branch
  593. see we have the person's branch they're ranked their hometown how they
  594. they're ranked their hometown how they died where they died right first middle
  595. died where they died right first middle last name age high school right
  596. last name age high school right marital status number of kids that had a
  597. marital status number of kids that had a gender right go on and on all those
  598. gender right go on and on all those things could be classified and so what
  599. things could be classified and so what we did is we pulled out all that what
  600. we did is we pulled out all that what would you guys call it in your slang
  601. would you guys call it in your slang labels or categories I just call them
  602. labels or categories I just call them columns or fields right okay good I'm
  603. columns or fields right okay good I'm learning a lot of jargon here today my
  604. learning a lot of jargon here today my favorite so far is fraud II if something
  605. favorite so far is fraud II if something has like a high score on the fraud
  606. has like a high score on the fraud system and so Malloy more on our team
  607. system and so Malloy more on our team who actually is a veteran herself and an
  608. who actually is a veteran herself and an educated librarian did that basically
  609. educated librarian did that basically and we used another open-source system
  610. and we used another open-source system that I don't hear as much about in this
  611. that I don't hear as much about in this conference but is essential to us which
  612. conference but is essential to us which is would anyone know what this is Django
  613. is would anyone know what this is Django it's the Django I mean you can't wait
  614. it's the Django I mean you can't wait twice it's the Django as a web framework
  615. twice it's the Django as a web framework that it's kind of from the website of
  616. that it's kind of from the website of Python that was actually invented at the
  617. Python that was actually invented at the lawrence journal world newspaper and was
  618. lawrence journal world newspaper and was used to launch websites ultimately like
  619. used to launch websites ultimately like instagram pinterest and many others use
  620. instagram pinterest and many others use it as like a framework for development
  621. it as like a framework for development on the back end but one of the great
  622. on the back end but one of the great features of it is as an instant admin so
  623. features of it is as an instant admin so if you design a database table that has
  624. if you design a database table that has like the columns we were just talking
  625. like the columns we were just talking about it will create this generic admin
  626. about it will create this generic admin with no code from you it can just like
  627. with no code from you it can just like infer it and do it and for what we do
  628. infer it and do it and for what we do which is often assembling databases that
  629. which is often assembling databases that don't exist before out of the news right
  630. don't exist before out of the news right a system like this to get a data entry
  631. a system like this to get a data entry platform like up and running is just
  632. platform like up and running is just huge right and so we put this is it we
  633. huge right and so we put this is it we built the system and here's that same
  634. built the system and here's that same guy in the same story first middle last
  635. guy in the same story first middle last name gender age unique slug hometown and
  636. name gender age unique slug hometown and when we had done that and assembled for
  637. when we had done that and assembled for all of them were able to do a lot more
  638. all of them were able to do a lot more we were able to build a website that had
  639. we were able to build a website that had a page for every single person and you
  640. a page for every single person and you could click the link of the hometown and
  641. could click the link of the hometown and you could see everyone who's died from
  642. you could see everyone who's died from that hometown and we could publish these
  643. that hometown and we could publish these pages before the obituaries were
  644. pages before the obituaries were finished get the page out there on the
  645. finished get the page out there on the web early get comments from people who
  646. web early get comments from people who knew them on the page and in turn turn
  647. knew them on the page and in turn turn that data into a product it's not just a
  648. that data into a product it's not just a traditional story but also after you've
  649. traditional story but also after you've accrued it you're also to then able to
  650. accrued it you're also to then able to mine that database you've you've created
  651. mine that database you've you've created for all sorts of enterprise stories one
  652. for all sorts of enterprise stories one we're counting them we know when the 500
  653. we're counting them we know when the 500 person died we can integrate population
  654. person died we can integrate population data and tell you what hometown has had
  655. data and tell you what hometown has had the most deaths per capita we could tell
  656. the most deaths per capita we could tell you what cemetery has the most guys
  657. you what cemetery has the most guys buried at it and then we can go write a
  658. buried at it and then we can go write a Memorial Day story there and we can take
  659. Memorial Day story there and we can take a more structured and you know numerate
  660. a more structured and you know numerate approach to how we do a lot of
  661. approach to how we do a lot of traditional stuff right and that's
  662. traditional stuff right and that's because we took the news and we turned
  663. because we took the news and we turned it into the database that didn't exist
  664. it into the database that didn't exist another example of that is the homicide
  665. another example of that is the homicide report at homicide at LA Times comm
  666. report at homicide at LA Times comm which is the database that's kept by a
  667. which is the database that's kept by a few people on our team mainly Nicole
  668. few people on our team mainly Nicole Santa Cruz who's like that correspondent
  669. Santa Cruz who's like that correspondent and writer who does a lot of it and the
  670. and writer who does a lot of it and the iris Lee who does a lot of the computer
  671. iris Lee who does a lot of the computer programming web development stuff and
  672. programming web development stuff and that also has a Django admin where we're
  673. that also has a Django admin where we're taking the source data which is just a
  674. taking the source data which is just a list of people who've been killed from
  675. list of people who've been killed from the county coroner's office and we're
  676. the county coroner's office and we're enriching that you know we only give us
  677. enriching that you know we only give us a couple data columns really not a lot
  678. a couple data columns really not a lot we're putting that in and then we're
  679. we're putting that in and then we're adding more and more stuff from our
  680. adding more and more stuff from our reporting and we're accruing over time
  681. reporting and we're accruing over time this this sort of unique original
  682. this this sort of unique original database about everyone who's been
  683. database about everyone who's been killed by another person and in the same
  684. killed by another person and in the same way we can we can use Django and a
  685. way we can we can use Django and a templated web framework to build a page
  686. templated web framework to build a page for every every person including this
  687. for every every person including this guy who you probably don't know this
  688. guy who you probably don't know this person who you probably don't know but
  689. person who you probably don't know but you should and this person you
  690. you should and this person you definitely do know right and so the
  691. definitely do know right and so the templating system and using even the
  692. templating system and using even the sparse data we have to push it out
  693. sparse data we have to push it out through the system allows us to create a
  694. through the system allows us to create a page for every single person who's died
  695. page for every single person who's died which sadly in many cases is the only
  696. which sadly in many cases is the only notice of the the murder of many of
  697. notice of the the murder of many of these people there's so little coverage
  698. these people there's so little coverage and then it becomes the kind of a
  699. and then it becomes the kind of a baseline from which we can try to reach
  700. baseline from which we can try to reach and do more elevated and insightful work
  701. and do more elevated and insightful work a recent example would be this story
  702. a recent example would be this story Nicole did about all the people who died
  703. Nicole did about all the people who died from street racing in LA County which is
  704. from street racing in LA County which is actually a lot of people where we took
  705. actually a lot of people where we took our homicide report data we connected it
  706. our homicide report data we connected it with some other data we did and we were
  707. with some other data we did and we were able to analyze what happens and like
  708. able to analyze what happens and like one key finding here is that the
  709. one key finding here is that the majority of people who do this who died
  710. majority of people who do this who died in these cases actually are passengers
  711. in these cases actually are passengers they're not the drivers which is said
  712. they're not the drivers which is said you read the story it's good and that
  713. you read the story it's good and that also is a Jupiter notebook that you can
  714. also is a Jupiter notebook that you can get
  715. get and besides stories we also take the
  716. and besides stories we also take the news and turn it into software which
  717. news and turn it into software which then turns into more stories so like we
  718. then turns into more stories so like we were doing a lecture
  719. were doing a lecture results you know the live election
  720. results you know the live election results you see on election night and
  721. results you see on election night and all the fancy maps and charts in every
  722. all the fancy maps and charts in every news organization this was ours from the
  723. news organization this was ours from the California primary and every single one
  724. California primary and every single one of those pages on every single news site
  725. of those pages on every single news site you look at the data all comes from the
  726. you look at the data all comes from the same source The Associated Press so you
  727. same source The Associated Press so you don't have to worry about thinking once
  728. don't have to worry about thinking once faster than the other better than the
  729. faster than the other better than the other they're all the exact identical
  730. other they're all the exact identical source that everybody pays in for and
  731. source that everybody pays in for and that data service actually has some open
  732. that data service actually has some open source wrappers built around it here's
  733. source wrappers built around it here's one we built a few years ago which was
  734. one we built a few years ago which was then succeeded by this and this is a
  735. then succeeded by this and this is a case of people in our news nerd world
  736. case of people in our news nerd world collaborating around the common problems
  737. collaborating around the common problems we have which are often gnarly datasets
  738. we have which are often gnarly datasets you know what I mean and gnarly datasets
  739. you know what I mean and gnarly datasets that need to be like refined and worked
  740. that need to be like refined and worked on and turned into data pipelines are
  741. on and turned into data pipelines are like the number one need in our news
  742. like the number one need in our news nerd world for open source to help solve
  743. nerd world for open source to help solve problems so that we can collaborate to
  744. problems so that we can collaborate to get it done like you know and so here's
  745. get it done like you know and so here's La County's crappy site the AP doesn't
  746. La County's crappy site the AP doesn't have LA County results LA County has
  747. have LA County results LA County has this site which you might think oh it
  748. this site which you might think oh it could be worse Ben but then you crack
  749. could be worse Ben but then you crack open the data file and here it is it's
  750. open the data file and here it is it's not just a fixed-width file it's a
  751. not just a fixed-width file it's a fixed-width file with different widths
  752. fixed-width file with different widths depending on what type of line that
  753. depending on what type of line that you're on and so Anthony Pesci and our
  754. you're on and so Anthony Pesci and our team did the miserable work of like
  755. team did the miserable work of like actually figuring out how to parse that
  756. actually figuring out how to parse that file to do that whatever you would call
  757. file to do that whatever you would call a variable fixed width or whatever that
  758. a variable fixed width or whatever that is and then we just yesterday released
  759. is and then we just yesterday released that as an open source library so that
  760. that as an open source library so that hopefully nobody else has to do that
  761. hopefully nobody else has to do that like gnarly parse again and when we have
  762. like gnarly parse again and when we have to do it again in two years we'll
  763. to do it again in two years we'll remember because we like put it into a
  764. remember because we like put it into a package right and if you're interested
  765. package right and if you're interested in our open source work I made another
  766. in our open source work I made another repo data desk slash packages which has
  767. repo data desk slash packages which has the more than 30 open source
  768. the more than 30 open source repositories that our team has put out
  769. repositories that our team has put out and maintains you'll notice that a
  770. and maintains you'll notice that a really large majority of them have to do
  771. really large majority of them have to do with dealing with gnarly data sets that
  772. with dealing with gnarly data sets that we go to a lot and then those that those
  773. we go to a lot and then those that those packages often then turn into stories so
  774. packages often then turn into stories so like anybody know what this side is
  775. what's in it though what's on it what's
  776. what's in it though what's on it what's in here now now not this this one but
  777. in here now now not this this one but this is called Cal access anybody ever
  778. this is called Cal access anybody ever been here this is my least favorite
  779. been here this is my least favorite website campaign-finance so all the
  780. website campaign-finance so all the money that goes into the governor's race
  781. money that goes into the governor's race and all the propositions and all the
  782. and all the propositions and all the State House races as opposed to Congress
  783. State House races as opposed to Congress and the federal stuff all the state
  784. and the federal stuff all the state races all that all that money is
  785. races all that all that money is reported into an awful database kept by
  786. reported into an awful database kept by the Secretary of State called Cal access
  787. the Secretary of State called Cal access and this is a really important database
  788. and this is a really important database but nobody does any analysis of it
  789. but nobody does any analysis of it because it's a it's a mess it's a it's
  790. because it's a it's a mess it's a it's awful right and so that's where we
  791. awful right and so that's where we created this group the California civic
  792. created this group the California civic data coalition to create a github
  793. data coalition to create a github pipeline that would then take that data
  794. pipeline that would then take that data and you know and smooth it out and
  795. and you know and smooth it out and format it into something that's usable
  796. format it into something that's usable and this is an ongoing project we have
  797. and this is an ongoing project we have with more than 150 contributors around
  798. with more than 150 contributors around the world and it's resulted in this
  799. the world and it's resulted in this still improving website where we create
  800. still improving website where we create clean documented CSVs that like a normal
  801. clean documented CSVs that like a normal person can just pull down the source
  802. person can just pull down the source data from the government is eighty-eight
  803. data from the government is eighty-eight tables 35 million records no
  804. tables 35 million records no documentation about 40 the tables are
  805. documentation about 40 the tables are defunct never been labeled the joins are
  806. defunct never been labeled the joins are all screwed it's such a mess so we're
  807. all screwed it's such a mess so we're trying to solve that and because we've
  808. trying to solve that and because we've got that far and we have these
  809. got that far and we have these simplified versions it's helping Ryan
  810. simplified versions it's helping Ryan and Malloy who you saw earlier team with
  811. and Malloy who you saw earlier team with people like Seema Mehta one of our
  812. people like Seema Mehta one of our political reporters to do more stories
  813. political reporters to do more stories about money in politics that are more
  814. about money in politics that are more ambitious analysis than would have been
  815. ambitious analysis than would have been done because if you got to spend six
  816. done because if you got to spend six months doing the analysis you're only
  817. months doing the analysis you're only gonna get that one story and do it again
  818. gonna get that one story and do it again the next time it's gonna take six months
  819. the next time it's gonna take six months again but we've reduced like the barrier
  820. again but we've reduced like the barrier to getting into that data and we're
  821. to getting into that data and we're trying to more and more which has allow
  822. trying to more and more which has allow us to do stories like Gavin Newsom being
  823. us to do stories like Gavin Newsom being the first guy to really take money from
  824. the first guy to really take money from pot do a dashboard of all the money
  825. pot do a dashboard of all the money coming into the state state state yeah
  826. coming into the state state state yeah the governor's race that updates really
  827. the governor's race that updates really frequently then mining that for analysis
  828. frequently then mining that for analysis like being able to point out this that
  829. like being able to point out this that the charter schools really put a lot of
  830. the charter schools really put a lot of money into V Ragosa and the primary
  831. money into V Ragosa and the primary being able to do historical analysis to
  832. being able to do historical analysis to show that that money from those outside
  833. show that that money from those outside groups was the most ever you know and
  834. groups was the most ever you know and giving that kind of context requires
  835. giving that kind of context requires having all that data at hand and then
  836. having all that data at hand and then more recently mining into every campaign
  837. more recently mining into every campaign donation Gavin Newsom are likely next
  838. donation Gavin Newsom are likely next governor
  839. governor has ever received to do kind of a story
  840. has ever received to do kind of a story about his his his him coming in and so
  841. about his his his him coming in and so California or at California civic data
  842. California or at California civic data dot o-r-g is where that's at and this is
  843. dot o-r-g is where that's at and this is kind of our biggest project right now of
  844. kind of our biggest project right now of trying to bring people together to solve
  845. trying to bring people together to solve a common data problem I've got tons of
  846. a common data problem I've got tons of problems and tickets and stuff in there
  847. problems and tickets and stuff in there if anyone's interested in money in
  848. if anyone's interested in money in politics this is a great place to get
  849. politics this is a great place to get involved there's other money in politics
  850. involved there's other money in politics things to get involved with too if you
  851. things to get involved with too if you curious just come talk to me I'm running
  852. curious just come talk to me I'm running out of time our team does other stuff
  853. out of time our team does other stuff like BOTS and tools and toys and other
  854. like BOTS and tools and toys and other but that's the end because I'm
  855. but that's the end because I'm out of time you can follow us at these
  856. out of time you can follow us at these links and again all the slides are at
  857. links and again all the slides are at this URL and that's that's all I got if
  858. this URL and that's that's all I got if you got questions holler
  859. so does anyone have any questions
  860. well that's an interesting question
  861. well that's an interesting question so the question is whether the click
  862. so the question is whether the click rate difference between interactive and
  863. rate difference between interactive and static graphics and this is a hot topic
  864. static graphics and this is a hot topic in the world of data visualization you
  865. in the world of data visualization you guys may remember like in the time of
  866. guys may remember like in the time of Flash graphics you know and when data
  867. Flash graphics you know and when data visualization really took off everything
  868. visualization really took off everything had lots of hovers and boxes and things
  869. had lots of hovers and boxes and things that jumped around and doodads dancing
  870. that jumped around and doodads dancing bears is what Tom to Iraq at the New
  871. bears is what Tom to Iraq at the New York Times used to call them and
  872. York Times used to call them and recently as the Internet's move more
  873. recently as the Internet's move more toward mobile phones you may be noticing
  874. toward mobile phones you may be noticing the graphics from high-end visit people
  875. the graphics from high-end visit people tend to be like just flat images you
  876. tend to be like just flat images you don't I mean because they know you're on
  877. don't I mean because they know you're on your phone and they think it's less
  878. your phone and they think it's less likely you're gonna interact there's a
  879. likely you're gonna interact there's a little bit of a dispute about like the
  880. little bit of a dispute about like the actual numbers behind that cuz I don't
  881. actual numbers behind that cuz I don't think there's been a serious study but
  882. think there's been a serious study but obviously the majority of people
  883. obviously the majority of people visiting LA Times comm and every new
  884. visiting LA Times comm and every new site are now doing it on a phone right
  885. site are now doing it on a phone right and so whatever you make it has to work
  886. and so whatever you make it has to work on a phone and has to be good on a phone
  887. on a phone and has to be good on a phone we can do things that are good on both
  888. we can do things that are good on both that are they're simplified on the phone
  889. that are they're simplified on the phone and are progressively enhanced on
  890. and are progressively enhanced on desktop right which is what we try to do
  891. desktop right which is what we try to do but you can't over invest in the desktop
  892. but you can't over invest in the desktop and in the desktop doodads my feeling is
  893. and in the desktop doodads my feeling is is that the interactivity is great and
  894. is that the interactivity is great and we don't want to lose it but it's got to
  895. we don't want to lose it but it's got to be it's got to be awesome you know what
  896. be it's got to be awesome you know what I mean the thing that it does the thing
  897. I mean the thing that it does the thing that the dancing bear has to be a really
  898. that the dancing bear has to be a really good dancing bear for it to be worth the
  899. good dancing bear for it to be worth the investment
  900. sure
  901. virtual reality we've done a couple
  902. virtual reality we've done a couple virtual reality experiments I don't know
  903. virtual reality experiments I don't know if we've like invested a ton maybe maybe
  904. if we've like invested a ton maybe maybe not you know I think there's some people
  905. not you know I think there's some people who think VR is the future of
  906. who think VR is the future of storytelling and there's some people who
  907. storytelling and there's some people who don't I don't know the answer to that I
  908. don't I don't know the answer to that I think VR is really cool you know I mean
  909. think VR is really cool you know I mean we've done some things that I'm really
  910. we've done some things that I'm really proud of one of my former colleagues
  911. proud of one of my former colleagues armando mangia made did a VR tour of
  912. armando mangia made did a VR tour of Mars of the area where the Mars rover
  913. Mars of the area where the Mars rover had like crawled around and you can like
  914. had like crawled around and you can like fly around and it more than anything
  915. fly around and it more than anything I've ever experienced it really helped
  916. I've ever experienced it really helped me understand the terrain of Mars you
  917. me understand the terrain of Mars you know so I thought it was an awesome like
  918. know so I thought it was an awesome like experience as we say in our business
  919. experience as we say in our business sorry but whether it's gonna be the
  920. sorry but whether it's gonna be the future of news as we also say in our
  921. future of news as we also say in our business I don't know
  922. business I don't know we'll find out
  923. well you know it's the newsroom is just
  924. well you know it's the newsroom is just a collection of people you know they
  925. a collection of people you know they mean and what's that oh it's question
  926. mean and what's that oh it's question was are there some people in the
  927. was are there some people in the newsroom who are less open to working
  928. newsroom who are less open to working with data right is that your question of
  929. with data right is that your question of some sections I don't think that Eddie
  930. some sections I don't think that Eddie you know I hate to generalize you know
  931. you know I hate to generalize you know as often as I do but no I think you know
  932. as often as I do but no I think you know it just kind of comes down to the person
  933. it just kind of comes down to the person you know our kind of general philosophy
  934. you know our kind of general philosophy is we want to work with people who want
  935. is we want to work with people who want to work with us and so we tend to seek
  936. to work with us and so we tend to seek out a partner people who like want to
  937. out a partner people who like want to have that relationship and we're lucky
  938. have that relationship and we're lucky to be in a situation where our job is
  939. to be in a situation where our job is just to be like a catalyst just like
  940. just to be like a catalyst just like make a cool story app and make like a
  941. make a cool story app and make like a thing happen and if we do that we're
  942. thing happen and if we do that we're good so we don't have to like break down
  943. good so we don't have to like break down walls or deal with people who don't want
  944. walls or deal with people who don't want to deal with us which is a great thing
  945. to deal with us which is a great thing about our set up in my opinion I got a
  946. about our set up in my opinion I got a question please so that that story you
  947. question please so that that story you open with it's kind of it's easy to
  948. open with it's kind of it's easy to connect with in common with this
  949. connect with in common with this populist level like the fat cat getting
  950. populist level like the fat cat getting the huge property discount because
  951. the huge property discount because they're weird pets or whatever but I'm
  952. they're weird pets or whatever but I'm wondering how often you guys run across
  953. wondering how often you guys run across stories that I mean the end of the day
  954. stories that I mean the end of the day LA Times is still a business I'm
  955. LA Times is still a business I'm wondering how often you guys run across
  956. wondering how often you guys run across the story that you feel like this is
  957. the story that you feel like this is interesting and it's worthwhile but we
  958. interesting and it's worthwhile but we got to kind of self select out because
  959. got to kind of self select out because it's not gonna generate that clicks that
  960. it's not gonna generate that clicks that we need to drive I'm never consciously
  961. we need to drive I'm never consciously you know what I mean that's never like a
  962. you know what I mean that's never like a thing where we know we can't do this
  963. thing where we know we can't do this story no one will read it that's never
  964. story no one will read it that's never happened you know I'm sure there is in
  965. happened you know I'm sure there is in like the collection of humans and
  966. like the collection of humans and institutions some sort of subconscious
  967. institutions some sort of subconscious or institutional bias around whatever
  968. or institutional bias around whatever like why do we write more about the
  969. like why do we write more about the Lakers than the Clippers you know what I
  970. Lakers than the Clippers you know what I mean more people care about the Lakers
  971. mean more people care about the Lakers you know there are definitely things
  972. you know there are definitely things like that no offense to Ryan who's a
  973. like that no offense to Ryan who's a Clippers fan but but no I've never in my
  974. Clippers fan but but no I've never in my entire journalistic career had anyone
  975. entire journalistic career had anyone say we can't do a good story because
  976. say we can't do a good story because people aren't interested in it one cool
  977. people aren't interested in it one cool thing about newsrooms is it's definitely
  978. thing about newsrooms is it's definitely full of people who are on the watch for
  979. full of people who are on the watch for stuff like that and are paranoid about
  980. stuff like that and are paranoid about it and also no one wants to get in the
  981. it and also no one wants to get in the way of a good story in a newsroom if you
  982. way of a good story in a newsroom if you got a good story everyone will be
  983. got a good story everyone will be supportive which is one of the cool
  984. supportive which is one of the cool things about being in the newsroom hello
  985. things about being in the newsroom hello thanks for talking I want to hear about
  986. thanks for talking I want to hear about a time when you made a decision - I'm
  987. a time when you made a decision - I'm back sensor data or like not share
  988. back sensor data or like not share certain information that right under how
  989. certain information that right under how you think about yeah right so in this
  990. you think about yeah right so in this case
  991. case actually put the whole property tax role
  992. actually put the whole property tax role with the names up there because I felt
  993. with the names up there because I felt like it would it's it's standard
  994. like it would it's it's standard practice in other counties and it's
  995. practice in other counties and it's ridiculous that it's not public I can
  996. ridiculous that it's not public I can look it up online in most places in
  997. look it up online in most places in America and I just kind of wanted to
  998. America and I just kind of wanted to make the point and so I did it even
  999. make the point and so I did it even though no one will notice but there's
  1000. though no one will notice but there's definitely other cases where we don't
  1001. definitely other cases where we don't because there's just no reason right if
  1002. because there's just no reason right if and it could potentially backfire on the
  1003. and it could potentially backfire on the story if P if that becomes a distraction
  1004. story if P if that becomes a distraction from what you're trying to tell so for
  1005. from what you're trying to tell so for instance one piece of public data that
  1006. instance one piece of public data that is public that is rarely published
  1007. is public that is rarely published online is voter registration data so how
  1008. online is voter registration data so how you vote is totally secret nobody knows
  1009. you vote is totally secret nobody knows how you vote but if you're registered to
  1010. how you vote but if you're registered to vote and in what party and whether you
  1011. vote and in what party and whether you vote yes or no is a public record and
  1012. vote yes or no is a public record and the political parties are using this
  1013. the political parties are using this data and getting it from the Secretary
  1014. data and getting it from the Secretary of State all the time right and we did a
  1015. of State all the time right and we did a story about a weird third party called
  1016. story about a weird third party called the American independent party which was
  1017. the American independent party which was formed by George Wallace to get on the
  1018. formed by George Wallace to get on the ticket in the 60s here in California and
  1019. ticket in the 60s here in California and there's thousands and thousands of
  1020. there's thousands and thousands of people including the new owner of the LA
  1021. people including the new owner of the LA Times by the way who accidentally
  1022. Times by the way who accidentally registered in this party thinking it was
  1023. registered in this party thinking it was being independent but you were actually
  1024. being independent but you were actually joining the American independent party
  1025. joining the American independent party oopsey right and so we did a story about
  1026. oopsey right and so we did a story about that whole oopsie and we made a look up
  1027. that whole oopsie and we made a look up so you could check am I in this like
  1028. so you could check am I in this like crazy party that I don't want to be in
  1029. crazy party that I don't want to be in and it would return like yes or no if
  1030. and it would return like yes or no if you put your stuff in but I didn't like
  1031. you put your stuff in but I didn't like surface the whole database so that
  1032. surface the whole database so that everybody could look up everybody else I
  1033. everybody could look up everybody else I put a little birthdate checker on there
  1034. put a little birthdate checker on there even though you wouldn't have to because
  1035. even though you wouldn't have to because I didn't want it to turn into a fishing
  1036. I didn't want it to turn into a fishing expedition I wanted to make it more of a
  1037. expedition I wanted to make it more of a service for readers and I thought that
  1038. service for readers and I thought that could distract from the story so that's
  1039. could distract from the story so that's a case where we it's an editorial
  1040. a case where we it's an editorial judgment you know totally legally we
  1041. judgment you know totally legally we could have published it but it just
  1042. could have published it but it just didn't fit with what our goals was we're
  1043. didn't fit with what our goals was we're in publishing does that make sense okay
  1044. so data science is really hot right now
  1045. so data science is really hot right now yeah how are you can what is your like
  1046. yeah how are you can what is your like value add in the job market are you
  1047. value add in the job market are you paying big salaries or is it just like
  1048. paying big salaries or is it just like all the work is like really rewarding or
  1049. all the work is like really rewarding or do you just say Ryan oh geez raises hell
  1050. do you just say Ryan oh geez raises hell yes we definitely could use a raise you
  1051. yes we definitely could use a raise you know what I mean we're in a business
  1052. know what I mean we're in a business that's collapsing let's be honest right
  1053. that's collapsing let's be honest right and and so there there's there's a
  1054. and and so there there's there's a little bit of a premium for what we do
  1055. little bit of a premium for what we do but not so much
  1056. but not so much that they're out there throwing a lot of
  1057. that they're out there throwing a lot of money around you know the reality is is
  1058. money around you know the reality is is almost everyone I know who does what
  1059. almost everyone I know who does what like we do are people who have like have
  1060. like we do are people who have like have have some of these skills that you guys
  1061. have some of these skills that you guys have or have an interest in them but
  1062. have or have an interest in them but aren't like SuperDuper skilled but we're
  1063. aren't like SuperDuper skilled but we're passionate about journalism and we kind
  1064. passionate about journalism and we kind of like you kind of see like wow if I
  1065. of like you kind of see like wow if I learn these skills that I do this stuff
  1066. learn these skills that I do this stuff we can do these cool stories that we
  1067. we can do these cool stories that we wouldn't otherwise do we can make the
  1068. wouldn't otherwise do we can make the news better we can do better stuff you
  1069. news better we can do better stuff you know and so for us it I think for most
  1070. know and so for us it I think for most people it kind of fits the two things
  1071. people it kind of fits the two things kind of go together your passion for
  1072. kind of go together your passion for news and your interest or maybe passion
  1073. news and your interest or maybe passion for programming and stuff help you like
  1074. for programming and stuff help you like push together and there definitely are
  1075. push together and there definitely are people who do what we do who now work
  1076. people who do what we do who now work I'd one of my guard in 2007 the guy who
  1077. I'd one of my guard in 2007 the guy who got hired the same day he works at a
  1078. got hired the same day he works at a hedge fund now right and I know people
  1079. hedge fund now right and I know people who work in mergers and acquisitions and
  1080. who work in mergers and acquisitions and thought out of that and I but that's not
  1081. thought out of that and I but that's not for me I wish you paid better maybe one
  1082. for me I wish you paid better maybe one day I'm trying to rewrite our job
  1083. day I'm trying to rewrite our job descriptions we'll see how that goes I
  1084. descriptions we'll see how that goes I give you some tips on that by the way if
  1085. give you some tips on that by the way if anyone's in a unionized newsroom or a
  1086. anyone's in a unionized newsroom or a workplace yeah so I have a question
  1087. workplace yeah so I have a question about the alternative facts news or fake
  1088. about the alternative facts news or fake news fake news right well it's sort of a
  1089. news fake news right well it's sort of a two-part question first of all do you
  1090. two-part question first of all do you see the role of the press in trying to
  1091. see the role of the press in trying to limit identify report on or debunk fake
  1092. limit identify report on or debunk fake news that are being spread around there
  1093. news that are being spread around there through most common with social media
  1094. through most common with social media channels and do you guys work did you
  1095. channels and do you guys work did you guys do any work in that in that area
  1096. guys do any work in that in that area specifically our team hasn't
  1097. specifically our team hasn't specifically worked on that but I had my
  1098. specifically worked on that but I had my opinion there's been a lot of great data
  1099. opinion there's been a lot of great data journalism about fake news like I've the
  1100. journalism about fake news like I've the story I feel like really caught on that
  1101. story I feel like really caught on that seemed him as far as I know was was well
  1102. seemed him as far as I know was was well done was the BuzzFeed news story by
  1103. done was the BuzzFeed news story by Craig Silverman about how the most
  1104. Craig Silverman about how the most shared stories before the last election
  1105. shared stories before the last election that you know that we're all fake you
  1106. that you know that we're all fake you know what I mean like the Pope endorsing
  1107. know what I mean like the Pope endorsing Trump or whatever and that was a case
  1108. Trump or whatever and that was a case where he took his data journalism skills
  1109. where he took his data journalism skills and he took it at this sort of inside
  1110. and he took it at this sort of inside the media topic and I thought it had an
  1111. the media topic and I thought it had an impact and in my opinion I think the
  1112. impact and in my opinion I think the media coverage about fake news the real
  1113. media coverage about fake news the real news about the fake news
  1114. news about the fake news has drawn a lot of attention to this
  1115. has drawn a lot of attention to this problem which i think is good you know
  1116. problem which i think is good you know and I'm all for people doing that you
  1117. and I'm all for people doing that you know I I hear about not being an expert
  1118. know I I hear about not being an expert I hear about research that says when you
  1119. I hear about research that says when you debunk false claims you just reinforce
  1120. debunk false claims you just reinforce those claims and a lot of people's minds
  1121. those claims and a lot of people's minds there's like behavioral research on this
  1122. there's like behavioral research on this I'm not an expert and it makes me worry
  1123. I'm not an expert and it makes me worry that maybe we're not helping things you
  1124. that maybe we're not helping things you know but I don't I don't have the
  1125. know but I don't I don't have the solution and as a journalist you know
  1126. solution and as a journalist you know kind of my default assumption is always
  1127. kind of my default assumption is always more sunlight is better more coverage is
  1128. more sunlight is better more coverage is better and so I'm all for it and I think
  1129. better and so I'm all for it and I think there's been a lot of great technology
  1130. there's been a lot of great technology people who've helped news organizations
  1131. people who've helped news organizations cover that story too so ok last question
  1132. cover that story too so ok last question you can bug me afterwards too ok my
  1133. you can bug me afterwards too ok my question is oh sorry yeah my question is
  1134. question is oh sorry yeah my question is can you talk a little bit about the data
  1135. can you talk a little bit about the data desk rolls and how you go from
  1136. desk rolls and how you go from collecting data is that a specific role
  1137. collecting data is that a specific role to somebody who's building the website
  1138. to somebody who's building the website and and producing the visualizations we
  1139. and and producing the visualizations we don't really have like like firmly
  1140. don't really have like like firmly design defined roles like oh you're the
  1141. design defined roles like oh you're the designer and you're the database
  1142. designer and you're the database administrator and you're the writer you
  1143. administrator and you're the writer you know what I mean
  1144. know what I mean maybe we should do that but my feeling
  1145. maybe we should do that but my feeling is on our team we're better off when
  1146. is on our team we're better off when everybody is kind of a generalist when
  1147. everybody is kind of a generalist when everybody is really comfortable I just
  1148. everybody is really comfortable I just say people on our team when we're
  1149. say people on our team when we're looking for like young people to break
  1150. looking for like young people to break in I don't care if you can code I care
  1151. in I don't care if you can code I care if you can like solve problems you know
  1152. if you can like solve problems you know what I mean and if you're creative about
  1153. what I mean and if you're creative about solving problems with computers and to
  1154. solving problems with computers and to me that's like the essential skill of
  1155. me that's like the essential skill of like what we're doing is being a hard
  1156. like what we're doing is being a hard worker who's creative it like doing that
  1157. worker who's creative it like doing that and the result is is we have a team it's
  1158. and the result is is we have a team it's kind of like a pretty odd ball crew of
  1159. kind of like a pretty odd ball crew of people who have like different skills
  1160. people who have like different skills like Ryan's who's here came through a
  1161. like Ryan's who's here came through a statistics program Malloy came through
  1162. statistics program Malloy came through the army right and libraries we have
  1163. the army right and libraries we have other people who came through
  1164. other people who came through traditional journalism schools and other
  1165. traditional journalism schools and other routes and we have some people who are
  1166. routes and we have some people who are better at HTML and CSS and JavaScript
  1167. better at HTML and CSS and JavaScript and other people who are better at
  1168. and other people who are better at Python but they learn from each other
  1169. Python but they learn from each other and just kind of we just kind of make it
  1170. and just kind of we just kind of make it make stuff happen you know so we don't
  1171. make stuff happen you know so we don't define it that much we might need to if
  1172. define it that much we might need to if we want to get paid more though huh and
  1173. we want to get paid more though huh and but no we don't thank you yep oh we
  1174. but no we don't thank you yep oh we could you want to do more questions Oh
  1175. could you want to do more questions Oh more questions I could talk all night
  1176. more questions I could talk all night guys I have a question
  1177. guys I have a question have you noticed yeah can I ask you a
  1178. have you noticed yeah can I ask you a question yeah please
  1179. question yeah please oh thanks I'm right here okay so last
  1180. oh thanks I'm right here okay so last night I saw the soloist oh and I would
  1181. night I saw the soloist oh and I would not have known who's Steve Lopez
  1182. not have known who's Steve Lopez two days ago yeah so that was cool that
  1183. two days ago yeah so that was cool that you mentioned him that is a really
  1184. you mentioned him that is a really wonderful movie and it's all about this
  1185. wonderful movie and it's all about this very kind of human story this
  1186. very kind of human story this idiosyncratic random meeting between two
  1187. idiosyncratic random meeting between two people which leads to this that's right
  1188. people which leads to this that's right big thing I was curious you know you
  1189. big thing I was curious you know you mentioned the future of journalism
  1190. mentioned the future of journalism you're talking about data journalism
  1191. you're talking about data journalism here do you see the the methods you're
  1192. here do you see the the methods you're talking about potentially growing and
  1193. talking about potentially growing and importance as they most certainly will I
  1194. importance as they most certainly will I think but when they do do you think that
  1195. think but when they do do you think that they could potentially wind up
  1196. they could potentially wind up displacing or replacing the kind of
  1197. displacing or replacing the kind of human journalism stories that we well
  1198. human journalism stories that we well said we're movie I don't know you know
  1199. said we're movie I don't know you know what I mean I mean you guys are the you
  1200. what I mean I mean you guys are the you know machine learning people you can
  1201. know machine learning people you can educate me about how far computers are
  1202. educate me about how far computers are going to get at writing stories you know
  1203. going to get at writing stories you know what I mean like the one bit of like
  1204. what I mean like the one bit of like automated news writing that our team has
  1205. automated news writing that our team has done is is related to quakes earthquakes
  1206. done is is related to quakes earthquakes so there's like a automatic there's data
  1207. so there's like a automatic there's data that gets sent out every time the
  1208. that gets sent out every time the government detects an earthquake and one
  1209. government detects an earthquake and one of my colleagues wrote a bot that
  1210. of my colleagues wrote a bot that automatically writes a blog post about
  1211. automatically writes a blog post about every earthquake that just has like the
  1212. every earthquake that just has like the basic data and like a map and then that
  1213. basic data and like a map and then that gets sent to the copy desk every time
  1214. gets sent to the copy desk every time the quake happens we call quake BOTS and
  1215. the quake happens we call quake BOTS and that's the sort of thing a lot of people
  1216. that's the sort of thing a lot of people look at and they say jibon you're you're
  1217. look at and they say jibon you're you're killing you're stealing our jobs you
  1218. killing you're stealing our jobs you know what I mean but the reality is in
  1219. know what I mean but the reality is in my opinion is that the jobs are already
  1220. my opinion is that the jobs are already gone
  1221. gone you know the LA Times is one third of
  1222. you know the LA Times is one third of the size of was if you look at the
  1223. the size of was if you look at the Bureau of Labor Statistics numbers the
  1224. Bureau of Labor Statistics numbers the number of people who work in newspapers
  1225. number of people who work in newspapers is is 50% what it was 10 or 15 years ago
  1226. is is 50% what it was 10 or 15 years ago like literally the industry is
  1227. like literally the industry is disappearing and that's because of the
  1228. disappearing and that's because of the change in the advertising model and the
  1229. change in the advertising model and the shift to the internet and the increased
  1230. shift to the internet and the increased competition and all these other things
  1231. competition and all these other things that have to be honest nothing to do
  1232. that have to be honest nothing to do with the automation of content creation
  1233. with the automation of content creation and everything to do with the automation
  1234. and everything to do with the automation of content delivery and distribution
  1235. of content delivery and distribution right and that's really what's changed
  1236. right and that's really what's changed our industry and I feel like that's kind
  1237. our industry and I feel like that's kind of boring and everybody loves the
  1238. of boring and everybody loves the internet you know
  1239. internet you know because it's awesome and so it's not as
  1240. because it's awesome and so it's not as exciting to talk about as like the
  1241. exciting to talk about as like the computers gonna write the story instead
  1242. computers gonna write the story instead but those those quake posts that the bot
  1243. but those those quake posts that the bot is doing nobody was doing them you know
  1244. is doing nobody was doing them you know what I mean it's not replacing anybody
  1245. what I mean it's not replacing anybody it's doing something we weren't doing
  1246. it's doing something we weren't doing and it's relieving the people who worked
  1247. and it's relieving the people who worked there from having to like scramble and
  1248. there from having to like scramble and write a little post every time their
  1249. write a little post every time their editor feels an earthquake great as
  1250. editor feels an earthquake great as opposed to doing an automated way so I
  1251. opposed to doing an automated way so I think that we are the industries it's
  1252. think that we are the industries it's such a crisis and in such a deep thing a
  1253. such a crisis and in such a deep thing a change I don't see that as having a huge
  1254. change I don't see that as having a huge effect you know maybe you know I think
  1255. effect you know maybe you know I think sure like data cleaning like I spend
  1256. sure like data cleaning like I spend most of my time doing as I tried to show
  1257. most of my time doing as I tried to show on that maybe you guys can automate that
  1258. on that maybe you guys can automate that work away for me please do you know what
  1259. work away for me please do you know what I mean I don't want to do that yeah so
  1260. I mean I don't want to do that yeah so because of the journalists and like
  1261. because of the journalists and like industry is so rough right now I'm
  1262. industry is so rough right now I'm curious if you have any like if you've
  1263. curious if you have any like if you've experienced any pushback on making
  1264. experienced any pushback on making things open-source and just kind of
  1265. things open-source and just kind of sharing this with other journalists
  1266. sharing this with other journalists versus you know keeping some of these
  1267. versus you know keeping some of these things private and in-house so that the
  1268. things private and in-house so that the LA Times has a monopoly on certain types
  1269. LA Times has a monopoly on certain types of stories now nobody's really paying
  1270. of stories now nobody's really paying attention I'll be honest with you and
  1271. attention I'll be honest with you and and I and if they when they do I'm kind
  1272. and I and if they when they do I'm kind of ready to make the case you know you
  1273. of ready to make the case you know you look at something like that election
  1274. look at something like that election stuff I went over really really fast but
  1275. stuff I went over really really fast but that's a case where we would have spent
  1276. that's a case where we would have spent weeks and weeks and weeks writing the
  1277. weeks and weeks and weeks writing the same code that everybody else was
  1278. same code that everybody else was writing you know what I mean and
  1279. writing you know what I mean and everybody's team is two or three or four
  1280. everybody's team is two or three or four or five people and we just don't have
  1281. or five people and we just don't have the resources to like do all this stuff
  1282. the resources to like do all this stuff on our own I don't think there's really
  1283. on our own I don't think there's really a choice other than open source to solve
  1284. a choice other than open source to solve a lot of the problems we have because we
  1285. a lot of the problems we have because we just don't have a hundred engineers to
  1286. just don't have a hundred engineers to throw it anything you know and so
  1287. throw it anything you know and so there's really only a couple places that
  1288. there's really only a couple places that can really have a competitive advantage
  1289. can really have a competitive advantage of this and they're the largest ones and
  1290. of this and they're the largest ones and if you're in a small or midsize place I
  1291. if you're in a small or midsize place I think open source is your only only real
  1292. think open source is your only only real opportunity now that said they're you
  1293. opportunity now that said they're you know what's the old Linux thing Fudd
  1294. know what's the old Linux thing Fudd you know fear uncertainty and doubt
  1295. you know fear uncertainty and doubt people raised about open source that
  1296. people raised about open source that definitely happens there was a long
  1297. definitely happens there was a long period of time where we had a lawyer who
  1298. period of time where we had a lawyer who just like didn't like open source for
  1299. just like didn't like open source for some reason so I just didn't put
  1300. some reason so I just didn't put licenses on our repo because I just
  1301. licenses on our repo because I just didn't want to have a conversation with
  1302. didn't want to have a conversation with a lawyer about a license I just put and
  1303. a lawyer about a license I just put and I get pull requests from the license
  1304. I get pull requests from the license nerd saying you should they had a
  1305. nerd saying you should they had a license and I was like yeah I'm just not
  1306. license and I was like yeah I'm just not gonna do that
  1307. gonna do that you know because because I don't want to
  1308. you know because because I don't want to have to defend that to the lawyer one
  1309. have to defend that to the lawyer one day you know day mean and then she she
  1310. day you know day mean and then she she left and now I put licenses on
  1311. left and now I put licenses on it's MIT guys I'm into MIT I don't know
  1312. it's MIT guys I'm into MIT I don't know about you I thank you so much for coming
  1313. about you I thank you so much for coming and speaking today thank you in the back
  1314. and speaking today thank you in the back my question is what is the current
  1315. my question is what is the current vision for the times and how are you
  1316. vision for the times and how are you gonna make the stories reach us and many
  1317. gonna make the stories reach us and many other folks of our generation who have
  1318. other folks of our generation who have kind of split off I know that I noticed
  1319. kind of split off I know that I noticed that you guys have hired for podcasts
  1320. that you guys have hired for podcasts and that's one way but what's the vision
  1321. and that's one way but what's the vision and how are you reaching us well that's
  1322. and how are you reaching us well that's above my pay grade and still being
  1323. above my pay grade and still being developed the LA Times has a new owner
  1324. developed the LA Times has a new owner his name is dr. Patrick soon Xian he is
  1325. his name is dr. Patrick soon Xian he is one of the richest people in Los Angeles
  1326. one of the richest people in Los Angeles and the richest doctor in the world he
  1327. and the richest doctor in the world he made his money on pharmaceuticals a and
  1328. made his money on pharmaceuticals a and other things and now he's in his 60s and
  1329. other things and now he's in his 60s and he's bought the LA Times as kind of a
  1330. he's bought the LA Times as kind of a legacy project for himself though he
  1331. legacy project for himself though he says he intends to run it fully as the
  1332. says he intends to run it fully as the business and not as a philanthropy and
  1333. business and not as a philanthropy and so as of just a few months ago we're an
  1334. so as of just a few months ago we're an entirely different ownership structure
  1335. entirely different ownership structure and fiscal situation than we were for
  1336. and fiscal situation than we were for the the 20 years that preceded him right
  1337. the the 20 years that preceded him right and what that's gonna mean for us is an
  1338. and what that's gonna mean for us is an independent like thing owned by a single
  1339. independent like thing owned by a single billionaire is still unclear you know
  1340. billionaire is still unclear you know and that strategy we have a new editor
  1341. and that strategy we have a new editor we're hiring for the first time in a
  1342. we're hiring for the first time in a long time which is great I'm more
  1343. long time which is great I'm more optimistic about some things than I have
  1344. optimistic about some things than I have been in a while which is good but what
  1345. been in a while which is good but what it's ultimately gonna mean I don't know
  1346. it's ultimately gonna mean I don't know you know I think if we follow the trends
  1347. you know I think if we follow the trends of the rest of the industry it probably
  1348. of the rest of the industry it probably means moving toward more towards a
  1349. means moving toward more towards a digital subscription model because
  1350. digital subscription model because digital advertising doesn't pay and it's
  1351. digital advertising doesn't pay and it's failing for pretty much everyone and so
  1352. failing for pretty much everyone and so in order to finance the news
  1353. in order to finance the news organization the readers have to pay
  1354. organization the readers have to pay more right and so that's why everyone's
  1355. more right and so that's why everyone's going to pay walls and that kind of
  1356. going to pay walls and that kind of thing and that's been very successful
  1357. thing and that's been very successful for a few places so I suspect well we're
  1358. for a few places so I suspect well we're gonna invest more than that but even
  1359. gonna invest more than that but even that really hasn't been announced I
  1360. that really hasn't been announced I think it's still being determined in El
  1361. think it's still being determined in El Segundo as we speak I'll be watching as
  1362. Segundo as we speak I'll be watching as closely as you trust me all right I
  1363. closely as you trust me all right I think I have the last one okay I get the
  1364. think I have the last one okay I get the last question twice in a row okay can I
  1365. last question twice in a row okay can I so one of my biggest pet peeves is poor
  1366. so one of my biggest pet peeves is poor scientific journalism I'm wondering if
  1367. scientific journalism I'm wondering if the data desk provide
  1368. the data desk provide it's for the LA Times some statistical
  1369. it's for the LA Times some statistical expertise for pieces and and kind of
  1370. expertise for pieces and and kind of what you guys what is your guys's role
  1371. what you guys what is your guys's role in that we do when called upon and we
  1372. in that we do when called upon and we try to on the stories we work on
  1373. try to on the stories we work on directly but it's not like there's a
  1374. directly but it's not like there's a desk that every story comes by and we
  1375. desk that every story comes by and we stamp it yes yes the math is right and I
  1376. stamp it yes yes the math is right and I like you have picked up the paper some
  1377. like you have picked up the paper some day and been like I don't know about
  1378. day and been like I don't know about using percentage change right there you
  1379. using percentage change right there you know what I mean or whatever right and
  1380. know what I mean or whatever right and or is it billion or million are we sure
  1381. or is it billion or million are we sure you know like that kind of thing and
  1382. you know like that kind of thing and there isn't like a centralized process
  1383. there isn't like a centralized process for that we have editors and copy
  1384. for that we have editors and copy editors and people who go over
  1385. editors and people who go over everything really carefully but there's
  1386. everything really carefully but there's not like a numbers editor who looks at
  1387. not like a numbers editor who looks at every number in the paper that's managed
  1388. every number in the paper that's managed kind of on a story by story basis and so
  1389. kind of on a story by story basis and so that we don't could be exhausting if I
  1390. that we don't could be exhausting if I did be honest
  1391. did be honest thank you okay I have a few lotto
  1392. thank you okay I have a few lotto tickets left
  1393. yeah I'm trying to think of a fair way
  1394. yeah I'm trying to think of a fair way to give these out what's that okay yeah
  1395. to give these out what's that okay yeah we could use them right definitely true
  1396. how about if you come up to me
  1397. how about if you come up to me afterwards and tell you tell me about an
  1398. afterwards and tell you tell me about an LA Times story you read recently proving
  1399. LA Times story you read recently proving you've read it I'll give you a lotto
  1400. you've read it I'll give you a lotto ticket okay thank you
  1401. ticket okay thank you [Applause]

Downloads

Slides PDF · Recording video · Extracted slide text · Timestamped transcript