Data Journalism Workshop

By • 14East Magazine in Zoom

Recording

Show the timestamped transcript
  1. awesome so we are now recording um so hello everyone thank you so much
  2. um so hello everyone thank you so much for coming out tonight um
  3. for coming out tonight um this is a introductory journalism
  4. this is a introductory journalism workshop my name is claire mallon and
  5. workshop my name is claire mallon and i'm the president of de paul's chapter
  6. i'm the president of de paul's chapter of the society of professional
  7. of the society of professional journalists and the engagement editor
  8. journalists and the engagement editor for 14 east magazine helping facilitate
  9. for 14 east magazine helping facilitate this workshop tonight is grace del
  10. this workshop tonight is grace del vecchio fort's niece magazine's
  11. vecchio fort's niece magazine's editor-in-chief and cam rodriguez sports
  12. editor-in-chief and cam rodriguez sports managing editor
  13. managing editor and with us today and leading our
  14. and with us today and leading our introductory data journalism workshop is
  15. introductory data journalism workshop is ben welsh ben is a los angeles times
  16. ben welsh ben is a los angeles times data journalist who currently serves as
  17. data journalist who currently serves as the visiting la times journalist at
  18. the visiting la times journalist at stanford university where he is
  19. stanford university where he is expanding where he's leading an
  20. expanding where he's leading an expansion of the big local news
  21. expansion of the big local news initiative
  22. initiative um ben stick with us while i just run
  23. um ben stick with us while i just run through all your amazing credits he
  24. through all your amazing credits he co-founded the times first digitally
  25. co-founded the times first digitally focused projects team and went on to
  26. focused projects team and went on to lead the modernization of the
  27. lead the modernization of the newspaper's graphics department his work
  28. newspaper's graphics department his work includes coverage of live election
  29. includes coverage of live election results across four presidential cycles
  30. results across four presidential cycles a mapping platform that's a new standard
  31. a mapping platform that's a new standard for defining la neighborhoods custom
  32. for defining la neighborhoods custom designs for dozens of flagship projects
  33. designs for dozens of flagship projects and the most complete resource on the
  34. and the most complete resource on the spread of cobit 19 in california
  35. spread of cobit 19 in california his data-driven reporting has led to
  36. his data-driven reporting has led to reforms in the la
  37. reforms in the la fire department's 9-1-1 system a revamp
  38. fire department's 9-1-1 system a revamp of the broken building inspection
  39. of the broken building inspection program the replacement of the los
  40. program the replacement of the los angeles police department's public crime
  41. angeles police department's public crime map as well as an increased fines
  42. map as well as an increased fines against exploitive landlords he has won
  43. against exploitive landlords he has won numerous journalism awards including a
  44. numerous journalism awards including a pulitzer prize for breaking news
  45. pulitzer prize for breaking news reporting as part of the la times team
  46. reporting as part of the la times team that covered the mass shooting in san
  47. that covered the mass shooting in san bernardino he's a proud depaul alum and
  48. bernardino he's a proud depaul alum and serves as a fellow for depaul's very own
  49. serves as a fellow for depaul's very own center for journalism integrity and
  50. center for journalism integrity and excellence he's a big believer in making
  51. excellence he's a big believer in making data openly accessible and frequently
  52. data openly accessible and frequently teaches data journalism and computer
  53. teaches data journalism and computer programming skills to students and
  54. programming skills to students and professionals like us tonight
  55. professionals like us tonight ben we are so lucky to have you here
  56. ben we are so lucky to have you here today
  57. today well thank you so much for having me
  58. well thank you so much for having me yeah of course we are very excited um
  59. yeah of course we are very excited um i'm going to go ahead and spotlight you
  60. i'm going to go ahead and spotlight you for everyone
  61. for everyone you're now in the spotlight um so ben is
  62. you're now in the spotlight um so ben is going to lead a introductory data
  63. going to lead a introductory data journalism web workshop specifically on
  64. journalism web workshop specifically on spreadsheets um
  65. spreadsheets um ben i'll pass it off to you you can go
  66. ben i'll pass it off to you you can go ahead and get us started okay great
  67. ahead and get us started okay great thank you for the introduction claire
  68. thank you for the introduction claire that was really generous um you know as
  69. that was really generous um you know as she said i'm a depaul graduate i'm from
  70. she said i'm a depaul graduate i'm from the class of 2004
  71. the class of 2004 which is a frightening amount of time
  72. which is a frightening amount of time has passed since then now that i'm
  73. has passed since then now that i'm forced to reflect on it but it's really
  74. forced to reflect on it but it's really great to be back at depaul and teaching
  75. great to be back at depaul and teaching and helping again if even remotely
  76. and helping again if even remotely because um
  77. because um you know without being too corny about
  78. you know without being too corny about it you know depaul really was a major
  79. it you know depaul really was a major factor in my career and my life i grew
  80. factor in my career and my life i grew up in eastern iowa
  81. up in eastern iowa near cedar rapids and iowa city and
  82. near cedar rapids and iowa city and moving to chicago at age 17 was it might
  83. moving to chicago at age 17 was it might as it was like paris france to me and it
  84. as it was like paris france to me and it was the biggest deal in my life and uh
  85. was the biggest deal in my life and uh you know other than meeting my wife and
  86. you know other than meeting my wife and getting married probably the best
  87. getting married probably the best decision i ever made so being involved
  88. decision i ever made so being involved with paul continuing involved with it is
  89. with paul continuing involved with it is really important to me and i'd be happy
  90. really important to me and i'd be happy to talk about that more but
  91. to talk about that more but i won't get too sappy on you um it was
  92. i won't get too sappy on you um it was after paul that i began my journalism
  93. after paul that i began my journalism experience um working with carol moraine
  94. experience um working with carol moraine and don mosley when they first took up
  95. and don mosley when they first took up their residence and it was with them
  96. their residence and it was with them that
  97. that i built my first spreadsheet so there
  98. i built my first spreadsheet so there was a story about a a suburb of chicago
  99. was a story about a a suburb of chicago that people were telling us maybe
  100. that people were telling us maybe had some crooked stuff going on with how
  101. had some crooked stuff going on with how legal bills were being charged against
  102. legal bills were being charged against the city and uh and don help me file my
  103. the city and uh and don help me file my first public records request to the city
  104. first public records request to the city just asking for all the legal fees
  105. just asking for all the legal fees printed out
  106. printed out it came back as one of those old school
  107. it came back as one of those old school printouts which maybe you guys have
  108. printouts which maybe you guys have never seen where the printer would have
  109. never seen where the printer would have the little tear strips along the side
  110. the little tear strips along the side and all the pages would be folded
  111. and all the pages would be folded together like an accordion
  112. together like an accordion and it was probably 20 or 30 feet long
  113. and it was probably 20 or 30 feet long and from this like 1980s printer and it
  114. and from this like 1980s printer and it had all the legal fees in there but they
  115. had all the legal fees in there but they were you know hard to read and spread
  116. were you know hard to read and spread out across this long scroll we kind of
  117. out across this long scroll we kind of thought well how are we going to get
  118. thought well how are we going to get from this to our to our story
  119. from this to our to our story and the answer was my first spreadsheet
  120. and the answer was my first spreadsheet which was to really just sit there and
  121. which was to really just sit there and go through it page by page and type the
  122. go through it page by page and type the numbers in
  123. numbers in and then add it up so that we could say
  124. and then add it up so that we could say this is how much money got spent over
  125. this is how much money got spent over what time and and that became the basis
  126. what time and and that became the basis for an investigative story that ran on
  127. for an investigative story that ran on wmaq
  128. wmaq and uh carol even climbed the ladder and
  129. and uh carol even climbed the ladder and then let the paper drop like live on
  130. then let the paper drop like live on camera to show how silly it was and and
  131. camera to show how silly it was and and that for me was kind of my first hit of
  132. that for me was kind of my first hit of it and what got me into data journalism
  133. it and what got me into data journalism was really that by learning some nerdy
  134. was really that by learning some nerdy stuff one i could kind of find my niche
  135. stuff one i could kind of find my niche like how i could contribute as someone
  136. like how i could contribute as someone who wasn't as experienced in other ways
  137. who wasn't as experienced in other ways and two it was just kind of like a
  138. and two it was just kind of like a shortcut to doing like really cool
  139. shortcut to doing like really cool investigative work you know because if
  140. investigative work you know because if you can learn these data skills
  141. you can learn these data skills even if it's just like an extra kind of
  142. even if it's just like an extra kind of power you have and that your main focus
  143. power you have and that your main focus as a specialist
  144. as a specialist it's something you can use to to just
  145. it's something you can use to to just get stories nobody else is getting and
  146. get stories nobody else is getting and to do stuff that really makes an impact
  147. to do stuff that really makes an impact that people notice and so um you know i
  148. that people notice and so um you know i ended up going down this whole road
  149. ended up going down this whole road where i kind of became a specialist and
  150. where i kind of became a specialist and so you know gradually becoming a
  151. so you know gradually becoming a computer programmer and really just
  152. computer programmer and really just focusing on data data data all day and
  153. focusing on data data data all day and you could go down that road if you're
  154. you could go down that road if you're interested and i'd be happy to talk to
  155. interested and i'd be happy to talk to you about it but really a lot of the
  156. you about it but really a lot of the most effective data journalists really
  157. most effective data journalists really just have it as like an extra tool in
  158. just have it as like an extra tool in their toolkit and they you know see
  159. their toolkit and they you know see themselves more as traditional
  160. themselves more as traditional journalists but they know enough about
  161. journalists but they know enough about data to pull off a story now and then
  162. data to pull off a story now and then and those are people often have some of
  163. and those are people often have some of the biggest impact and so i think
  164. the biggest impact and so i think you don't have to look at these skills
  165. you don't have to look at these skills as something that have to consume your
  166. as something that have to consume your whole career or life unless you really
  167. whole career or life unless you really want them to
  168. want them to so
  169. so um so the training i'm going to give
  170. um so the training i'm going to give tonight is is really just like
  171. tonight is is really just like spreadsheets 101 it's a really basic
  172. spreadsheets 101 it's a really basic introduction to like what is a
  173. introduction to like what is a spreadsheet and how do you use it and
  174. spreadsheet and how do you use it and then how to use it to kind of analyze
  175. then how to use it to kind of analyze data to ask and answer questions to
  176. data to ask and answer questions to interview it and so we're going to start
  177. interview it and so we're going to start with sort of some phony data that we
  178. with sort of some phony data that we make up just to like keep things simple
  179. make up just to like keep things simple and we're going to cover some of the
  180. and we're going to cover some of the fundamentals of working with a
  181. fundamentals of working with a spreadsheet and then after we get some
  182. spreadsheet and then after we get some practice we're going to introduce a real
  183. practice we're going to introduce a real data set from the actual news in chicago
  184. data set from the actual news in chicago and and try to ask it some questions try
  185. and and try to ask it some questions try to interview it like we would another
  186. to interview it like we would another source and that data set is the
  187. source and that data set is the complaints against chicago police which
  188. complaints against chicago police which is a database that was sort of
  189. is a database that was sort of crowbarred out of the government a few
  190. crowbarred out of the government a few years ago by the invisible institute i
  191. years ago by the invisible institute i bet you guys have seen it in the news
  192. bet you guys have seen it in the news from time to time
  193. from time to time okay so that's kind of the plan
  194. okay so that's kind of the plan all right that's that's the pre-spiel
  195. all right that's that's the pre-spiel for the actual spiel
  196. for the actual spiel um before we begin does anybody have any
  197. um before we begin does anybody have any questions
  198. questions or things they
  199. or things they want to ask i'm happy to answer anything
  200. and if no one has any questions now if you have questions throughout the
  201. you have questions throughout the workshop uh cam grace and i will be
  202. workshop uh cam grace and i will be answering questions in the chat so you
  203. answering questions in the chat so you can just shoot them there and we'll try
  204. can just shoot them there and we'll try and help you troubleshoot and if not
  205. and help you troubleshoot and if not we'll bring them to ben's attention
  206. we'll bring them to ben's attention looks like ava has a question oh what's
  207. looks like ava has a question oh what's your question
  208. your question yeah um i was just wondering i know
  209. yeah um i was just wondering i know excel is like the most common um
  210. excel is like the most common um platform for working with spreadsheets
  211. platform for working with spreadsheets but are there any other any other
  212. but are there any other any other types of programs or software that data
  213. types of programs or software that data journalists typically use
  214. journalists typically use yeah sure so when a you know spreadsheet
  215. yeah sure so when a you know spreadsheet is really kind of a generic term you
  216. is really kind of a generic term you know it's like soda
  217. know it's like soda and excel is like coca-cola right you
  218. and excel is like coca-cola right you know what i mean it's kind of the brand
  219. know what i mean it's kind of the brand name of the biggest product but there's
  220. name of the biggest product but there's other sodas on the market so to speak
  221. other sodas on the market so to speak when it comes to spreadsheets in fact
  222. when it comes to spreadsheets in fact the original spreadsheet was called
  223. the original spreadsheet was called fizzy calc if you've heard of that from
  224. fizzy calc if you've heard of that from the 1980s
  225. the 1980s um
  226. um and you know there's one that comes on
  227. and you know there's one that comes on apple computers called numbers
  228. apple computers called numbers um you know i'm a super nerd so i use
  229. um you know i'm a super nerd so i use linux and there's like an open source
  230. linux and there's like an open source one that's free
  231. one that's free and then we're actually in this class
  232. and then we're actually in this class going to use google sheets which is sort
  233. going to use google sheets which is sort of like google's competitor to excel
  234. of like google's competitor to excel it's available in a web browser and it's
  235. it's available in a web browser and it's free which is why we're teaching it in
  236. free which is why we're teaching it in this class
  237. this class i think probably
  238. i think probably if you were to compare them all excel
  239. if you were to compare them all excel probably is the best because
  240. probably is the best because it just can handle data sets that are a
  241. it just can handle data sets that are a little larger
  242. little larger and it has a lot of features and it's
  243. and it has a lot of features and it's really well tested but when it comes to
  244. really well tested but when it comes to like the basic features that we're going
  245. like the basic features that we're going to use for this uh class and which most
  246. to use for this uh class and which most journalists use they all have the basic
  247. journalists use they all have the basic features and you can really use all of
  248. features and you can really use all of them to do it and the stuff we cover in
  249. them to do it and the stuff we cover in google sheets everything we cover is in
  250. google sheets everything we cover is in excel it's just like the button is going
  251. excel it's just like the button is going to be in a slightly different place or
  252. to be in a slightly different place or they might have like a different nerdy
  253. they might have like a different nerdy name for some feature or something right
  254. name for some feature or something right so spreadsheets it's like you know we're
  255. so spreadsheets it's like you know we're going to do use basketball as our
  256. going to do use basketball as our example and so to me spreadsheets and
  257. example and so to me spreadsheets and data journalism is like tripling you
  258. data journalism is like tripling you know it's just like the fundamental
  259. know it's just like the fundamental skill of what is data how do you
  260. skill of what is data how do you understand it how do you how do you
  261. understand it how do you how do you structure it manipulate it and analyze
  262. structure it manipulate it and analyze it so that's where it all begins and so
  263. it so that's where it all begins and so i really think that it's the fundamental
  264. i really think that it's the fundamental tool however as you start working with
  265. tool however as you start working with larger data sets that have more and more
  266. larger data sets that have more and more records or you have a database that has
  267. records or you have a database that has many tables that need to be joined
  268. many tables that need to be joined together kind of in a complex system
  269. together kind of in a complex system or if you're going to begin to like um
  270. or if you're going to begin to like um automate things to routinely gather data
  271. automate things to routinely gather data from the internet or other things like
  272. from the internet or other things like that that's where you begin to introduce
  273. that that's where you begin to introduce tools that go beyond the spreadsheet you
  274. tools that go beyond the spreadsheet you know and so those include like big
  275. know and so those include like big relational database software they
  276. relational database software they include computer programming languages
  277. include computer programming languages that are built for statistical analysis
  278. that are built for statistical analysis you know maybe you've heard of r or
  279. you know maybe you've heard of r or python right and you know a lot of
  280. python right and you know a lot of people sometimes start learning those
  281. people sometimes start learning those skills i think that that's fine it's
  282. skills i think that that's fine it's fine to start with those but i really do
  283. fine to start with those but i really do think that the spreadsheet is the is the
  284. think that the spreadsheet is the is the basic building block and i think quite a
  285. basic building block and i think quite a lot of data journalism can and should be
  286. lot of data journalism can and should be done in the spreadsheet and doesn't
  287. done in the spreadsheet and doesn't require the bigger tools
  288. require the bigger tools did that answer your question
  289. did that answer your question yes it did thank you
  290. yes it did thank you [Music]
  291. excuse me i have a little bit of a cough i have
  292. i have flu like two weeks ago not covered but i
  293. flu like two weeks ago not covered but i am still coughing and so that's going to
  294. am still coughing and so that's going to happen tonight and i just apologize in
  295. happen tonight and i just apologize in advance it's just a sign of what a
  296. advance it's just a sign of what a weekly i am okay that's still carrying
  297. weekly i am okay that's still carrying this around
  298. this around all right so if there's no other
  299. all right so if there's no other questions i'm just going to dive in
  300. questions i'm just going to dive in which is going to start with sharing my
  301. which is going to start with sharing my screen so bear with me here
  302. all right can you guys see my screen yes
  303. yes okay so this is a blank google sheet
  304. okay so this is a blank google sheet this is like the very beginning of a
  305. this is like the very beginning of a spreadsheet within google sheets but let
  306. spreadsheet within google sheets but let me show you how to get there if you
  307. me show you how to get there if you haven't been there before
  308. haven't been there before the starting place i'm going to go back
  309. the starting place i'm going to go back to a tab and it's just in google drive
  310. to a tab and it's just in google drive so that's
  311. so that's drive.google.com
  312. drive.google.com right which is
  313. right which is this is kind of like the microsoft
  314. this is kind of like the microsoft office of google i mean you guys are
  315. office of google i mean you guys are college students i bet you've all seen
  316. college students i bet you've all seen this before right so this will require
  317. this before right so this will require that you have a google account i think
  318. that you have a google account i think and that you log into google so i would
  319. and that you log into google so i would just ask everybody to start off
  320. just ask everybody to start off by going to drive.google.com
  321. by going to drive.google.com and kind of getting to this landing page
  322. and kind of getting to this landing page which is all your like folders right
  323. which is all your like folders right and you can see i've got some stuff in
  324. and you can see i've got some stuff in my life and then some things i've been
  325. my life and then some things i've been monkeying today
  326. monkeying today okay
  327. okay i'm going to give everybody you know a
  328. i'm going to give everybody you know a couple seconds just to get there because
  329. couple seconds just to get there because you got to log into your google account
  330. you got to log into your google account etc
  331. etc excuse me
  332. all right if you're having trouble again just feel free to raise the hand button
  333. just feel free to raise the hand button and one of the other hosts can help you
  334. and one of the other hosts can help you along and if i'm going too fast or too
  335. along and if i'm going too fast or too slow please speak up but i'm going to
  336. slow please speak up but i'm going to begin
  337. begin the first step is just to create a new
  338. the first step is just to create a new blank spreadsheet you know a canvas for
  339. blank spreadsheet you know a canvas for you to begin to create and you do that
  340. you to begin to create and you do that by clicking the new button in the upper
  341. by clicking the new button in the upper left hand corner
  342. left hand corner and then there's this pull down which
  343. and then there's this pull down which has the different types of things that
  344. has the different types of things that google drive can do google docs that
  345. google drive can do google docs that you've all seen they have a powerpoint
  346. you've all seen they have a powerpoint thing
  347. thing a little form doodly deal right but
  348. a little form doodly deal right but right there in the middle is google
  349. right there in the middle is google sheets sheets being short for
  350. sheets sheets being short for spreadsheets so i click that
  351. spreadsheets so i click that and boom
  352. and boom i've got my new blank spreadsheet which
  353. i've got my new blank spreadsheet which is oftentimes called a workbook in the
  354. is oftentimes called a workbook in the language of spreadsheet nerds right
  355. language of spreadsheet nerds right and it starts off blank we're just going
  356. and it starts off blank we're just going to start from scratch so we can cover
  357. to start from scratch so we can cover some of the fundamentals with some phony
  358. some of the fundamentals with some phony data so the first thing i'm going to do
  359. data so the first thing i'm going to do is i'm going to go and i'm going to name
  360. is i'm going to go and i'm going to name my spreadsheet so i'm going to click in
  361. my spreadsheet so i'm going to click in the upper left hand corner on that
  362. the upper left hand corner on that untitled spreadsheet i'm going to give
  363. untitled spreadsheet i'm going to give it a name
  364. it a name and uh you know forgive me for the
  365. and uh you know forgive me for the sports stuff but we're going to call
  366. sports stuff but we're going to call this uh you know our dream team because
  367. this uh you know our dream team because our what we're going to do in our first
  368. our what we're going to do in our first spreadsheet is we're going to create our
  369. spreadsheet is we're going to create our dream team of basketball players our
  370. dream team of basketball players our favorite basketball players that we all
  371. favorite basketball players that we all love
  372. love and then you know start a starting
  373. and then you know start a starting basketball team has five people in it
  374. basketball team has five people in it right
  375. right and so in a spreadsheet you know each
  376. and so in a spreadsheet you know each row tends to be a record so one two
  377. row tends to be a record so one two three four five so we're gonna fill in
  378. three four five so we're gonna fill in five rows one for each player
  379. five rows one for each player and each column or field
  380. and each column or field tends to be
  381. tends to be uh an attribute or a piece of metadata
  382. uh an attribute or a piece of metadata about the person it's their name it's
  383. about the person it's their name it's their birthday it's their salary these
  384. their birthday it's their salary these are the things we'll fill in and that
  385. are the things we'll fill in and that sort of grid or structure is what
  386. sort of grid or structure is what makes it a spreadsheet
  387. makes it a spreadsheet and so
  388. and so the first thing that really comes with
  389. the first thing that really comes with any spreadsheet is defining what your
  390. any spreadsheet is defining what your columns are going to be what's the data
  391. columns are going to be what's the data that you're going to track within each
  392. that you're going to track within each row
  393. row and so my goal is to make my basketball
  394. and so my goal is to make my basketball dream team and so i'm going to the first
  395. dream team and so i'm going to the first column is going to be the name of the
  396. column is going to be the name of the player right so i'm just going to click
  397. player right so i'm just going to click in right there
  398. in right there a1 see it's a is the column one is the
  399. a1 see it's a is the column one is the row it's just like battleship
  400. row it's just like battleship right
  401. right a
  402. a one that's my first column in my first
  403. one that's my first column in my first row and i'm gonna say name this is these
  404. row and i'm gonna say name this is these are work this is what's called the
  405. are work this is what's called the header
  406. header right it's the kind of first row the
  407. right it's the kind of first row the front of the sheet
  408. front of the sheet and then each player has a position that
  409. and then each player has a position that they play right like in basketball where
  410. they play right like in basketball where do they play on the court so my next
  411. do they play on the court so my next column on the type position you guys do
  412. column on the type position you guys do this too play along
  413. this too play along okay
  414. okay name position
  415. name position and then since we want to do a little
  416. and then since we want to do a little math as part of this example we want to
  417. math as part of this example we want to get into
  418. get into how we can calculate stuff once we have
  419. how we can calculate stuff once we have data put together we're going to have a
  420. data put together we're going to have a column that is
  421. column that is current salary is what i'm going to call
  422. current salary is what i'm going to call so this is what they get paid right now
  423. so this is what they get paid right now and then we're going to make a kind of
  424. and then we're going to make a kind of fun fantasy column which is what we're
  425. fun fantasy column which is what we're going to pay them from like our big
  426. going to pay them from like our big budget right and so we're going to say
  427. budget right and so we're going to say like new salary
  428. like new salary right so just by clicking in
  429. right so just by clicking in typing hitting enter i've been able to
  430. typing hitting enter i've been able to put in these header fields
  431. put in these header fields into the first row and then each each
  432. into the first row and then each each row beneath it will have the values for
  433. row beneath it will have the values for that record
  434. i jumped in a little bit ahead of here there's some other stuff around that
  435. there's some other stuff around that might be worth going over you see that
  436. might be worth going over you see that just like in microsoft word or any uh
  437. just like in microsoft word or any uh document editing thing you've used
  438. document editing thing you've used there's kind of formatting ribbon of
  439. there's kind of formatting ribbon of options up here which allow you to sort
  440. options up here which allow you to sort of like style and muck around with the
  441. of like style and muck around with the stuff that you put in so i would just
  442. stuff that you put in so i would just say let's everybody just click on a1
  443. say let's everybody just click on a1 hold down and drag to the right you see
  444. hold down and drag to the right you see how i did that and i've now selected
  445. how i did that and i've now selected four cells
  446. four cells i'm gonna do that then i'm just gonna
  447. i'm gonna do that then i'm just gonna hit that b
  448. hit that b right there on the ribbon and just like
  449. right there on the ribbon and just like in microsoft word or google docs it's
  450. in microsoft word or google docs it's gonna bold it right and so now we've
  451. gonna bold it right and so now we've bolded the first row of our data to make
  452. bolded the first row of our data to make it clear these are the headers
  453. it clear these are the headers right and there's lots of other options
  454. right and there's lots of other options here which allow you to do background
  455. here which allow you to do background colors italics and change the font
  456. colors italics and change the font when it comes to numbers there's some
  457. when it comes to numbers there's some important ways to format that we'll get
  458. important ways to format that we'll get into and just a lot of stuff so if you
  459. into and just a lot of stuff so if you look around up here there's a lot of
  460. look around up here there's a lot of different options ultimately you
  461. different options ultimately you probably only need four or five of them
  462. probably only need four or five of them to kind of do what we need to do but
  463. to kind of do what we need to do but there's a bunch of stuff
  464. some of those things will of course be different in excel versus in google
  465. different in excel versus in google sheets um on mac versus pc and so sort
  466. sheets um on mac versus pc and so sort of if you switch from one spreadsheet
  467. of if you switch from one spreadsheet program to another all the same buttons
  468. program to another all the same buttons are kind of up there they're just going
  469. are kind of up there they're just going to look a little different and be in
  470. to look a little different and be in different places
  471. different places and kind of just being patient about it
  472. and kind of just being patient about it is and finding what you're after is kind
  473. is and finding what you're after is kind of what's necessary
  474. of what's necessary there's some so that's stuff that you
  475. there's some so that's stuff that you see in documents but there's other
  476. see in documents but there's other formatting things that are specific to
  477. formatting things that are specific to spreadsheets so for instance our columns
  478. spreadsheets so for instance our columns our columns and rows can be resized
  479. our columns and rows can be resized right so you see here they each have the
  480. right so you see here they each have the standard size but if i hover
  481. standard size but if i hover over column d i can make it wider i can
  482. over column d i can make it wider i can make it narrower
  483. make it narrower you can do that for the rows make them
  484. you can do that for the rows make them taller or shorter
  485. taller or shorter right there's even a trick where if you
  486. right there's even a trick where if you click in the upper left and that blank
  487. click in the upper left and that blank box it selects the whole spreadsheet
  488. box it selects the whole spreadsheet we'll use that later like that's a
  489. we'll use that later like that's a little trick
  490. little trick if you do that and then
  491. if you do that and then double click on the headers it auto
  492. double click on the headers it auto sizes them to fit the data did you see
  493. sizes them to fit the data did you see how that happened
  494. how that happened there's a million little tricks like
  495. there's a million little tricks like that you don't need to know them all but
  496. that you don't need to know them all but there's a lot underneath the hood here
  497. there's a lot underneath the hood here that's really powerful that you'll start
  498. that's really powerful that you'll start to learn more and more of as you get
  499. to learn more and more of as you get more practice
  500. more practice all right
  501. um one little trick is that you can make the header row kind of special by
  502. the header row kind of special by grabbing the bottom of that upper
  503. grabbing the bottom of that upper upper left corner box which doesn't have
  504. upper left corner box which doesn't have a name as far as i know the magic box
  505. a name as far as i know the magic box i'll call it
  506. i'll call it i'm going to grab that top one and pull
  507. i'm going to grab that top one and pull it down and you can see that now it's
  508. it down and you can see that now it's kind of put my header
  509. kind of put my header right with a little line below it to
  510. right with a little line below it to break it out and after i scroll now you
  511. break it out and after i scroll now you can see how there's row two it sort of
  512. can see how there's row two it sort of disappears and then comes back that's
  513. disappears and then comes back that's gonna freeze that first row so if you
  514. gonna freeze that first row so if you add more data and scroll it'll stay at
  515. add more data and scroll it'll stay at the top that's just one nice little
  516. the top that's just one nice little trick not essential but nice to know
  517. trick not essential but nice to know okay everybody get that far
  518. okay great so now we're going to fill this in we need five basketball players
  519. this in we need five basketball players i need people to shout it out from the
  520. i need people to shout it out from the audience who do you like who do you want
  521. audience who do you like who do you want on your dream team
  522. steph curry steph curry
  523. steph curry he's healthy again not a bad pick
  524. he's healthy again not a bad pick okay he plays
  525. what he's a guard it's okay it's okay
  526. it's okay it's okay he's a little fast guy
  527. he's a little fast guy all right somebody else who's on your
  528. all right somebody else who's on your dream team
  529. dream team we can throw in lebron to make it
  530. we can throw in lebron to make it interesting lebron james this is how
  531. interesting lebron james this is how they do the nba all-star game you know
  532. i'd call him a guard but that's debatable
  533. debatable jason tatum
  534. jason tatum who's that defense
  535. who's that defense jason tatum is that's what the y
  536. jason tatum is that's what the y yes okay we're going to say he's playing
  537. yes okay we're going to say he's playing forward i don't know if that's true yes
  538. forward i don't know if that's true yes he is a forward
  539. he is a forward okay
  540. okay celtic's defense last night was looking
  541. celtic's defense last night was looking good in the second half
  542. good in the second half matisse diable
  543. matisse diable how do you spell that
  544. how do you spell that t
  545. t i
  546. i s s e
  547. s s e and then i think his last name is
  548. and then i think his last name is t h y
  549. t h y b
  550. b u l l e he's a guard forward for the six
  551. u l l e he's a guard forward for the six series okay and then i'm picking last
  552. series okay and then i'm picking last because this is my spreadsheet my center
  553. because this is my spreadsheet my center is britney grinder guys come on
  554. is britney grinder guys come on [Laughter]
  555. [Laughter] we're bringing her home
  556. we're bringing her home okay
  557. okay so we got our starting five
  558. so we got our starting five and you can see here it's one row per
  559. and you can see here it's one row per person right and whenever you get a
  560. person right and whenever you get a spreadsheet from somebody else you make
  561. spreadsheet from somebody else you make one it's always good to think about what
  562. one it's always good to think about what is a row like is a row a person is it a
  563. is a row like is a row a person is it a transaction from like your credit card
  564. transaction from like your credit card statement
  565. statement is it um an incident that's been
  566. is it um an incident that's been recorded by a government agency is it a
  567. recorded by a government agency is it a thing in the world just like what is the
  568. thing in the world just like what is the row and we'll see this later with the
  569. row and we'll see this later with the police data because there can be sort of
  570. police data because there can be sort of interesting nuances to how you interpret
  571. interesting nuances to how you interpret the data based on what the row kind of
  572. the data based on what the row kind of is right and so in our case
  573. is right and so in our case the row is a person or a player right
  574. the row is a person or a player right okay and so we've got five rows but it's
  575. okay and so we've got five rows but it's actually six rows right because it's a
  576. actually six rows right because it's a header but there's five records
  577. header but there's five records and then we've got two columns we filled
  578. and then we've got two columns we filled out name
  579. out name and position
  580. and position right
  581. right and then the one i'm going to get is
  582. and then the one i'm going to get is their current salary so to do this i'm
  583. their current salary so to do this i'm going to go to basketballreference.com
  584. going to go to basketballreference.com which is just like a nerdy website that
  585. which is just like a nerdy website that has data about players and i'm going to
  586. has data about players and i'm going to get their salary so steph curry's most
  587. get their salary so steph curry's most recent salary guys buckle up is 43
  588. recent salary guys buckle up is 43 million dollars
  589. million dollars geez
  590. geez right so i'm gonna punch that in go
  591. right so i'm gonna punch that in go ahead and punch it in yourself you don't
  592. ahead and punch it in yourself you don't the numbers we put in here don't really
  593. the numbers we put in here don't really matter you can kind of fudge them you
  594. matter you can kind of fudge them you know let me make this a little bigger
  595. know let me make this a little bigger too while we're doing it but i'm going
  596. too while we're doing it but i'm going to punch it in there and you notice that
  597. to punch it in there and you notice that i don't put in any commas or the dollar
  598. i don't put in any commas or the dollar signs because that stuff's for humans
  599. signs because that stuff's for humans that stuff's not for computers the
  600. that stuff's not for computers the computer just wants the number right we
  601. computer just wants the number right we can format the number to look however we
  602. can format the number to look however we want later and i'll show you that in a
  603. want later and i'll show you that in a minute but oftentimes when you're
  604. minute but oftentimes when you're punching in numbers into the into the
  605. punching in numbers into the into the computer especially when you're copying
  606. computer especially when you're copying pasting like i did if you include things
  607. pasting like i did if you include things like dollar signs or commas the computer
  608. like dollar signs or commas the computer can get confused and think it's a word
  609. can get confused and think it's a word and not a number right because the
  610. and not a number right because the computer always has to guess is this a
  611. computer always has to guess is this a number or a word or
  612. number or a word or um
  613. um or what and that affects how it gets
  614. or what and that affects how it gets interpreted when you begin doing
  615. interpreted when you begin doing mathematical formulas and trying to uh
  616. mathematical formulas and trying to uh analyze it that's called the data type
  617. analyze it that's called the data type and so what type of information is in
  618. and so what type of information is in the cell and you know characters versus
  619. the cell and you know characters versus numbers and then dates those three are
  620. numbers and then dates those three are kind of the most common data types and
  621. kind of the most common data types and making sure that the spreadsheet is kind
  622. making sure that the spreadsheet is kind of kosher and happy with them is part of
  623. of kosher and happy with them is part of you know preparing your data for
  624. you know preparing your data for analysis so i put the number in there
  625. analysis so i put the number in there with no commas or dollar signs for now
  626. with no commas or dollar signs for now all right we're gonna go look up lebron
  627. all right we're gonna go look up lebron next
  628. salary a little less making a little less than
  629. a little less making a little less than step
  630. step 39 million
  631. 39 million i'm again just going to pull out those
  632. i'm again just going to pull out those other things oops i put it in the see
  633. other things oops i put it in the see what i did
  634. what i did i made a mistake i put it over steps
  635. i made a mistake i put it over steps number
  636. number so what i can do is i'm going to copy it
  637. so what i can do is i'm going to copy it to make sure i got it but then if i do
  638. to make sure i got it but then if i do edit undo i get steps numbered back
  639. edit undo i get steps numbered back right and the reason it went into that
  640. right and the reason it went into that other one is because the blue selector
  641. other one is because the blue selector box was around c2 i need to click and
  642. box was around c2 i need to click and put it on c3 and then paste it in and
  643. put it on c3 and then paste it in and hit enter which moves me down a row
  644. hit enter which moves me down a row and that's how i get lebron's number
  645. and that's how i get lebron's number incorrectly all right
  646. incorrectly all right let's do jason tatum i like jason tatum
  647. let's do jason tatum i like jason tatum my favorite celtic is marcus smart
  648. my favorite celtic is marcus smart though
  649. though he's wild
  650. he's wild crazy
  651. crazy all right we're going to go get the
  652. all right we're going to go get the salary
  653. salary is it in here
  654. is it in here there it is only 9 million
  655. i said that right i'm just going to search for him copy and paste it oh
  656. search for him copy and paste it oh that bit my
  657. that bit my my like mouse wheel went crazy
  658. my like mouse wheel went crazy okay there we go
  659. paste that in and then i checked this earlier spoiler
  660. and then i checked this earlier spoiler they don't have women's basketball
  661. they don't have women's basketball salaries on the site so i'm just going
  662. salaries on the site so i'm just going to google it real quick
  663. to google it real quick and you maybe want to vet this a little
  664. and you maybe want to vet this a little more but i'm just going to take this
  665. more but i'm just going to take this first site here and go with it and
  666. first site here and go with it and brittany griner what a rip off only
  667. brittany griner what a rip off only making two hundred thousand dollars
  668. making two hundred thousand dollars so we're going to put her in there last
  669. so we're going to put her in there last and we've got those numbers in so i'd
  670. and we've got those numbers in so i'd encourage everybody to just punch in
  671. encourage everybody to just punch in these numbers or really any numbers you
  672. these numbers or really any numbers you want because we're just going to teach
  673. want because we're just going to teach some principles of how to do math it
  674. some principles of how to do math it doesn't really matter what the numbers
  675. doesn't really matter what the numbers are for what we're about to learn
  676. are for what we're about to learn you can see that kind of gathering that
  677. you can see that kind of gathering that data was a key to begin creating
  678. data was a key to begin creating something we can analyze and oftentimes
  679. something we can analyze and oftentimes a lot of the best data journalism
  680. a lot of the best data journalism stories come from data that you create
  681. stories come from data that you create that there's kind of unstructured
  682. that there's kind of unstructured information in the world or sprinkled
  683. information in the world or sprinkled around the internet or hidden in the
  684. around the internet or hidden in the government office and you plug it into a
  685. government office and you plug it into a spreadsheet so you can get it to a place
  686. spreadsheet so you can get it to a place where you can analyze it
  687. where you can analyze it and that's where you're often doing work
  688. and that's where you're often doing work that no one else has done before or done
  689. that no one else has done before or done quite that way that's how i got that
  690. quite that way that's how i got that story way back when
  691. story way back when um everybody good any questions
  692. i think we're good all right so there's one column we're missing which is the
  693. one column we're missing which is the new salary so which is what we're gonna
  694. new salary so which is what we're gonna pay him all right so i'm just gonna
  695. pay him all right so i'm just gonna throw it out to the crowd what should we
  696. throw it out to the crowd what should we pay steph curry somebody shout out a
  697. pay steph curry somebody shout out a number
  698. 30 million 30 million he makes it though
  699. okay lebron what's he getting
  700. 25 million 25 million okay
  701. 25 million okay all right steph's still top dog all
  702. all right steph's still top dog all right jason tatum
  703. 100 million 100 million
  704. 20 million 20 million
  705. you got to get britney at least that much yeah i don't know give her give her
  706. much yeah i don't know give her give her 50 mil give her give her 50 billion give
  707. 50 mil give her give her 50 billion give her
  708. her i mean when she gets home she deserves
  709. i mean when she gets home she deserves something i'll tell you that yeah
  710. okay so we've just got to punch some numbers
  711. so we've just got to punch some numbers in you can punch in different ones you
  712. in you can punch in different ones you want again it's not a big deal but now
  713. want again it's not a big deal but now we've kind of got a complete data set
  714. we've kind of got a complete data set that's ready for a little bit of
  715. that's ready for a little bit of analysis and we can cover some fun
  716. analysis and we can cover some fun spreadsheet tricks um one thing here's
  717. spreadsheet tricks um one thing here's something i teased earlier is you know i
  718. something i teased earlier is you know i told you to punch in these numbers
  719. told you to punch in these numbers without commas or dollar signs so the
  720. without commas or dollar signs so the computer would be happy but that makes
  721. computer would be happy but that makes them kind of hard to read so what if you
  722. them kind of hard to read so what if you want to make it easy for you to read but
  723. want to make it easy for you to read but still be able to analyze it right the
  724. still be able to analyze it right the way you do that is you say select the
  725. way you do that is you say select the column so if i click on the c up at the
  726. column so if i click on the c up at the top that selects all of column c so do
  727. top that selects all of column c so do that
  728. that and then there's a format menu here at
  729. and then there's a format menu here at the top
  730. the top you go and click on format and then
  731. you go and click on format and then there's a
  732. there's a option and you can see here a lot of
  733. option and you can see here a lot of different preset ways that google sheets
  734. different preset ways that google sheets is able to format a number depending on
  735. is able to format a number depending on what kind of thing it is right should it
  736. what kind of thing it is right should it have a decimal place is it a percent
  737. have a decimal place is it a percent that should be taken by a hundred maybe
  738. that should be taken by a hundred maybe you want scientific notation i sure as
  739. you want scientific notation i sure as hell don't but maybe you do
  740. hell don't but maybe you do um
  741. um do you want it rounded or not you know
  742. do you want it rounded or not you know is this a date at not a number like all
  743. is this a date at not a number like all these different things can get sorted
  744. these different things can get sorted out with the format menu so in our case
  745. out with the format menu so in our case i think we want currency rounded right
  746. i think we want currency rounded right we wanted to have a dollar we want the
  747. we wanted to have a dollar we want the commas after every three
  748. commas after every three values and we don't really need the
  749. values and we don't really need the decimal places right
  750. decimal places right so i'm going to select currency rounded
  751. so i'm going to select currency rounded you can pick whatever you'd like it's up
  752. you can pick whatever you'd like it's up to you but you see i that that magically
  753. to you but you see i that that magically put in
  754. put in the commas and the dollar sign on the
  755. the commas and the dollar sign on the front then but if i click on one of the
  756. front then but if i click on one of the values and i look into the
  757. values and i look into the editor here at the top you see that
  758. editor here at the top you see that there aren't actually any commas in it
  759. there aren't actually any commas in it right it's because what's stored in the
  760. right it's because what's stored in the data behind the scenes with sheets is
  761. data behind the scenes with sheets is just the number but then the formatting
  762. just the number but then the formatting for us to look at is sort of like icing
  763. for us to look at is sort of like icing on the cake right
  764. on the cake right and i'm going to do the same thing for
  765. and i'm going to do the same thing for salary i'm going to click on d
  766. salary i'm going to click on d and there's even a little shortcut do
  767. and there's even a little shortcut do you see this little dollar sign here
  768. you see this little dollar sign here in the uh ribbon
  769. in the uh ribbon that will do it for you but you see that
  770. that will do it for you but you see that actually puts those decimal places in
  771. actually puts those decimal places in there which i don't want but there's
  772. there which i don't want but there's another shortcut for that right here
  773. another shortcut for that right here this button will decrease the decimal
  774. this button will decrease the decimal places
  775. places so i hit that twice and boom i've got
  776. so i hit that twice and boom i've got some nice numbers
  777. okay now if you were in a different country
  778. now if you were in a different country sheets would be smart enough to maybe
  779. sheets would be smart enough to maybe put a different currency value in front
  780. put a different currency value in front or use the commas in a different way
  781. or use the commas in a different way because these are
  782. because these are as i suspect you guys know but just say
  783. as i suspect you guys know but just say these are you know american or english
  784. these are you know american or english conventions that aren't shared by
  785. conventions that aren't shared by everyone in the world and so computer
  786. everyone in the world and so computer software often has to internationalize
  787. software often has to internationalize itself to the current locales
  788. itself to the current locales okay
  789. okay everybody get that in
  790. okay so now we've got our starting data set it's what we're here but we want to
  791. set it's what we're here but we want to analyze it we want to ask some questions
  792. analyze it we want to ask some questions and answer them
  793. and answer them numerically
  794. numerically um using the spreadsheet it'll work for
  795. um using the spreadsheet it'll work for our five records but also what's great
  796. our five records but also what's great about spreadsheets is it would work for
  797. about spreadsheets is it would work for 500 million records if you had enough
  798. 500 million records if you had enough these tricks that we learn on this
  799. these tricks that we learn on this little small data set can be easily
  800. little small data set can be easily applied to lots and lots of data once we
  801. applied to lots and lots of data once we learn how to do them right so i'm going
  802. learn how to do them right so i'm going to add a new column which is going to be
  803. to add a new column which is going to be my new calculated thing
  804. my new calculated thing that i'm going to pull out of thin air
  805. that i'm going to pull out of thin air using excel and it's going to be change
  806. using excel and it's going to be change because i want to know the change in
  807. because i want to know the change in salary for each of the five players from
  808. salary for each of the five players from their current salary to what we're going
  809. their current salary to what we're going to pay them right so i just type in
  810. to pay them right so i just type in change i hit b
  811. change i hit b to control b to bold it you know hit
  812. to control b to bold it you know hit that b right there
  813. that b right there and then i click down to e2 so
  814. and then i click down to e2 so here's a really tough math question guys
  815. here's a really tough math question guys how would you calculate the difference
  816. how would you calculate the difference between someone's new salary and their
  817. between someone's new salary and their current salary
  818. subtraction subtraction yeah right and so really you take the new
  819. right and so really you take the new salary you then subtract from it the
  820. salary you then subtract from it the current salary and the difference would
  821. current salary and the difference would be the change right
  822. be the change right and so that could be 30 million
  823. and so that could be 30 million uh minus 43 million in the case of steph
  824. uh minus 43 million in the case of steph curry right and in the case of lebron
  825. curry right and in the case of lebron james it's 25 million minus 39
  826. james it's 25 million minus 39 right so you could just type in these
  827. right so you could just type in these numbers into a calculator and then type
  828. numbers into a calculator and then type the results into the spreadsheet but
  829. the results into the spreadsheet but that would be a lot of work we can have
  830. that would be a lot of work we can have the spreadsheet automatically calculated
  831. the spreadsheet automatically calculated by substituting the battleship positions
  832. by substituting the battleship positions of the cells in the spreadsheet
  833. of the cells in the spreadsheet into a mathematical formula and you do
  834. into a mathematical formula and you do that
  835. that by clicking into the cell where you want
  836. by clicking into the cell where you want to work so i'm going to click on e2 and
  837. to work so i'm going to click on e2 and then you start off your cell with an
  838. then you start off your cell with an equal sign if you just type equal you
  839. equal sign if you just type equal you can then write a mathematical equation
  840. can then write a mathematical equation and the spreadsheet will magically do it
  841. and the spreadsheet will magically do it for you you can see here that google's
  842. for you you can see here that google's even suggesting one for me d2 minus c2
  843. even suggesting one for me d2 minus c2 right it read my mind guys the ai is
  844. right it read my mind guys the ai is here
  845. here and like that is exactly what we want to
  846. and like that is exactly what we want to do we want to take the cell
  847. do we want to take the cell d2 which is steph curry's 30 million
  848. d2 which is steph curry's 30 million dollar salary and we want to subtract
  849. dollar salary and we want to subtract from it cell c2 which is steph curry's
  850. from it cell c2 which is steph curry's current salary so you just do equal d 2
  851. current salary so you just do equal d 2 minus c
  852. minus c 2 right
  853. 2 right hit enter
  854. hit enter boom
  855. boom the result steph cam steph curry is
  856. the result steph cam steph curry is getting his pay cut by 13 million bucks
  857. getting his pay cut by 13 million bucks maybe he did it voluntarily he wants to
  858. maybe he did it voluntarily he wants to play with these guys you know
  859. play with these guys you know um
  860. um and you can see that that's
  861. and you can see that that's automatically been done and then whoa
  862. automatically been done and then whoa check out this like pop-up on my screen
  863. check out this like pop-up on my screen autofill
  864. autofill right google sheets is already saying
  865. right google sheets is already saying well you've done this uh formula for the
  866. well you've done this uh formula for the first cell you want to roll it down
  867. first cell you want to roll it down across the subsequent cells and i'm like
  868. across the subsequent cells and i'm like oh hell yes and so i click that little
  869. oh hell yes and so i click that little plus button there to auto fill and boom
  870. plus button there to auto fill and boom you can see that the change in salaries
  871. you can see that the change in salaries has been automatically calculated for
  872. has been automatically calculated for each row in the spreadsheet here it did
  873. each row in the spreadsheet here it did it for five but if we had everybody in
  874. it for five but if we had everybody in the mba it would do it for ever for
  875. the mba it would do it for ever for every row in the data set all the way to
  876. every row in the data set all the way to the bottom right so
  877. the bottom right so and you can see if i click into that
  878. and you can see if i click into that second cell for lebron instead of d2 to
  879. second cell for lebron instead of d2 to c2 it's now d3 to c3 if i click on the
  880. c2 it's now d3 to c3 if i click on the next one it's d4 to c4 and you can see
  881. next one it's d4 to c4 and you can see that the spreadsheet has just sort of
  882. that the spreadsheet has just sort of upped the number right for each
  883. upped the number right for each subsequent row as it applied the formula
  884. subsequent row as it applied the formula down anybody have problems with that
  885. no okay so then we're going to make another column we're going to get
  886. another column we're going to get statistical i got to stretch for this
  887. statistical i got to stretch for this we're going to do percent change
  888. we're going to do percent change right so in f1 i'm going to type percent
  889. right so in f1 i'm going to type percent change i'm going to bold it and so we
  890. change i'm going to bold it and so we want to know on a percentage basis
  891. want to know on a percentage basis what's the change for each person right
  892. what's the change for each person right so who here is such a mathematical
  893. so who here is such a mathematical wizard that they can share with the
  894. wizard that they can share with the group
  895. group the formula for percent change somebody
  896. the formula for percent change somebody here knows it
  897. math math math i think you got the wrong crowd i'm not
  898. i think you got the wrong crowd i'm not gonna lie i don't know
  899. all right so maybe not you can be you can get pretty far in
  900. you can be you can get pretty far in journalism with just one or two
  901. journalism with just one or two mathematical tricks but every report i
  902. mathematical tricks but every report i was gonna say
  903. was gonna say you know you should know how to do a
  904. you know you should know how to do a percent change right which is you know
  905. percent change right which is you know like if if there's 10 of something and
  906. like if if there's 10 of something and it goes up to 20 it's doubled or that's
  907. it goes up to 20 it's doubled or that's a hundred percent change right an
  908. a hundred percent change right an increase or decrease and that's a way of
  909. increase or decrease and that's a way of coming up with changes that you can
  910. coming up with changes that you can compare for different groups obviously
  911. compare for different groups obviously there's pitfalls to it really small
  912. there's pitfalls to it really small numbers can have really large percent
  913. numbers can have really large percent changes that can be misleading right
  914. changes that can be misleading right there can be volatility in the data from
  915. there can be volatility in the data from data point to data point that can lead
  916. data point to data point that can lead to noise which is why everybody does
  917. to noise which is why everybody does seven day averages with coven if you've
  918. seven day averages with coven if you've noticed right
  919. noticed right um but it's a really good formula to
  920. um but it's a really good formula to learn and um the way i keep it memorized
  921. learn and um the way i keep it memorized in my head because i'm not a
  922. in my head because i'm not a mathematical
  923. mathematical with either is with this little say
  924. with either is with this little say it's new minus old divided by old right
  925. it's new minus old divided by old right so you take the new value which is what
  926. so you take the new value which is what we're going to pay somebody you subtract
  927. we're going to pay somebody you subtract from it
  928. from it the old value hey we already did that in
  929. the old value hey we already did that in column e right that's new minus old
  930. column e right that's new minus old right there and then you divide that
  931. right there and then you divide that against the old value or the original
  932. against the old value or the original value
  933. value and that will return the percentage
  934. and that will return the percentage change
  935. change and so it's this minus this divided by
  936. and so it's this minus this divided by this
  937. this right is what's going to get us there
  938. right is what's going to get us there and so you can
  939. in the spreadsheet really write formulas of pretty great complexity following the
  940. of pretty great complexity following the same principles that we just did before
  941. same principles that we just did before so i want to do new minus old what do i
  942. so i want to do new minus old what do i type just to do that right here guys
  943. type just to do that right here guys what should i type in somebody
  944. d3 or sorry
  945. or sorry wait yeah d2 it's all right here we got
  946. wait yeah d2 it's all right here we got it
  947. it minus c2 minus c2 and then i want to
  948. minus c2 minus c2 and then i want to divide it against c2 so that's divided
  949. divide it against c2 so that's divided by c2 right all right so that's that's
  950. by c2 right all right so that's that's my first hunch does anybody see what's
  951. my first hunch does anybody see what's wrong with this anybody remember from
  952. wrong with this anybody remember from high school or
  953. high school or elementary school
  954. the order of operations order of operations right so in math it'll do the
  955. operations right so in math it'll do the division before it does the subtraction
  956. division before it does the subtraction right which would screw up the math and
  957. right which would screw up the math and so just like in like pre-algebra class
  958. so just like in like pre-algebra class you can put a little percentage sign
  959. you can put a little percentage sign around the subtraction d2 minus c2
  960. around the subtraction d2 minus c2 inside
  961. inside parentheses which will which will ensure
  962. parentheses which will which will ensure that that runs first right and then the
  963. that that runs first right and then the result of that is then divided into c2
  964. result of that is then divided into c2 and so if i do that and hit enter
  965. and so if i do that and hit enter right you can see that it's negative 0.3
  966. right you can see that it's negative 0.3 well is that a percent what's wrong
  967. you need to format it as a percent right because remember percentages are
  968. right because remember percentages are often just you know fractions between 0
  969. often just you know fractions between 0 and 1 or 1 and negative 1.
  970. and 1 or 1 and negative 1. and when we read them we often take them
  971. and when we read them we often take them times a hundred and like round them off
  972. times a hundred and like round them off right
  973. right and so you could you could put times 100
  974. and so you could you could put times 100 in your formula and that would work and
  975. in your formula and that would work and there'd be nothing wrong with it right
  976. there'd be nothing wrong with it right but like another trick is just to run it
  977. but like another trick is just to run it this way and then hit this little
  978. this way and then hit this little percentage formatting button right there
  979. percentage formatting button right there right and boom that'll do the math for
  980. right and boom that'll do the math for you and that tells us steph curry's
  981. you and that tells us steph curry's taking a 30
  982. taking a 30 pay cut
  983. pay cut right
  984. right okay so there was that auto fill trick
  985. okay so there was that auto fill trick before which is great but there's
  986. before which is great but there's another way to roll down your formulas i
  987. another way to roll down your formulas i want to make sure everybody knows
  988. want to make sure everybody knows clicking here on f2 you see the blue box
  989. clicking here on f2 you see the blue box around it and then in the lower right
  990. around it and then in the lower right hand corner there's kind of this like
  991. hand corner there's kind of this like little fatty box there
  992. little fatty box there and that also doesn't have a name as far
  993. and that also doesn't have a name as far as i know i think some people call it
  994. as i know i think some people call it the magic corner
  995. the magic corner or whatever but it's really just kind of
  996. or whatever but it's really just kind of this little handlebar and you'll notice
  997. this little handlebar and you'll notice when you hover over it the cursor
  998. when you hover over it the cursor changes into this crosshairs again i
  999. changes into this crosshairs again i don't know why a crosshairs just nerds
  1000. don't know why a crosshairs just nerds did it that way but what you're going to
  1001. did it that way but what you're going to do is you're going to hover till you get
  1002. do is you're going to hover till you get that crosshairs you're going to click
  1003. that crosshairs you're going to click and hold it down and then if you drag
  1004. and hold it down and then if you drag that
  1005. that down to f6 you can see that it applies
  1006. down to f6 you can see that it applies the formula across all those cells
  1007. the formula across all those cells that's another way to do the autofill
  1008. that's another way to do the autofill is to just grab that corner and drag it
  1009. is to just grab that corner and drag it down another way is just to double click
  1010. down another way is just to double click it
  1011. it and boom it'll fill all the way to the
  1012. and boom it'll fill all the way to the bottom for you
  1013. bottom for you and so we could see that we gave some
  1014. and so we could see that we gave some pretty big raises at the bottom of our
  1015. pretty big raises at the bottom of our sheet and a little at the top we took
  1016. sheet and a little at the top we took from the one percent guys
  1017. from the one percent guys we're we're the robin hood owners here
  1018. we're we're the robin hood owners here okay
  1019. okay that work for everybody any questions
  1020. there's actually a really bad joke about new minus old divided by old which is uh
  1021. new minus old divided by old which is uh you know it's there's like it's got it's
  1022. you know it's there's like it's got it's even like structured like a joke hey did
  1023. even like structured like a joke hey did anybody here get into journalism to make
  1024. anybody here get into journalism to make a lot of money
  1025. a lot of money no
  1026. no get it
  1027. i don't know if i like that but that is a joke that people tell
  1028. but that is a joke that people tell it is a joke that people tell that will
  1029. it is a joke that people tell that will be my halloween joke next year
  1030. be my halloween joke next year yeah
  1031. um okay so we've we've
  1032. okay so we've we've computed some new columns or calculated
  1033. computed some new columns or calculated some new columns by tacking them on and
  1034. some new columns by tacking them on and writing formulas right and so that's
  1035. writing formulas right and so that's called computing or calculating but like
  1036. called computing or calculating but like another common mathematical thing is to
  1037. another common mathematical thing is to aggregate it right is to add it all up i
  1038. aggregate it right is to add it all up i don't want to annotate a new column or a
  1039. don't want to annotate a new column or a new field on each row i want to look
  1040. new field on each row i want to look across all the fields and come up with
  1041. across all the fields and come up with some high level statistics right like
  1042. some high level statistics right like what is the total amount of money that
  1043. what is the total amount of money that we're spending as a team
  1044. we're spending as a team right what's the average salary on the
  1045. right what's the average salary on the team or the median right and those
  1046. team or the median right and those aggregated statistics can also be
  1047. aggregated statistics can also be calculated really easily
  1048. calculated really easily in a spreadsheet tool
  1049. in a spreadsheet tool using different formulas using functions
  1050. using different formulas using functions that are pre-written to do aggregations
  1051. that are pre-written to do aggregations that work a little differently okay and
  1052. that work a little differently okay and to do that we're going to go down um
  1053. to do that we're going to go down um beneath our data set here we want to
  1054. beneath our data set here we want to kind of keep this as its own thing we
  1055. kind of keep this as its own thing we don't want what we're about to do to get
  1056. don't want what we're about to do to get kind of confused with it as we work it
  1057. kind of confused with it as we work it later but um
  1058. later but um so we're going to go down below we're
  1059. so we're going to go down below we're going to create a little island a little
  1060. going to create a little island a little separate data set down below so just
  1061. separate data set down below so just click like three or four rows
  1062. click like three or four rows down below the total maybe click on like
  1063. down below the total maybe click on like c9
  1064. c9 and the question that i want to answer
  1065. and the question that i want to answer here
  1066. here is what was the total amount of money
  1067. is what was the total amount of money that all these players last year
  1068. that all these players last year right and the mathematical term for
  1069. right and the mathematical term for totaling something is summing it right
  1070. totaling something is summing it right the sum total and so if you hit equal
  1071. the sum total and so if you hit equal again and then just type the word sum
  1072. again and then just type the word sum see this like magic pull down whether
  1073. see this like magic pull down whether you know it or not available to you
  1074. you know it or not available to you within your spreadsheet is literally
  1075. within your spreadsheet is literally hundreds of magic functions that can
  1076. hundreds of magic functions that can take in cells or lists of cells and
  1077. take in cells or lists of cells and return some computed or aggregated
  1078. return some computed or aggregated result right and that includes all the
  1079. result right and that includes all the mathematical operations that the
  1080. mathematical operations that the calculator can do right like the sum and
  1081. calculator can do right like the sum and so by typing equal sum and then a
  1082. so by typing equal sum and then a parenthesis it's expecting you after the
  1083. parenthesis it's expecting you after the parenthesis to tell it what you want it
  1084. parenthesis to tell it what you want it to sum and i wanted to sum these five
  1085. to sum and i wanted to sum these five rows right in column c and so i'm gonna
  1086. rows right in column c and so i'm gonna while that little open parenthesis is
  1087. while that little open parenthesis is hanging there i'm just gonna go click
  1088. hanging there i'm just gonna go click and drag down across those five rows and
  1089. and drag down across those five rows and you can see it now says c2 colon c6 so
  1090. you can see it now says c2 colon c6 so that just means the series of rows that
  1091. that just means the series of rows that starts here and ends there right that
  1092. starts here and ends there right that list of them is going to be fed into the
  1093. list of them is going to be fed into the sum function
  1094. sum function i then close the parenthesis and we have
  1095. i then close the parenthesis and we have a complete function
  1096. a complete function and then hit enter
  1097. and then hit enter boom the sum total right and we can see
  1098. boom the sum total right and we can see that the current salaries of all of our
  1099. that the current salaries of all of our players together
  1100. players together is 95 million dollars and just here on
  1101. is 95 million dollars and just here on the left i'm going to type sum
  1102. the left i'm going to type sum just so that's kind of labeled right
  1103. just so that's kind of labeled right now
  1104. now if i want to
  1105. if i want to do that same thing for our new salary
  1106. do that same thing for our new salary what's our payroll this year on our
  1107. what's our payroll this year on our dream team
  1108. dream team using only tricks we've learned this far
  1109. using only tricks we've learned this far can anyone guess how i could with just a
  1110. can anyone guess how i could with just a flick of the wrist quickly calculate
  1111. flick of the wrist quickly calculate everything the same thing for column d
  1112. the magic corner the magic corner so if i click on c2
  1113. i click on c2 i go to the magic corner hold and drag
  1114. i go to the magic corner hold and drag to the right
  1115. to the right boom we can see when i click on it that
  1116. boom we can see when i click on it that c2 to c6 is now d2 to d6 right it's just
  1117. c2 to c6 is now d2 to d6 right it's just kind of magically carried over to the
  1118. kind of magically carried over to the next column and we can see that our
  1119. next column and we can see that our payroll has more than doubled
  1120. payroll has more than doubled right
  1121. right and you know i might see this as like
  1122. and you know i might see this as like the head of basketball operations on our
  1123. the head of basketball operations on our dream team and say we really gotta bring
  1124. dream team and say we really gotta bring down how much jason tatum's making like
  1125. down how much jason tatum's making like he's good but i'm gonna cut that in half
  1126. he's good but i'm gonna cut that in half and so if i just go up above and i
  1127. and so if i just go up above and i instead put in 50 million cut his pay in
  1128. instead put in 50 million cut his pay in half see how everything instantly
  1129. half see how everything instantly updated right so our sum total below
  1130. updated right so our sum total below went down 50 million and his his change
  1131. went down 50 million and his his change values instantly updated right if i put
  1132. values instantly updated right if i put one dollar in there you would see that
  1133. one dollar in there you would see that they changed to reflect that if i undo
  1134. they changed to reflect that if i undo and go back to where i was before it
  1135. and go back to where i was before it changes and this is part of the magic of
  1136. changes and this is part of the magic of the spreadsheet right is that once the
  1137. the spreadsheet right is that once the formula is in place if the numbers
  1138. formula is in place if the numbers change everything automatically updates
  1139. okay so sums one good statistic another one is the average right which is just
  1140. one is the average right which is just what's the sort of take all of them
  1141. what's the sort of take all of them together and divide it by the number of
  1142. together and divide it by the number of members
  1143. members what do we get so the average can be
  1144. what do we get so the average can be calculated by an equal
  1145. calculated by an equal uh mean
  1146. uh mean no average sometimes it's called the
  1147. no average sometimes it's called the mean sometimes it's called the average
  1148. mean sometimes it's called the average i often struggle is it average or is it
  1149. i often struggle is it average or is it mean guys
  1150. mean guys i think it's is it mean so let's just
  1151. i think it's is it mean so let's just try it
  1152. try it no so like the way i could do this like
  1153. no so like the way i could do this like let's say you're ben and you can't
  1154. let's say you're ben and you can't remember you just go to your buddy
  1155. remember you just go to your buddy google and you do like google
  1156. google and you do like google sheets
  1157. sheets average formula yeah what is that again
  1158. average formula yeah what is that again here's google has a website and they
  1159. here's google has a website and they have a page for every one of these
  1160. have a page for every one of these formulas that like tells you what the
  1161. formulas that like tells you what the name of it is defines it tell you how to
  1162. name of it is defines it tell you how to do it you can see here on the right
  1163. do it you can see here on the right there's literally hundreds of these like
  1164. there's literally hundreds of these like crazy math functions
  1165. crazy math functions and you can see that no band in this
  1166. and you can see that no band in this case it's not called the mean it's
  1167. case it's not called the mean it's called the average so i do equal average
  1168. called the average so i do equal average open parenthesis drag my c cells
  1169. open parenthesis drag my c cells hit enter
  1170. hit enter boom
  1171. boom right we can see that
  1172. right we can see that the average salary
  1173. the average salary is 19 000.
  1174. is 19 000. i then can just grab the corner drag to
  1175. i then can just grab the corner drag to the right we can see that's gone up to
  1176. the right we can see that's gone up to 35 million right
  1177. 35 million right that's because we really raised up
  1178. that's because we really raised up people on the low end right because as
  1179. people on the low end right because as we know in averages things that are
  1180. we know in averages things that are really high or really low can kind of
  1181. really high or really low can kind of skew the numbers right so when it comes
  1182. skew the numbers right so when it comes to something like home sales or income
  1183. to something like home sales or income as is very well publicized there is a
  1184. as is very well publicized there is a small number of people that have really
  1185. small number of people that have really expensive homes and make lots and lots
  1186. expensive homes and make lots and lots of money and so they can kind of skew
  1187. of money and so they can kind of skew the results of an average in case it's
  1188. the results of an average in case it's not in a way that's saying wrong but
  1189. not in a way that's saying wrong but they can be a little misleading about
  1190. they can be a little misleading about what the typical sort of situation is
  1191. what the typical sort of situation is right does anybody know what the
  1192. right does anybody know what the statistical solution
  1193. statistical solution or alternative to an average is in a
  1194. or alternative to an average is in a case where you have skewed data
  1195. so the standard deviation that's part of it but that's not the
  1196. that's part of it but that's not the number that's not exactly it
  1197. number that's not exactly it the standard deviation is a little more
  1198. the standard deviation is a little more sophisticated
  1199. looking at the median the median who said that
  1200. too i don't know if i i don't know i feel like claire and i said at the same
  1201. feel like claire and i said at the same time
  1202. time okay i didn't say anything no you got it
  1203. okay i didn't say anything no you got it you didn't say it
  1204. you didn't say it no do you want what's the median explain
  1205. no do you want what's the median explain it the median is the one that's in the
  1206. it the median is the one that's in the dead middle and so it's not affected by
  1207. dead middle and so it's not affected by you know the really high
  1208. you know the really high the highest number or the lowest number
  1209. the highest number or the lowest number it's just kind of like what's right in
  1210. it's just kind of like what's right in the middle of the set of numbers yeah
  1211. the middle of the set of numbers yeah you just sort everybody from highest to
  1212. you just sort everybody from highest to lowest like say there's 100 people you
  1213. lowest like say there's 100 people you sort them and after you sort them by the
  1214. sort them and after you sort them by the value whoever is the 50th one is the
  1215. value whoever is the 50th one is the media right it's also you know it's also
  1216. media right it's also you know it's also the 50th percentile is like another way
  1217. the 50th percentile is like another way of thinking about it if you're familiar
  1218. of thinking about it if you're familiar with percentiles
  1219. with percentiles and you know you'll notice this in the
  1220. and you know you'll notice this in the news when people talk about home sales
  1221. news when people talk about home sales this is really common right it's always
  1222. this is really common right it's always the median sales price of homes in the
  1223. the median sales price of homes in the news right and that's the reason is
  1224. news right and that's the reason is because there's really a small number of
  1225. because there's really a small number of high ones that skew the number and there
  1226. high ones that skew the number and there could be a difference and so really
  1227. could be a difference and so really whenever you're doing a story if you
  1228. whenever you're doing a story if you calculate an average it's kind of good
  1229. calculate an average it's kind of good practice to look at the median just to
  1230. practice to look at the median just to kind of see if there's much difference
  1231. kind of see if there's much difference between the two the average while
  1232. between the two the average while statistically not as good in some ways
  1233. statistically not as good in some ways is is sometimes better to use because
  1234. is is sometimes better to use because it's just more commonplace you know when
  1235. it's just more commonplace you know when you're writing a news story people know
  1236. you're writing a news story people know what that is it's easy to get across
  1237. what that is it's easy to get across right
  1238. right and so um
  1239. and so um i'll sometimes use it if they're really
  1240. i'll sometimes use it if they're really close just because it's easier but the
  1241. close just because it's easier but the key thing to really contemplate is is my
  1242. key thing to really contemplate is is my data set skewed you know is there a few
  1243. data set skewed you know is there a few records that are really far out
  1244. records that are really far out different in the case of brittany griner
  1245. different in the case of brittany griner i think we kind of have a little bit of
  1246. i think we kind of have a little bit of skewing there right
  1247. skewing there right um so to calculate the median i'm going
  1248. um so to calculate the median i'm going to make another row to put median and
  1249. to make another row to put median and like you literally just do equal median
  1250. like you literally just do equal median right it's that easy
  1251. right it's that easy open parenthesis
  1252. open parenthesis drag
  1253. drag close parenthesis
  1254. close parenthesis boom and we can see there is quite a
  1255. boom and we can see there is quite a difference there right
  1256. difference there right uh now in the case of the median it's
  1257. uh now in the case of the median it's jason tatum who is the medium value
  1258. jason tatum who is the medium value right of the five he ranks in the middle
  1259. right of the five he ranks in the middle so he becomes the median if i drag it to
  1260. so he becomes the median if i drag it to the right and look at the median our
  1261. the right and look at the median our median becomes 30 million steph curry
  1262. any questions about the average the median
  1263. there's many other things minimum maximum
  1264. maximum standard deviation is one which helps
  1265. standard deviation is one which helps you understand the distribution
  1266. you understand the distribution the curve
  1267. okay then we're going to cover we're going to move on to our next basic
  1268. we're going to move on to our next basic skill which is sorting and filtering
  1269. skill which is sorting and filtering right so let's say you have a data set
  1270. right so let's say you have a data set but you want to resort it to say what
  1271. but you want to resort it to say what comes out on top what comes out on the
  1272. comes out on top what comes out on the bottom you want to focus in just on one
  1273. bottom you want to focus in just on one subset of the records this is where
  1274. subset of the records this is where sorting and filtering comes in handy and
  1275. sorting and filtering comes in handy and you can do that really easily in any
  1276. you can do that really easily in any spreadsheet tool by putting sort of a
  1277. spreadsheet tool by putting sort of a magic filter at the top
  1278. magic filter at the top you can do so what we want to do is want
  1279. you can do so what we want to do is want to select all of our data so i just
  1280. to select all of our data so i just clicked on f6 and i dragged and i
  1281. clicked on f6 and i dragged and i selected all the data that's outside of
  1282. selected all the data that's outside of my little island my statistical island
  1283. my little island my statistical island at the bottom there
  1284. at the bottom there and then
  1285. and then [Music]
  1286. [Music] i'm going to click on the data pull down
  1287. i'm going to click on the data pull down menu at the top and you see that there's
  1288. menu at the top and you see that there's this option called create a filter
  1289. this option called create a filter i want everybody to click on that
  1290. i want everybody to click on that and you'll see that not a lot of changes
  1291. and you'll see that not a lot of changes but there's these kind of magic green
  1292. but there's these kind of magic green arrows on each of our header rows right
  1293. arrows on each of our header rows right and excel has something just like this
  1294. and excel has something just like this it's just in a slightly different place
  1295. it's just in a slightly different place and these magic arrows now have like
  1296. and these magic arrows now have like options for us that allow us to
  1297. options for us that allow us to manipulate
  1298. manipulate our data set right so if i wanted to
  1299. our data set right so if i wanted to sort people by their current salary from
  1300. sort people by their current salary from highest to lowest so maybe i could see
  1301. highest to lowest so maybe i could see the median
  1302. the median i click the little green guy
  1303. i click the little green guy and you can see that there's two sorting
  1304. and you can see that there's two sorting options a to z sorting in ascending
  1305. options a to z sorting in ascending order or z to a sorting in descending
  1306. order or z to a sorting in descending order and so since i want the top salary
  1307. order and so since i want the top salary at the top
  1308. at the top i'm going to sort z to a
  1309. i'm going to sort z to a and we can see that that shifted things
  1310. and we can see that that shifted things with jason tatum at the bottom i think
  1311. with jason tatum at the bottom i think it was like that to start with i'm going
  1312. it was like that to start with i'm going to do it on d and do z to a and you can
  1313. to do it on d and do z to a and you can see that they resort right we now have
  1314. see that they resort right we now have tatum and grinder at the top and matisse
  1315. tatum and grinder at the top and matisse thigh bowl at the bottom if we wanted to
  1316. thigh bowl at the bottom if we wanted to sort by percentage change who had the
  1317. sort by percentage change who had the biggest percentage change i could do a
  1318. biggest percentage change i could do a sort there as well
  1319. and if you're looking at a column like name it'll just sort alphabetically
  1320. name it'll just sort alphabetically right as opposed to sorting numerically
  1321. right as opposed to sorting numerically and that's where the data type comes in
  1322. and that's where the data type comes in because depending on whether your column
  1323. because depending on whether your column is words or numbers or dates how the
  1324. is words or numbers or dates how the computer sorts it differs right and so
  1325. computer sorts it differs right and so that's why it's important to kind of get
  1326. that's why it's important to kind of get that right
  1327. that right the other thing is a filter let's say we
  1328. the other thing is a filter let's say we wanted to look just at our guards for
  1329. wanted to look just at our guards for instance i click on the position column
  1330. instance i click on the position column you can see that it gives you the three
  1331. you can see that it gives you the three unique values there's center forwards
  1332. unique values there's center forwards and guards if i just uncheck
  1333. and guards if i just uncheck center and forward see i just clicked on
  1334. center and forward see i just clicked on there to uncheck it and hit ok
  1335. there to uncheck it and hit ok boom the data set is filtered down to
  1336. boom the data set is filtered down to just the guards and you can see that
  1337. just the guards and you can see that there's now a little funnel icon
  1338. there's now a little funnel icon to indicate we've done a filter
  1339. to indicate we've done a filter if i click again and i switch it to
  1340. if i click again and i switch it to forward
  1341. forward it's filtered to our one or two forwards
  1342. it's filtered to our one or two forwards right if i click it again and i click on
  1343. right if i click it again and i click on select all
  1344. select all we have everybody
  1345. we have everybody that was a character filter based on a
  1346. that was a character filter based on a categorical value but you can also do
  1347. categorical value but you can also do filters that are numerical let's say we
  1348. filters that are numerical let's say we want to see everyone who makes more than
  1349. want to see everyone who makes more than 10 million dollars right you can do
  1350. 10 million dollars right you can do what's called filtering by condition
  1351. what's called filtering by condition and then you can say greater than less
  1352. and then you can say greater than less than equal to et cetera and because this
  1353. than equal to et cetera and because this is a numerical field you can do your
  1354. is a numerical field you can do your filters with a numerical expression so
  1355. filters with a numerical expression so if i say i want to see everyone who made
  1356. if i say i want to see everyone who made more than 25 million
  1357. more than 25 million i just would select greater than the
  1358. i just would select greater than the condition i'd type in to great 25
  1359. condition i'd type in to great 25 million
  1360. million and you can see that there's only two
  1361. and you can see that there's only two players who then qualify right and this
  1362. players who then qualify right and this these sorting and filtering can be a
  1363. these sorting and filtering can be a good way to ask and answer some basic
  1364. good way to ask and answer some basic questions that you have about your data
  1365. questions that you have about your data set as you interview it as you try to
  1366. set as you interview it as you try to find a story in it right
  1367. okay any questions about sorting and
  1368. any questions about sorting and filtering
  1369. okay so i'm going to do one pop quiz question so if i wanted to get the grand
  1370. question so if i wanted to get the grand total of money spent across both years
  1371. total of money spent across both years of all salaries
  1372. of all salaries how would i do that what would i do in
  1373. how would i do that what would i do in the spreadsheet somebody pipe up and
  1374. the spreadsheet somebody pipe up and tell me what to type
  1375. i think you would do equal sum and then take your
  1376. take your magic little dragger and go across
  1377. magic little dragger and go across current salary and new salary
  1378. current salary and new salary just like that right selecting both
  1379. just like that right selecting both that's definitely a way to do it
  1380. that's definitely a way to do it we get 270 million another way would be
  1381. we get 270 million another way would be to sum the computed columns i could just
  1382. to sum the computed columns i could just click on the two totals for each year
  1383. click on the two totals for each year there
  1384. there and do that and that would work as well
  1385. and do that and that would work as well it comes out the same because you can do
  1386. it comes out the same because you can do formulas of formulas right and it all
  1387. formulas of formulas right and it all adds up together
  1388. adds up together i might even say this is current this is
  1389. i might even say this is current this is new and this is total okay if i wanted
  1390. new and this is total okay if i wanted to get the average salary across both
  1391. to get the average salary across both years how would i do that
  1392. it would just be equals average and then you would do
  1393. you would do parentheses current salary and then
  1394. parentheses current salary and then through the new salary right
  1395. through the new salary right yep equal average and drag them across
  1396. yep equal average and drag them across there so it's the same thing
  1397. there so it's the same thing that claire did for some it's just we
  1398. that claire did for some it's just we typed average instead of sum as our
  1399. typed average instead of sum as our formula right because it's the same
  1400. formula right because it's the same range of cells it's called a range the
  1401. range of cells it's called a range the sort of list of cells but it's just
  1402. sort of list of cells but it's just going into a different function
  1403. going into a different function right and i could do the same thing with
  1404. right and i could do the same thing with median as well
  1405. and if i wanted to do the change in my salary from year to year i could just
  1406. salary from year to year i could just simply do equals
  1407. simply do equals this
  1408. this minus this right and we've calculated
  1409. minus this right and we've calculated that i can autofill and we can see the
  1410. that i can autofill and we can see the change in each of those values from year
  1411. change in each of those values from year to year as a grand total too
  1412. to year as a grand total too and because we've kept this separately
  1413. and because we've kept this separately as an island the sort and filter doesn't
  1414. as an island the sort and filter doesn't really affect this that's why it's nice
  1415. really affect this that's why it's nice to leave that little gap in there
  1416. that's covered sorting and filtering i have one more basic skill thing we're
  1417. have one more basic skill thing we're going to learn which is sort of the
  1418. going to learn which is sort of the crowning skill of spreadsheets before we
  1419. crowning skill of spreadsheets before we get into our real data set but before i
  1420. get into our real data set but before i do that i just want to give a second in
  1421. do that i just want to give a second in case there's any questions
  1422. okay great so the next thing we're going to cover is how to group and aggregate
  1423. to cover is how to group and aggregate for groups within the data set right we
  1424. for groups within the data set right we were able to compute a value for each
  1425. were able to compute a value for each row we were able to aggregate values
  1426. row we were able to aggregate values across the entire data set but there's
  1427. across the entire data set but there's another thing that's a really common
  1428. another thing that's a really common kind of deal
  1429. kind of deal which is to look at to do aggregate
  1430. which is to look at to do aggregate values for groups of rows as opposed for
  1431. values for groups of rows as opposed for all the rows so for instance if you
  1432. all the rows so for instance if you wanted to ask the question what's the
  1433. wanted to ask the question what's the total amount of money we're paying our
  1434. total amount of money we're paying our guards versus our forwards right or if
  1435. guards versus our forwards right or if we added a column that was gender let's
  1436. we added a column that was gender let's do it let's add a column this gender
  1437. do it let's add a column this gender right oh see how i did that i clicked on
  1438. right oh see how i did that i clicked on b
  1439. b i right clicked it says insert one
  1440. i right clicked it says insert one column left boom i get a new column so
  1441. column left boom i get a new column so if i do gender and i just do you know f
  1442. if i do gender and i just do you know f m
  1443. m m right for the two the genders in this
  1444. m right for the two the genders in this case or the sexes i should say um
  1445. case or the sexes i should say um let's re-label that right
  1446. let's re-label that right because i think brittany griner is
  1447. because i think brittany griner is non-binary is that correct i can't be
  1448. non-binary is that correct i can't be corrected on this book yeah i think they
  1449. corrected on this book yeah i think they are
  1450. are yeah and um
  1451. yeah and um and so you know forgive the vulgarity of
  1452. and so you know forgive the vulgarity of this classification just to make the
  1453. this classification just to make the point
  1454. point um
  1455. um uh
  1456. uh you know by putting in these these
  1457. you know by putting in these these categorical values we've created uh
  1458. categorical values we've created uh records that can effectively be grouped
  1459. records that can effectively be grouped together for analysis and so you can use
  1460. together for analysis and so you can use this position or this sex column to look
  1461. this position or this sex column to look at what's the average salary for men
  1462. at what's the average salary for men versus women right to maybe do a gender
  1463. versus women right to maybe do a gender pay gap analysis or to look at the
  1464. pay gap analysis or to look at the different positions to see how the team
  1465. different positions to see how the team is investing in one thing versus another
  1466. is investing in one thing versus another or look at it by age or other kind of
  1467. or look at it by age or other kind of common attributes eye color if you
  1468. common attributes eye color if you wanted to right and to do those kind of
  1469. wanted to right and to do those kind of group end counts or group and sums which
  1470. group end counts or group and sums which really are pretty common
  1471. really are pretty common there's a technique that every
  1472. there's a technique that every spreadsheet can do and for some weird
  1473. spreadsheet can do and for some weird nerdy reason is almost always called a
  1474. nerdy reason is almost always called a pivot table have you guys heard this
  1475. pivot table have you guys heard this before
  1476. so a pivot table is just a really dumb piece of jargon from like the 1980s like
  1477. piece of jargon from like the 1980s like clippy the paperclip or whatever which
  1478. clippy the paperclip or whatever which really is just a sort of brand name in
  1479. really is just a sort of brand name in the spreadsheet software for grouping
  1480. the spreadsheet software for grouping and counting or grouping and summing or
  1481. and counting or grouping and summing or grouping and averaging using one of your
  1482. grouping and averaging using one of your columns and this is a really powerful
  1483. columns and this is a really powerful tool that can you know win you a
  1484. tool that can you know win you a pulitzer prize or whatever and
  1485. pulitzer prize or whatever and oftentimes result in good stories
  1486. oftentimes result in good stories because with just a couple clicks of the
  1487. because with just a couple clicks of the mouse right and so
  1488. mouse right and so let's do it let's make a pivot table so
  1489. let's do it let's make a pivot table so we want to select the data that we want
  1490. we want to select the data that we want to group and count so again i'm just
  1491. to group and count so again i'm just going to drag my mouse over kind of
  1492. going to drag my mouse over kind of everything from a1 to g6 right
  1493. and so that now is selected in blue kind of my data set and the and i want to ask
  1494. of my data set and the and i want to ask questions that have to do with position
  1495. questions that have to do with position and have to do with sex right and so i
  1496. and have to do with sex right and so i selected the data that i'd like to pivot
  1497. selected the data that i'd like to pivot and then again this button will be in a
  1498. and then again this button will be in a different place in depending on which
  1499. different place in depending on which program you're using but it's almost
  1500. program you're using but it's almost always called a pivot table
  1501. always called a pivot table and in google sheets you click on insert
  1502. and in google sheets you click on insert and then you see that right there pivot
  1503. and then you see that right there pivot table it can also put in charts that's a
  1504. table it can also put in charts that's a whole other world we won't get to
  1505. whole other world we won't get to but
  1506. but insert pivot table if you do that it's
  1507. insert pivot table if you do that it's going to say well here what data do you
  1508. going to say well here what data do you want me to use and this is the data
  1509. want me to use and this is the data range a1 to g6 that's good that's what i
  1510. range a1 to g6 that's good that's what i want
  1511. want and then it's like okay where do you
  1512. and then it's like okay where do you want to put the pivot table and the
  1513. want to put the pivot table and the right answer almost always is new sheet
  1514. right answer almost always is new sheet which is good so just assuming you see
  1515. which is good so just assuming you see something like that just hit create
  1516. something like that just hit create and that's going to take you to a new
  1517. and that's going to take you to a new spreadsheet
  1518. spreadsheet and you might be like holy cow where'd
  1519. and you might be like holy cow where'd my data go right and this is where it's
  1520. my data go right and this is where it's important to kind of stop and make sure
  1521. important to kind of stop and make sure you get one of the basic user interface
  1522. you get one of the basic user interface things about a spreadsheet that i
  1523. things about a spreadsheet that i haven't covered which is that your
  1524. haven't covered which is that your spreadsheet is a workbook that can have
  1525. spreadsheet is a workbook that can have multiple spreadsheets within it you can
  1526. multiple spreadsheets within it you can have kind of a bundle of spreadsheets in
  1527. have kind of a bundle of spreadsheets in your book so to speak and those are
  1528. your book so to speak and those are controlled in the lower left hand corner
  1529. controlled in the lower left hand corner do you see these little tabs down here
  1530. do you see these little tabs down here so sheet1 if i click there takes me back
  1531. so sheet1 if i click there takes me back to where i started i didn't lose my data
  1532. to where i started i didn't lose my data but then the pivot table was inserted
  1533. but then the pivot table was inserted into sheet two what's called pivot table
  1534. into sheet two what's called pivot table one by default right
  1535. one by default right which is
  1536. which is um where the new calculated values are
  1537. um where the new calculated values are going to be placed you can even name
  1538. going to be placed you can even name your tabs so if i click on the arrow on
  1539. your tabs so if i click on the arrow on sheet one and hit rename i can type
  1540. sheet one and hit rename i can type roster right because that's like my
  1541. roster right because that's like my roster of players
  1542. roster of players but we're going to go back to the pivot
  1543. but we're going to go back to the pivot table now we're going to use this to um
  1544. table now we're going to use this to um to group and count by position right
  1545. to group and count by position right and so you can see here on the left is
  1546. and so you can see here on the left is sort of a blank canvas on which we're
  1547. sort of a blank canvas on which we're going to group and count our data and
  1548. going to group and count our data and then on the right are some like little
  1549. then on the right are some like little knobs and like goofy things which is how
  1550. knobs and like goofy things which is how you manipulate the pivot table right you
  1551. you manipulate the pivot table right you can see it as rows columns and values
  1552. can see it as rows columns and values are the key ones so the row is really
  1553. are the key ones so the row is really the thing you want to group by right so
  1554. the thing you want to group by right so like which of the columns do you want to
  1555. like which of the columns do you want to roll up into like subtotals and for us
  1556. roll up into like subtotals and for us that's position and you can see here on
  1557. that's position and you can see here on the right hand side all of our columns
  1558. the right hand side all of our columns are like available so you just click on
  1559. are like available so you just click on position and you drag it into the rows
  1560. position and you drag it into the rows area of the pivot table and boom you see
  1561. area of the pivot table and boom you see automatically there on the left it's
  1562. automatically there on the left it's created a list of just the unique values
  1563. created a list of just the unique values by position
  1564. by position right center forward and guard and now i
  1565. right center forward and guard and now i want to calculate values for each of
  1566. want to calculate values for each of those so i want to get the total amount
  1567. those so i want to get the total amount of salary we're now spending so i'm
  1568. of salary we're now spending so i'm going to grab our new salary column
  1569. going to grab our new salary column i'm going to drag that to the values
  1570. i'm going to drag that to the values area i'm going to drop it and you can
  1571. area i'm going to drop it and you can see that here it says summarize by so
  1572. see that here it says summarize by so that's the that's the function or
  1573. that's the that's the function or formula to run on that column and if i
  1574. formula to run on that column and if i click on it you can see the sum and
  1575. click on it you can see the sum and count and average and max and min and
  1576. count and average and max and min and median and standard deviation and you
  1577. median and standard deviation and you know they can you could pick but some is
  1578. know they can you could pick but some is what we wanted to start with so i'm good
  1579. what we wanted to start with so i'm good with that we can see that we're spending
  1580. with that we can see that we're spending 50 million on center 70 on forward 55 of
  1581. 50 million on center 70 on forward 55 of card so if i want to get the average
  1582. card so if i want to get the average it's as easy as taking new salary again
  1583. it's as easy as taking new salary again dragging and dropping it into values and
  1584. dragging and dropping it into values and you can see wow another column magically
  1585. you can see wow another column magically appears
  1586. appears i can select average
  1587. i can select average boom that's now got the averages
  1588. boom that's now got the averages automatically calculated right and i can
  1589. automatically calculated right and i can do this for counts too oftentimes the
  1590. do this for counts too oftentimes the most common way to use this is just to
  1591. most common way to use this is just to count how many people
  1592. count how many people are in each category that you're
  1593. are in each category that you're interested in right and so if i just
  1594. interested in right and so if i just take
  1595. take current salary
  1596. current salary drop it a third or you really can do it
  1597. drop it a third or you really can do it for any field but let's do new salary a
  1598. for any field but let's do new salary a third time
  1599. third time if i select count a count a is just like
  1600. if i select count a count a is just like the weird way of saying count the number
  1601. the weird way of saying count the number of records it's just like a weird
  1602. of records it's just like a weird tradition
  1603. tradition and if you do that it quickly tells us
  1604. and if you do that it quickly tells us there's two forwards two guards and one
  1605. there's two forwards two guards and one center right and you guys can see how
  1606. center right and you guys can see how this basic pivot table we just did if
  1607. this basic pivot table we just did if you were to put in like the city of
  1608. you were to put in like the city of chicago's government salaries database
  1609. chicago's government salaries database right you could compare how much the
  1610. right you could compare how much the average firefighter makes to the average
  1611. average firefighter makes to the average police officer you could look at
  1612. police officer you could look at different ranks within
  1613. different ranks within an agency and see how much the
  1614. an agency and see how much the executives get paid versus the lower
  1615. executives get paid versus the lower level workers and this these basic
  1616. level workers and this these basic tricks can be done to do all kinds of
  1617. tricks can be done to do all kinds of stories and analysis and the pivot is
  1618. stories and analysis and the pivot is often your way to get there
  1619. often your way to get there and there's a lot more to it and a lot
  1620. and there's a lot more to it and a lot more can do but that's really the basics
  1621. more can do but that's really the basics so we've grouped here by position just
  1622. so we've grouped here by position just to kind of show how easy it is i'm going
  1623. to kind of show how easy it is i'm going to click the little x there to get rid
  1624. to click the little x there to get rid of position and then i'm just going to
  1625. of position and then i'm just going to drop the sex column into rows now right
  1626. drop the sex column into rows now right and you can see that i've left the
  1627. and you can see that i've left the calculated values there and by switching
  1628. calculated values there and by switching what was in the rows it's now grouping
  1629. what was in the rows it's now grouping by sex as opposed to grouping by
  1630. by sex as opposed to grouping by position and the same analysis is just
  1631. position and the same analysis is just immediately run for all of them pretty
  1632. immediately run for all of them pretty nice right
  1633. nice right and a lot of times for data journals and
  1634. and a lot of times for data journals and stories the numbers that appear in your
  1635. stories the numbers that appear in your pivot table just go right into your
  1636. pivot table just go right into your story you know after you vet the data
  1637. story you know after you vet the data and do all that other stuff of course
  1638. and do all that other stuff of course but this is often where you come up with
  1639. but this is often where you come up with what is going to be reported
  1640. we'll do a more sophisticated pivot table in a minute but that's really the
  1641. table in a minute but that's really the basics of it you just learned
  1642. basics of it you just learned any questions about pivot tables
  1643. no okay well those are the basic skills that i wanted to cover that's kind of
  1644. that i wanted to cover that's kind of part one of this class and so now i
  1645. part one of this class and so now i think we're gonna advance to kind of
  1646. think we're gonna advance to kind of part two which is instead of creating
  1647. part two which is instead of creating sort of a funny data set on our own
  1648. sort of a funny data set on our own we're going to look at a serious data
  1649. we're going to look at a serious data set something that came from the real
  1650. set something that came from the real world and we're going to apply
  1651. world and we're going to apply some of those
  1652. some of those skills we just learned to ask and answer
  1653. skills we just learned to ask and answer questions of this real data set okay and
  1654. questions of this real data set okay and so the data set that we're going to use
  1655. so the data set that we're going to use comes from a group called the invisible
  1656. comes from a group called the invisible institute does anybody here know what
  1657. institute does anybody here know what that is
  1658. the invisible institute is a chicago-based sort of non-profit group
  1659. chicago-based sort of non-profit group that produces investigative journalism
  1660. that produces investigative journalism documentaries even an art exhibit that's
  1661. documentaries even an art exhibit that's currently on display at the ball is
  1662. currently on display at the ball is connected to the invisible institute and
  1663. connected to the invisible institute and it's sort of one
  1664. it's sort of one um permutation of kind of a shifting
  1665. um permutation of kind of a shifting group of chicago people who over the
  1666. group of chicago people who over the last couple decades have put a lot of
  1667. last couple decades have put a lot of effort
  1668. effort into
  1669. into exposing abuses by the chicago police
  1670. exposing abuses by the chicago police department through public records
  1671. department through public records requests lawsuits and other things um
  1672. requests lawsuits and other things um they are connected in a complicated way
  1673. they are connected in a complicated way to the john burge torture story if
  1674. to the john burge torture story if you're familiar with that if you're not
  1675. you're familiar with that if you're not google it um and one of their big data
  1676. google it um and one of their big data projects that they did a few years ago
  1677. projects that they did a few years ago and they were really one of the first
  1678. and they were really one of the first groups in the whole country to do this
  1679. groups in the whole country to do this is they were they um one through a
  1680. is they were they um one through a freedom of information act request not
  1681. freedom of information act request not an email not a memo not the sort of
  1682. an email not a memo not the sort of typical documents you might associate
  1683. typical documents you might associate with foia but a database you know
  1684. with foia but a database you know there's a literally a bureaucratic
  1685. there's a literally a bureaucratic process in the city of chicago where if
  1686. process in the city of chicago where if you believe a police officer has
  1687. you believe a police officer has mistreated you you can file a complaint
  1688. mistreated you you can file a complaint and that complaint depending on how it
  1689. and that complaint depending on how it plays off kicks off a kind of like
  1690. plays off kicks off a kind of like um i guess
  1691. um i guess adjudication process that results in um
  1692. adjudication process that results in um an investigation in some cases and then
  1693. an investigation in some cases and then a decision about whether your complaint
  1694. a decision about whether your complaint is upheld or not and then whether the
  1695. is upheld or not and then whether the officer should be disciplined
  1696. officer should be disciplined and that
  1697. and that government process generates data right
  1698. government process generates data right there is literally like a glorified
  1699. there is literally like a glorified spreadsheet or database system inside
  1700. spreadsheet or database system inside the city of chicago that's tracking all
  1701. the city of chicago that's tracking all those things and those databases are
  1702. those things and those databases are things that you can file public or
  1703. things that you can file public or records requests for and literally get
  1704. records requests for and literally get the raw data to analyze on your own and
  1705. the raw data to analyze on your own and so one whole sort of strain of data
  1706. so one whole sort of strain of data journalism one type of data story is not
  1707. journalism one type of data story is not building your own database but prying
  1708. building your own database but prying loose a database from the government and
  1709. loose a database from the government and then kind of figuring out how it works
  1710. then kind of figuring out how it works and what's really in it and then
  1711. and what's really in it and then analyzing it to do stories or in the
  1712. analyzing it to do stories or in the case of you know now on the web just
  1713. case of you know now on the web just republish the whole darn thing and
  1714. republish the whole darn thing and that's what the invisible institute did
  1715. that's what the invisible institute did i think in 2016 or so i can't remember
  1716. i think in 2016 or so i can't remember exactly that was a pretty controversial
  1717. exactly that was a pretty controversial time is they built the website where
  1718. time is they built the website where they just published um the complaints
  1719. they just published um the complaints against police officers you know one
  1720. against police officers you know one argument for this
  1721. argument for this is transparency just on its own but also
  1722. is transparency just on its own but also that um
  1723. that um that uh when people are defending
  1724. that uh when people are defending themselves in court after a police
  1725. themselves in court after a police officer is involved in testimony or
  1726. officer is involved in testimony or other parts of the investigation they
  1727. other parts of the investigation they you know the one argument is that they
  1728. you know the one argument is that they have a right to be able to you know find
  1729. have a right to be able to you know find out if the officer testifying against
  1730. out if the officer testifying against them has a track record that might not
  1731. them has a track record that might not be great and so um i suspect the defense
  1732. be great and so um i suspect the defense attorneys are probably the top user of
  1733. attorneys are probably the top user of this website you can see here that it
  1734. this website you can see here that it has an analysis of the whole system
  1735. has an analysis of the whole system overall how many people in aggregate
  1736. overall how many people in aggregate have had allegations against them
  1737. have had allegations against them discipline and then they even have a
  1738. discipline and then they even have a page for individual officers and this is
  1739. page for individual officers and this is part of what makes it controversial i
  1740. part of what makes it controversial i suppose you can see here they have a
  1741. suppose you can see here they have a readout on individual officers and so
  1742. readout on individual officers and so this database
  1743. this database fueled a whole lot of stories in chicago
  1744. fueled a whole lot of stories in chicago when it first came out
  1745. when it first came out many people in local news copied it in
  1746. many people in local news copied it in subsequent years and did similar stories
  1747. subsequent years and did similar stories there was a whole reform to the chicago
  1748. there was a whole reform to the chicago system for managing this to introduce
  1749. system for managing this to introduce the civilian review board
  1750. the civilian review board and a lot of it really stems back to the
  1751. and a lot of it really stems back to the work this group did and one thing they
  1752. work this group did and one thing they did that i think is quite admirable is
  1753. did that i think is quite admirable is they just publish the whole database in
  1754. they just publish the whole database in raw spreadsheet format right so they
  1755. raw spreadsheet format right so they made a pretty website they wrote stories
  1756. made a pretty website they wrote stories where they analyze the thing but then
  1757. where they analyze the thing but then they also just put up the data for other
  1758. they also just put up the data for other people to use which means other
  1759. people to use which means other journalists can analyze it academics
  1760. journalists can analyze it academics criminologists get their hands on it and
  1761. criminologists get their hands on it and maybe people who wanted to vet their
  1762. maybe people who wanted to vet their work right who might you know ultimately
  1763. work right who might you know ultimately try to poke a hole in it and i think
  1764. try to poke a hole in it and i think that that is just a really great kind of
  1765. that that is just a really great kind of you know scientific practice that is
  1766. you know scientific practice that is awesome and so kudos to them for that
  1767. awesome and so kudos to them for that and here on their website is this kind
  1768. and here on their website is this kind of odd page download the data
  1769. of odd page download the data and this it links off to this even
  1770. and this it links off to this even stranger nerdy website called github
  1771. stranger nerdy website called github where you can get all their computer
  1772. where you can get all their computer code but
  1773. code but if you were to click into this little
  1774. if you were to click into this little link
  1775. link you would find like literally a dropbox
  1776. you would find like literally a dropbox that has all the data they got and kind
  1777. that has all the data they got and kind of cleaned up from the government
  1778. of cleaned up from the government including the actual four-year request
  1779. including the actual four-year request themselves which is always kind of
  1780. themselves which is always kind of interesting to read you can see here
  1781. interesting to read you can see here that they literally wrote a letter dear
  1782. that they literally wrote a letter dear foia officer
  1783. foia officer under the freedom of information act
  1784. under the freedom of information act request i i request your database and
  1785. request i i request your database and you can see they even included the names
  1786. you can see they even included the names of the fields in the database they
  1787. of the fields in the database they wanted right i was trying to get rajiv
  1788. wanted right i was trying to get rajiv who was involved in this to join us but
  1789. who was involved in this to join us but he couldn't tonight but i suspect that
  1790. he couldn't tonight but i suspect that they did some reporting ahead of time to
  1791. they did some reporting ahead of time to know what was in there so that they
  1792. know what was in there so that they could make sure to request it so that it
  1793. could make sure to request it so that it didn't get left out in the response
  1794. didn't get left out in the response because one issue that could happen is
  1795. because one issue that could happen is is you might request the database but do
  1796. is you might request the database but do they really give you the whole thing
  1797. they really give you the whole thing a lot of time they don't want to right
  1798. a lot of time they don't want to right and so being specific in this case was
  1799. and so being specific in this case was probably part of their strategy to make
  1800. probably part of their strategy to make sure they got it at the end of the day
  1801. sure they got it at the end of the day um
  1802. um and writing that type of foia is a whole
  1803. and writing that type of foia is a whole sort of art that we can talk about if
  1804. sort of art that we can talk about if you want to
  1805. you want to um and but these files are really big
  1806. um and but these files are really big because they have hundreds of thousands
  1807. because they have hundreds of thousands of records and i don't want to like
  1808. of records and i don't want to like strain everybody's computer here in
  1809. strain everybody's computer here in class
  1810. class and they might require some heavier duty
  1811. and they might require some heavier duty programming tools to do like a super
  1812. programming tools to do like a super duper analysis so what i've done is i
  1813. duper analysis so what i've done is i talked with rajiv who's behind it i
  1814. talked with rajiv who's behind it i created like a little
  1815. created like a little extract
  1816. extract i went into the data i merged a couple
  1817. i went into the data i merged a couple tables into a single table and then i
  1818. tables into a single table and then i filtered it down to just one year worth
  1819. filtered it down to just one year worth of complaints this is from the year 2014
  1820. of complaints this is from the year 2014 and so what we have here is sort of a
  1821. and so what we have here is sort of a trimmed down simplified version of the
  1822. trimmed down simplified version of the data set that has i've removed a bunch
  1823. data set that has i've removed a bunch of columns and i've removed a bunch of
  1824. of columns and i've removed a bunch of rows to kind of get to something that is
  1825. rows to kind of get to something that is derived from the original original
  1826. derived from the original original analysis but is all real data
  1827. analysis but is all real data this process of like cleaning the data
  1828. this process of like cleaning the data of merging different tables filtering it
  1829. of merging different tables filtering it down getting rid of stuff you don't want
  1830. down getting rid of stuff you don't want is often like 80 90 of this type of data
  1831. is often like 80 90 of this type of data story because the stuff you get from the
  1832. story because the stuff you get from the government is often like cryptic or
  1833. government is often like cryptic or incomplete or like gnarly and like
  1834. incomplete or like gnarly and like understanding what's there just so you
  1835. understanding what's there just so you can trim it down to something nice and
  1836. can trim it down to something nice and clean that you can analyze is a whole
  1837. clean that you can analyze is a whole reporting effort and often requires
  1838. reporting effort and often requires another set of programming skills we're
  1839. another set of programming skills we're not going to cover that here in class
  1840. not going to cover that here in class okay sorry so what we're going to do is
  1841. okay sorry so what we're going to do is i'm going to share a link to the
  1842. i'm going to share a link to the spreadsheet
  1843. spreadsheet in the chat and i want everybody to open
  1844. in the chat and i want everybody to open it in google sheets and i'm going to
  1845. it in google sheets and i'm going to show you how to make your own copy of it
  1846. show you how to make your own copy of it really quickly okay
  1847. really quickly okay so in google sheets if you haven't done
  1848. so in google sheets if you haven't done it this is part of what makes it great
  1849. it this is part of what makes it great is you can click the share button in the
  1850. is you can click the share button in the upper right you can add people to
  1851. upper right you can add people to privately share it which is often useful
  1852. privately share it which is often useful but you also here can just get a link
  1853. but you also here can just get a link that you can give to the whole world or
  1854. that you can give to the whole world or your co-workers and this is how a lot of
  1855. your co-workers and this is how a lot of data journalists share data within their
  1856. data journalists share data within their newsroom it's how we make charts at the
  1857. newsroom it's how we make charts at the la times is people make a google sheet
  1858. la times is people make a google sheet and we plug it into the chart tool right
  1859. and we plug it into the chart tool right and the way you give you sort of or plug
  1860. and the way you give you sort of or plug it in somewhere give it to someone else
  1861. it in somewhere give it to someone else is you click that share button and then
  1862. is you click that share button and then here there's this get link section
  1863. here there's this get link section i'm going to click you know it's
  1864. i'm going to click you know it's restricted right now it starts as
  1865. restricted right now it starts as private i'm going to open it up so i
  1866. private i'm going to open it up so i click there and i'm going to say anybody
  1867. click there and i'm going to say anybody who has the link
  1868. who has the link can actually not just view but edit the
  1869. can actually not just view but edit the spreadsheet that's a little dangerous
  1870. spreadsheet that's a little dangerous you might not want to do that for stuff
  1871. you might not want to do that for stuff for your story but for this class this
  1872. for your story but for this class this is now like an open door to come into
  1873. is now like an open door to come into this spreadsheet and do whatever you
  1874. this spreadsheet and do whatever you want with it right
  1875. want with it right so i'm going to drop that into the chat
  1876. so i'm going to drop that into the chat here in class let me just find the chat
  1877. here in class let me just find the chat real quick
  1878. okay boom you guys see that link i want you
  1879. boom you guys see that link i want you to jump in
  1880. to jump in we should start seeing like people's
  1881. we should start seeing like people's little like google icons coming there
  1882. little like google icons coming there they are
  1883. they are we got a buffalo and a kiwi a mink a bat
  1884. we got a buffalo and a kiwi a mink a bat an elephant
  1885. an elephant i'm just gonna give folks a second to
  1886. i'm just gonna give folks a second to jump in
  1887. jump in [Music]
  1888. okay now i want you to duplicate it and make your own copy so if you go to the
  1889. make your own copy so if you go to the file menu you see this make a copy
  1890. file menu you see this make a copy option fourth one down once you just
  1891. option fourth one down once you just click that and that should like open up
  1892. click that and that should like open up a new tab on your computer
  1893. a new tab on your computer where you have to give it a name so i'm
  1894. where you have to give it a name so i'm gonna call mine copy
  1895. gonna call mine copy i'm gonna say make a copy and you can
  1896. i'm gonna say make a copy and you can see it's just gonna open a new tab
  1897. see it's just gonna open a new tab and make another copy of the data
  1898. and make another copy of the data excuse me
  1899. excuse me and now i'm going to go back to my
  1900. and now i'm going to go back to my original one because i don't need the
  1901. original one because i don't need the copy but i want everybody to do that so
  1902. copy but i want everybody to do that so just do file
  1903. just do file make a copy
  1904. okay so now we have our data set
  1905. so now we have our data set there's a couple like good practices
  1906. there's a couple like good practices anytime you get like a data set from a
  1907. anytime you get like a data set from a government or a source to help you just
  1908. government or a source to help you just like gradually begin to understand it
  1909. like gradually begin to understand it right and one of those things is what i
  1910. right and one of those things is what i talked about earlier which is like
  1911. talked about earlier which is like figuring out
  1912. figuring out what is a row right like in the
  1913. what is a row right like in the philosophical sense what is defined by
  1914. philosophical sense what is defined by each row in this data set right and if i
  1915. each row in this data set right and if i start looking at this i see okay i've
  1916. start looking at this i see okay i've got my columns there's a log number
  1917. got my columns there's a log number log no
  1918. log no i've seen a lot of data sets so i'm
  1919. i've seen a lot of data sets so i'm guessing that's a log number right
  1920. guessing that's a log number right there's a date a complaint date there's
  1921. there's a date a complaint date there's the officer's first name last name
  1922. the officer's first name last name their employee number maybe that's their
  1923. their employee number maybe that's their badge number i'm not sure i might have
  1924. badge number i'm not sure i might have to ask somebody to find that out
  1925. to ask somebody to find that out we've got the allegation against them
  1926. we've got the allegation against them which has some descriptions
  1927. which has some descriptions and then we've got the finding code
  1928. and then we've got the finding code which like finding is kind of weird
  1929. which like finding is kind of weird maybe that's the ultimate decision or
  1930. maybe that's the ultimate decision or disposition
  1931. disposition of the case those are all things i'm
  1932. of the case those are all things i'm like guessing right from just like what
  1933. like guessing right from just like what is this i'm smelling the data and like i
  1934. is this i'm smelling the data and like i think that's right right and you can
  1935. think that's right right and you can precede your like initial analysis of a
  1936. precede your like initial analysis of a data set just kind of
  1937. data set just kind of guessing what you know it is but
  1938. guessing what you know it is but ultimately you have to test those
  1939. ultimately you have to test those assumptions against the data itself
  1940. assumptions against the data itself and also just by maybe just asking the
  1941. and also just by maybe just asking the source you know like what is this column
  1942. source you know like what is this column is this is this defining or is that
  1943. is this is this defining or is that that's the result or is it not you know
  1944. that's the result or is it not you know and then one thing is i just like look
  1945. and then one thing is i just like look at the first few rows and i kind of
  1946. at the first few rows and i kind of sniff the data right i notice oh hey
  1947. sniff the data right i notice oh hey those log numbers repeat like we have
  1948. those log numbers repeat like we have look at this this log number is the same
  1949. look at this this log number is the same for like the first 23 records
  1950. for like the first 23 records right
  1951. right and hey roberto lopianco is in here
  1952. and hey roberto lopianco is in here twice with the same number that's that's
  1953. twice with the same number that's that's probably the same guy right
  1954. probably the same guy right so
  1955. so each log so each row isn't a single
  1956. each log so each row isn't a single complaint i don't think because it has
  1957. complaint i don't think because it has multiple people or maybe a complaint can
  1958. multiple people or maybe a complaint can have more than one officer
  1959. have more than one officer yeah that's it probably right so
  1960. yeah that's it probably right so the relationship between records or like
  1961. the relationship between records or like the data is often you have to think
  1962. the data is often you have to think about those relationships are often like
  1963. about those relationships are often like one to one or one to many and so we have
  1964. one to one or one to many and so we have here like one log number that has many
  1965. here like one log number that has many officers right and if you think about it
  1966. officers right and if you think about it that probably means when you file a
  1967. that probably means when you file a complaint you can accuse multiple people
  1968. complaint you can accuse multiple people which passes the common sense test and
  1969. which passes the common sense test and kind of smells okay but you might want
  1970. kind of smells okay but you might want to make sure that you might want to look
  1971. to make sure that you might want to look at the form
  1972. at the form right that's used to file a complaint
  1973. right that's used to file a complaint oftentimes if you don't understand your
  1974. oftentimes if you don't understand your data from a government database getting
  1975. data from a government database getting the form that it comes from can answer
  1976. the form that it comes from can answer those questions faster than asking
  1977. those questions faster than asking anybody right because if you just look
  1978. anybody right because if you just look at the blanks you fill in on any
  1979. at the blanks you fill in on any government form that almost always
  1980. government form that almost always corresponds to fields and tables in a
  1981. corresponds to fields and tables in a database right and so if i was to report
  1982. database right and so if i was to report this out that's what i would look for
  1983. this out that's what i would look for and i always look for that sometimes
  1984. and i always look for that sometimes i'll file a public records request for
  1985. i'll file a public records request for the form right because i can't get
  1986. the form right because i can't get anybody to answer my question and i know
  1987. anybody to answer my question and i know if i get the form i got something you
  1988. if i get the form i got something you know that i can go off that i can have
  1989. know that i can go off that i can have some
  1990. some confidence in right
  1991. confidence in right it'll i'll often get request the forms
  1992. it'll i'll often get request the forms before the database so that i know what
  1993. before the database so that i know what fields to ask for right and that's one
  1994. fields to ask for right and that's one way you can kind of know what they got
  1995. way you can kind of know what they got you know you can reverse engineer the
  1996. you know you can reverse engineer the database
  1997. database and then we see here robert lobianco is
  1998. and then we see here robert lobianco is in here twice what's that about
  1999. in here twice what's that about and if i look at this allegation
  2000. and if i look at this allegation category we see there's different
  2001. category we see there's different allegations so that suggests to me the
  2002. allegations so that suggests to me the complaint has a one-to-many relationship
  2003. complaint has a one-to-many relationship to the officers and then the officers
  2004. to the officers and then the officers have a one-to-many relationship to the
  2005. have a one-to-many relationship to the allegations which also makes sense right
  2006. allegations which also makes sense right so it's like i could
  2007. so it's like i could i could make an allegation against
  2008. i could make an allegation against multiple officers and i could say that
  2009. multiple officers and i could say that each one did multiple things wrong right
  2010. each one did multiple things wrong right and so you can see here that just within
  2011. and so you can see here that just within this one complaint there actually is
  2012. this one complaint there actually is like five or six officers and each one
  2013. like five or six officers and each one of them has multiple allegations against
  2014. of them has multiple allegations against them
  2015. them right
  2016. right okay so we're starting to get a sense of
  2017. okay so we're starting to get a sense of like i would say and this is the
  2018. like i would say and this is the question i was almost what's a row
  2019. question i was almost what's a row what's a row how would i define it i
  2020. what's a row how would i define it i would say the row is an allegation
  2021. would say the row is an allegation that's what i think i would call it
  2022. that's what i think i would call it right so each row is an allegation
  2023. right so each row is an allegation against an officer linked to a complaint
  2024. against an officer linked to a complaint i think
  2025. i think and like that definition of my head
  2026. and like that definition of my head often gets revised as i like better
  2027. often gets revised as i like better understand the data like i was working
  2028. understand the data like i was working with claire and her classmates on a
  2029. with claire and her classmates on a story last year and we thought we was
  2030. story last year and we thought we was about trees in the city of chicago and
  2031. about trees in the city of chicago and there was this one column that was just
  2032. there was this one column that was just like a number and it was just and we
  2033. like a number and it was just and we didn't know what it was and we thought
  2034. didn't know what it was and we thought each row is a tree each row is one tree
  2035. each row is a tree each row is one tree like one right but then we asked them
  2036. like one right but then we asked them hey what's this weird number column they
  2037. hey what's this weird number column they were like oh that's the number of trees
  2038. were like oh that's the number of trees we planted
  2039. we planted and so each row was in order to plant
  2040. and so each row was in order to plant trees
  2041. trees and that number told you how many trees
  2042. and that number told you how many trees there were and so the difference between
  2043. there were and so the difference between just counting each record is one and
  2044. just counting each record is one and summing that numerical thing is the
  2045. summing that numerical thing is the difference between your story being
  2046. difference between your story being right your story being wrong right and
  2047. right your story being wrong right and so
  2048. so doing this kind of data smelling
  2049. doing this kind of data smelling figuring it out what each row is
  2050. figuring it out what each row is really pushing yourself to kind of like
  2051. really pushing yourself to kind of like guess what each column is
  2052. guess what each column is and you can often just say these 10
  2053. and you can often just say these 10 columns i don't even think i need to
  2054. columns i don't even think i need to care about right but
  2055. care about right but you got to do that process
  2056. you got to do that process and then so we kind of got we came up
  2057. and then so we kind of got we came up with our working theory of what a row is
  2058. with our working theory of what a row is right and then another good thing is
  2059. right and then another good thing is just to look at like how much data you
  2060. just to look at like how much data you have and whether it's all filled in or
  2061. have and whether it's all filled in or not right because you have this data set
  2062. not right because you have this data set we don't know how many rows there are we
  2063. we don't know how many rows there are we don't know if there's empty values all
  2064. don't know if there's empty values all over the database it could be incomplete
  2065. over the database it could be incomplete and a really great way to do that is
  2066. and a really great way to do that is just to scroll around the four corners
  2067. just to scroll around the four corners of the database right just to like
  2068. of the database right just to like literally scroll to the bottom and see
  2069. literally scroll to the bottom and see what you've got and like see if stuff is
  2070. what you've got and like see if stuff is filled in so i'm just gonna page down
  2071. filled in so i'm just gonna page down really quick
  2072. really quick see i don't see a lot of empty cells
  2073. see i don't see a lot of empty cells there are some i've noticed look there's
  2074. there are some i've noticed look there's some empty cells that's interesting so
  2075. some empty cells that's interesting so sometimes there isn't a finding well
  2076. sometimes there isn't a finding well that's interesting why wouldn't there be
  2077. that's interesting why wouldn't there be a finding sometimes is it maybe still
  2078. a finding sometimes is it maybe still under investigation
  2079. under investigation maybe right is it um a flaw that they
  2080. maybe right is it um a flaw that they failed to put it in i gotta tuck that
  2081. failed to put it in i gotta tuck that that's that's a data smell right i smell
  2082. that's that's a data smell right i smell that and i'm like okay i'm gonna make a
  2083. that and i'm like okay i'm gonna make a little note for myself in my notepad
  2084. little note for myself in my notepad that might or might not be something
  2085. that might or might not be something that matters down the road but i wanna
  2086. that matters down the road but i wanna just like accrue my little list of like
  2087. just like accrue my little list of like you know status smells so i know
  2088. you know status smells so i know what's going on
  2089. what's going on and you got to get to know your data
  2090. and you got to get to know your data like this you don't want to run too fast
  2091. like this you don't want to run too fast and so you get to the bottom and we can
  2092. and so you get to the bottom and we can see that there's 4
  2093. see that there's 4 338 rows subtract our one header row we
  2094. 338 rows subtract our one header row we know there's 4
  2095. know there's 4 337 complaints right
  2096. 337 complaints right in this like set of data that ben gave
  2097. in this like set of data that ben gave there's other integrity checks you could
  2098. there's other integrity checks you could do like for instance i think this is
  2099. do like for instance i think this is everything in 2014 you might want to do
  2100. everything in 2014 you might want to do totals by month just to make sure we're
  2101. totals by month just to make sure we're not missing a couple months right you
  2102. not missing a couple months right you could do these little integrity checks
  2103. could do these little integrity checks on the data that help you like see kind
  2104. on the data that help you like see kind of what's up with it right
  2105. okay so we kind of smelled it we think it's for 2014 this is all the
  2106. it's for 2014 this is all the allegations made against officers in the
  2107. allegations made against officers in the city
  2108. city now i want to use some of the tricks
  2109. now i want to use some of the tricks that we've covered already to ask and
  2110. that we've covered already to ask and answer some questions right
  2111. answer some questions right so one thing i i want to look at is like
  2112. so one thing i i want to look at is like a story that's been done before so here
  2113. a story that's been done before so here was a story that ran in the chicago
  2114. was a story that ran in the chicago reporter where they've done lots and
  2115. reporter where they've done lots and lots of great data journalism over the
  2116. lots of great data journalism over the years they're having some management
  2117. years they're having some management problems right now like many journalism
  2118. problems right now like many journalism outlets are but the chicago reporter has
  2119. outlets are but the chicago reporter has a tremendous track record of great data
  2120. a tremendous track record of great data journalism especially around issues of
  2121. journalism especially around issues of race and policing and one story they did
  2122. race and policing and one story they did it looks like six years ago is
  2123. it looks like six years ago is they looked at a subsequent release they
  2124. they looked at a subsequent release they looked at this very same data set
  2125. looked at this very same data set right
  2126. right and they
  2127. and they looked at they analyzed something called
  2128. looked at they analyzed something called the affidavit requirement
  2129. the affidavit requirement and what they found
  2130. and what they found looking at like literally the same
  2131. looking at like literally the same spreadsheet that we have that's on the
  2132. spreadsheet that we have that's on the website is they found let me see if we
  2133. website is they found let me see if we can find the finding here
  2134. can find the finding here they found there were 17 000 complaints
  2135. they found there were 17 000 complaints in the three years they looked at which
  2136. in the three years they looked at which really would just be
  2137. really would just be scrolling to the bottom of the data set
  2138. scrolling to the bottom of the data set right in the same way we just did they
  2139. right in the same way we just did they found out investigators didn't open
  2140. found out investigators didn't open cases in 58
  2141. cases in 58 because they were marked no affidavit
  2142. because they were marked no affidavit well that looks familiar i see that
  2143. well that looks familiar i see that right here no affidavit right that's in
  2144. right here no affidavit right that's in my finding code
  2145. my finding code and if you read more into it it turns
  2146. and if you read more into it it turns out that
  2147. out that under the the rules of how police
  2148. under the the rules of how police complaints work which were hammered out
  2149. complaints work which were hammered out in a past labor contract negotiation
  2150. in a past labor contract negotiation with the police union um
  2151. with the police union um i'm not a lawyer so i might not get this
  2152. i'm not a lawyer so i might not get this exactly right but my understanding is is
  2153. exactly right but my understanding is is you can't just file a complaint you also
  2154. you can't just file a complaint you also have to file a legal affidavit where you
  2155. have to file a legal affidavit where you kind of swear that what you're saying is
  2156. kind of swear that what you're saying is what really happened which is a kind of
  2157. what really happened which is a kind of hoop a procedural hoop that each
  2158. hoop a procedural hoop that each complainant has to jump through for
  2159. complainant has to jump through for their complaint to actually be
  2160. their complaint to actually be investigated by the police and it turns
  2161. investigated by the police and it turns out that it's really really common
  2162. out that it's really really common for that to be done according to the
  2163. for that to be done according to the analysis of this group and then this
  2164. analysis of this group and then this piece kind of has a little bit of an
  2165. piece kind of has a little bit of an opinion angle to it where it's saying if
  2166. opinion angle to it where it's saying if they were to get rid of that requirement
  2167. they were to get rid of that requirement it would lead to a lot more
  2168. it would lead to a lot more investigations and a lot more discipline
  2169. investigations and a lot more discipline against police is kind of potential
  2170. against police is kind of potential right is what the story says
  2171. right is what the story says so
  2172. so if i wanted to do a similar analysis for
  2173. if i wanted to do a similar analysis for this data set i want to figure out
  2174. this data set i want to figure out of the what was the number again of the
  2175. ah here's a fun trick if you hit control
  2176. here's a fun trick if you hit control down control and hit up it jumps to the
  2177. down control and hit up it jumps to the last record of our 4
  2178. last record of our 4 37 complaints what percentage were no
  2179. 37 complaints what percentage were no affidavit right like that's what i want
  2180. affidavit right like that's what i want to figure out and let's say you filed a
  2181. to figure out and let's say you filed a foia and got last year's complaints and
  2182. foia and got last year's complaints and you were the first reporter to get it
  2183. you were the first reporter to get it you could run this analysis to like
  2184. you could run this analysis to like update this old story right so if i
  2185. update this old story right so if i wanted to do that i want to know what's
  2186. wanted to do that i want to know what's the number
  2187. the number of
  2188. of allegations
  2189. allegations that that were
  2190. that that were um dropped because they were no
  2191. um dropped because they were no affidavit
  2192. affidavit no affidavit was filed and i want to
  2193. no affidavit was filed and i want to know that as a percentage of the total
  2194. know that as a percentage of the total what trick that we've used so far would
  2195. what trick that we've used so far would help us get there who can think of
  2196. help us get there who can think of something
  2197. it's the pivot table guys right because we've got our categorical data that we
  2198. we've got our categorical data that we want to group and count we want to know
  2199. want to group and count we want to know for each of the findings how many there
  2200. for each of the findings how many there are right
  2201. are right and so i'm going to click in the upper
  2202. and so i'm going to click in the upper left on the selector to select my whole
  2203. left on the selector to select my whole data set
  2204. data set i'm going to do insert
  2205. i'm going to do insert and then my buddy pivot table
  2206. and then my buddy pivot table oh yeah
  2207. selected new sheet
  2208. sheet okay so now i want to group by the
  2209. okay so now i want to group by the finding code column who can tell me how
  2210. finding code column who can tell me how to do that
  2211. finding code and then under row yes we got our columns here i grab drag that
  2212. got our columns here i grab drag that and i drop it into row and so that's one
  2213. and i drop it into row and so that's one per row and so we can now see
  2214. per row and so we can now see that these are all the potential values
  2215. that these are all the potential values there's some empty rows as we saw when
  2216. there's some empty rows as we saw when we were scrolling
  2217. we were scrolling there's some that are under
  2218. there's some that are under additional investigation required
  2219. additional investigation required some exonerated no affidavit there it is
  2220. some exonerated no affidavit there it is not sustained sustained which means they
  2221. not sustained sustained which means they said it was legit the complaint right
  2222. said it was legit the complaint right and then unfounded right and you might
  2223. and then unfounded right and you might need to report out what are all these
  2224. need to report out what are all these values right you know and that's part of
  2225. values right you know and that's part of the process of getting to know your data
  2226. the process of getting to know your data and so i just want to count
  2227. and so i just want to count how many allegations
  2228. how many allegations there are
  2229. there are for each one of these groups so how do i
  2230. for each one of these groups so how do i do that
  2231. do that pull finding code to values and do by
  2232. pull finding code to values and do by count a
  2233. count a yeah that would work so if i take
  2234. yeah that would work so if i take finding code the same column drop it
  2235. finding code the same column drop it there and leave it as count a
  2236. there and leave it as count a we can see that that instantly fills in
  2237. we can see that that instantly fills in right and we can see that there are two
  2238. right and we can see that there are two rows where additional investigation is
  2239. rows where additional investigation is required
  2240. required 297
  2241. 297 right
  2242. right where they're exonerated and we see
  2243. where they're exonerated and we see there's
  2244. there's 1286 that are no affidavit out of our
  2245. 1286 that are no affidavit out of our total of four two two three now wait
  2246. total of four two two three now wait hold on a second four two two three is
  2247. hold on a second four two two three is our total and this is part of the data
  2248. our total and this is part of the data smells it's like didn't we have more
  2249. smells it's like didn't we have more rows than that when i was looking
  2250. rows than that when i was looking right well if i go here and hit control
  2251. right well if i go here and hit control to the bottom
  2252. to the bottom there were
  2253. there were 4
  2254. 4 38 so shouldn't there be a little more
  2255. 38 so shouldn't there be a little more than that does anybody know why that why
  2256. than that does anybody know why that why we have that difference
  2257. we have that difference there's the ones that are potentially
  2258. there's the ones that are potentially still being investigated so the blank
  2259. still being investigated so the blank their left is empty and i think it's i
  2260. their left is empty and i think it's i think i didn't anticipate this but i
  2261. think i didn't anticipate this but i think that's because we use the finding
  2262. think that's because we use the finding code for our value
  2263. code for our value i said and those that's empty in some
  2264. i said and those that's empty in some cases and in the cases where it's empty
  2265. cases and in the cases where it's empty it's not counting it so i bet if we take
  2266. it's not counting it so i bet if we take log no which we know is filled in for
  2267. log no which we know is filled in for every row put that in and do count a
  2268. every row put that in and do count a look at that we get the total right and
  2269. look at that we get the total right and we see now that there's 114 that are
  2270. we see now that there's 114 that are blank you see that at the top there
  2271. blank you see that at the top there and so that was just like a little weird
  2272. and so that was just like a little weird wrinkle of how google sheets works right
  2273. wrinkle of how google sheets works right if we did our value in our account using
  2274. if we did our value in our account using a column that had null or empty cells
  2275. a column that had null or empty cells right it didn't count the null or empty
  2276. right it didn't count the null or empty ones right is that really the end of the
  2277. ones right is that really the end of the world no
  2278. world no in this case right and you might say
  2279. in this case right and you might say well for the purposes of this analysis i
  2280. well for the purposes of this analysis i might want to subtract those right like
  2281. might want to subtract those right like when i calculate the percentage if
  2282. when i calculate the percentage if they're still in progress cases so it
  2283. they're still in progress cases so it might actually be better to exclude them
  2284. might actually be better to exclude them methodologically but
  2285. methodologically but the lesson here i think to take home is
  2286. the lesson here i think to take home is that a little slip of the finger or the
  2287. that a little slip of the finger or the data the software not doing exactly what
  2288. data the software not doing exactly what you expected to do can sometimes change
  2289. you expected to do can sometimes change the analysis in a way that maybe you
  2290. the analysis in a way that maybe you don't even notice right and so that's
  2291. don't even notice right and so that's why when when you work with these pivot
  2292. why when when you work with these pivot tables
  2293. tables and other programming tools it's
  2294. and other programming tools it's important that every step to just stop
  2295. important that every step to just stop and look and be like did these numbers
  2296. and look and be like did these numbers add up
  2297. add up is this what i would expect it to do
  2298. is this what i would expect it to do because you'll often find it's not even
  2299. because you'll often find it's not even your fault like in this case it was just
  2300. your fault like in this case it was just kind of a weird thing and the way i was
  2301. kind of a weird thing and the way i was able to do that here just to recreate it
  2302. able to do that here just to recreate it is i just looked at the total
  2303. is i just looked at the total right
  2304. right and i said well wait wasn't it supposed
  2305. and i said well wait wasn't it supposed to be 4 300 or something and the reason
  2306. to be 4 300 or something and the reason i even had that thought is because i did
  2307. i even had that thought is because i did my four corners check right and i tucked
  2308. my four corners check right and i tucked away in my head
  2309. away in my head that's what i think it should be and
  2310. that's what i think it should be and that kind of back and forth of like
  2311. that kind of back and forth of like fumbling with it is really just part of
  2312. fumbling with it is really just part of the process and i've been doing this
  2313. the process and i've been doing this nearly 20 years and like i do this on
  2314. nearly 20 years and like i do this on everything i'm working on like on that
  2315. everything i'm working on like on that trees thing last year i probably screwed
  2316. trees thing last year i probably screwed up four or five things and we run around
  2317. up four or five things and we run around in circles like do i really know what
  2318. in circles like do i really know what this is and like that struggle is really
  2319. this is and like that struggle is really just part of the process right and the
  2320. just part of the process right and the fact that you're doing that is what's
  2321. fact that you're doing that is what's separating you from the competition
  2322. separating you from the competition right you're actually figuring it out
  2323. right you're actually figuring it out and you're going to get a story somebody
  2324. and you're going to get a story somebody else doesn't right and so when
  2325. else doesn't right and so when it was when i feel that frustration i'm
  2326. it was when i feel that frustration i'm as frustrated as anybody 20 years in i
  2327. as frustrated as anybody 20 years in i still feel the frustration every day but
  2328. still feel the frustration every day but i've got to the point where i realize i
  2329. i've got to the point where i realize i have the zen
  2330. have the zen that like oh that's good right it's
  2331. that like oh that's good right it's better that i had the frustration before
  2332. better that i had the frustration before the story came out than after the story
  2333. the story came out than after the story came out it's
  2334. came out it's right because i want to know it ahead of
  2335. right because i want to know it ahead of time of course so i'm right
  2336. time of course so i'm right and i don't have to write a correction
  2337. and i don't have to write a correction right but also it's usually a sign that
  2338. right but also it's usually a sign that you're kind of you're
  2339. you're kind of you're cutting some new territory right you're
  2340. cutting some new territory right you're there maybe beyond everybody else so
  2341. there maybe beyond everybody else so that's good
  2342. that's good okay so now we still haven't quite
  2343. okay so now we still haven't quite answered the question we know there's
  2344. answered the question we know there's 1286 no affidavits we know that's the
  2345. 1286 no affidavits we know that's the total oh wait let me go back to the log
  2346. total oh wait let me go back to the log number so that uh
  2347. number so that uh we have the full count
  2348. we have the full count okay so now how using only tricks we've
  2349. okay so now how using only tricks we've learned before but using them a little
  2350. learned before but using them a little bit differently how can we calculate the
  2351. bit differently how can we calculate the percentage of cases that are no
  2352. percentage of cases that are no affidavit anybody
  2353. i just want to put it right there in c5 i want to calculate it what do i do
  2354. so how do you start a formula who remembers
  2355. rules equals so you type equals you can really just start riff and math so
  2356. can really just start riff and math so the percentage is just this
  2357. the percentage is just this right
  2358. right divided into this right
  2359. divided into this right boom
  2360. boom 0.29 hit the percentage sign
  2361. 0.29 hit the percentage sign 29
  2362. 29 we're no affidavit
  2363. we're no affidavit right
  2364. right and so that's 29
  2365. and so that's 29 of allegations right now if you look at
  2366. of allegations right now if you look at the story i think they actually don't do
  2367. the story i think they actually don't do percent of allegations they do percent
  2368. percent of allegations they do percent of cases
  2369. of cases right and this is where the relationship
  2370. right and this is where the relationship in your data and how you think about it
  2371. in your data and how you think about it can really vary and a case isn't defined
  2372. can really vary and a case isn't defined by a row in our data set right a case is
  2373. by a row in our data set right a case is defined by the log node or so we think
  2374. defined by the log node or so we think we have to report that out and so really
  2375. we have to report that out and so really we would want to get into like the
  2376. we would want to get into like the unique log nodes or something maybe a
  2377. unique log nodes or something maybe a little more sophisticated and if so if
  2378. little more sophisticated and if so if we say rather than counting all records
  2379. we say rather than counting all records only count the unique log numbers right
  2380. only count the unique log numbers right after you've grouped and then how many
  2381. after you've grouped and then how many unique log numbers are in each group we
  2382. unique log numbers are in each group we can see that while we may have 4 000
  2383. can see that while we may have 4 000 allegations we only have 1200 complaints
  2384. allegations we only have 1200 complaints right and this is again thinking
  2385. right and this is again thinking conceptually about what your data is
  2386. conceptually about what your data is right and you'll notice that in in these
  2387. right and you'll notice that in in these stories as you read them how that's
  2388. stories as you read them how that's managed is really part of kind of
  2389. managed is really part of kind of getting it right or being artful and
  2390. getting it right or being artful and precise with your data and so i might
  2391. precise with your data and so i might say if my initial hunches hold up that
  2392. say if my initial hunches hold up that 34
  2393. 34 of complaints had no affidavit
  2394. of complaints had no affidavit but that
  2395. but that 29
  2396. 29 of allegations
  2397. of allegations right uh had no affidavit and this kind
  2398. right uh had no affidavit and this kind of subtlety of like how you think about
  2399. of subtlety of like how you think about or work it really is important to be
  2400. or work it really is important to be thinking about it every step of your
  2401. thinking about it every step of your analysis a lot of journalists who do
  2402. analysis a lot of journalists who do this keep what they call a data diary
  2403. this keep what they call a data diary which is where they literally write down
  2404. which is where they literally write down kind of the decisions they're making the
  2405. kind of the decisions they're making the questions that are coming up as they go
  2406. questions that are coming up as they go along and then they go back to that
  2407. along and then they go back to that again and again and oftentimes you do a
  2408. again and again and oftentimes you do a lot of sort of doodles and curly cues as
  2409. lot of sort of doodles and curly cues as you mess around with the data and then
  2410. you mess around with the data and then you figure out well this is the number i
  2411. you figure out well this is the number i want to report or i want to build my
  2412. want to report or i want to build my story around and then you return to that
  2413. story around and then you return to that diary to the process all the steps you
  2414. diary to the process all the steps you took and you try to make sure is
  2415. took and you try to make sure is everything i'm doing on each step like
  2416. everything i'm doing on each step like exactly what i need it to be you know
  2417. exactly what i need it to be you know what i mean for this to be right are
  2418. what i mean for this to be right are there any shaky assumptions i'm making
  2419. there any shaky assumptions i'm making is there anything about the structure of
  2420. is there anything about the structure of the data that i'm unsure about or it
  2421. the data that i'm unsure about or it doesn't feel right or seem right
  2422. doesn't feel right or seem right is the language i'm using to describing
  2423. is the language i'm using to describing it correct that's really the kind of
  2424. it correct that's really the kind of bulletproofing process of data
  2425. bulletproofing process of data journalism that is similar to
  2426. journalism that is similar to bulletproofing other stories but just
  2427. bulletproofing other stories but just has a more technical nature and for me
  2428. has a more technical nature and for me personally having done this more and
  2429. personally having done this more and more this is where computer programming
  2430. more this is where computer programming comes into it for me because what i like
  2431. comes into it for me because what i like to do as because i've gotten nerdier is
  2432. to do as because i've gotten nerdier is literally write computer code that
  2433. literally write computer code that executes each step of the data
  2434. executes each step of the data transformation so that i can really
  2435. transformation so that i can really carefully regulate what i do with the
  2436. carefully regulate what i do with the data right
  2437. data right and know each step and then i can go
  2438. and know each step and then i can go back and go over it and go over and go
  2439. back and go over it and go over and go over it until i feel confident that i
  2440. over it until i feel confident that i have a strong and that's harder to do
  2441. have a strong and that's harder to do when you're working in a spreadsheet
  2442. when you're working in a spreadsheet because you really have like the
  2443. because you really have like the spreadsheet's like a machete right it's
  2444. spreadsheet's like a machete right it's not a scalpel you're sort of hacking at
  2445. not a scalpel you're sort of hacking at the data moving it around trying a lot
  2446. the data moving it around trying a lot of different things with pivots and you
  2447. of different things with pivots and you can sometimes get lost in it or make a
  2448. can sometimes get lost in it or make a mistake but there's ways to avoid that
  2449. mistake but there's ways to avoid that too like one really common thing
  2450. too like one really common thing stress is you know you should always
  2451. stress is you know you should always keep a pure and unedited copy of your
  2452. keep a pure and unedited copy of your data
  2453. data that you have not added any columns to
  2454. that you have not added any columns to or much around with so you're always
  2455. or much around with so you're always able to retrace your footsteps back to
  2456. able to retrace your footsteps back to where you began right it could live as
  2457. where you began right it could live as an email attachment from a government
  2458. an email attachment from a government official that's fine you might want to
  2459. official that's fine you might want to save it to your computer but you don't
  2460. save it to your computer but you don't want to end up in a situation where you
  2461. want to end up in a situation where you lose track of where you started because
  2462. lose track of where you started because then it can be really hard to find
  2463. then it can be really hard to find mistakes or make sure verify that what
  2464. mistakes or make sure verify that what you've done is correct
  2465. you've done is correct i think we're getting to the point where
  2466. i think we're getting to the point where i'm supposed to stop spilling and so i
  2467. i'm supposed to stop spilling and so i figured i would stop with like a really
  2468. figured i would stop with like a really um sanctimonious moral lesson like that
  2469. um sanctimonious moral lesson like that was
  2470. was uh which i i appreciate you guys sort of
  2471. uh which i i appreciate you guys sort of sitting you know
  2472. sitting you know enduring but that's kind of the spiel
  2473. enduring but that's kind of the spiel there's a lot of other questions and
  2474. there's a lot of other questions and stuff we could answer in the spreadsheet
  2475. stuff we could answer in the spreadsheet but i don't want to go on too long and i
  2476. but i don't want to go on too long and i want to give us time to just have kind
  2477. want to give us time to just have kind of a
  2478. of a discussion where we can answer
  2479. discussion where we can answer specific technical questions you have or
  2480. specific technical questions you have or just have a broader conversation about
  2481. just have a broader conversation about data journalism chicago depaul
  2482. data journalism chicago depaul the cubs chances this fall
  2483. the cubs chances this fall whatever you want to get into claire
  2484. whatever you want to get into claire what do you think
  2485. what do you think yeah that sounds good i think my video
  2486. yeah that sounds good i think my video might have gone out there for a second
  2487. might have gone out there for a second it was pretty funny but uh yeah i
  2488. it was pretty funny but uh yeah i i like to take these last 15 minutes for
  2489. i like to take these last 15 minutes for us to do a q a
  2490. us to do a q a um as ben said if you guys have any
  2491. um as ben said if you guys have any questions for him or about his work or
  2492. questions for him or about his work or about data driven reporting and data
  2493. about data driven reporting and data journalism so haley it looks like you
  2494. journalism so haley it looks like you have a question go ahead
  2495. have a question go ahead yeah so my first well my question is you
  2496. yeah so my first well my question is you know if you create so if you find this
  2497. know if you create so if you find this number by yourself say so we find like
  2498. number by yourself say so we find like that 29
  2499. that 29 um number
  2500. um number if you don't
  2501. if you don't if you s you know you put this in a
  2502. if you s you know you put this in a piece that you're working on and you
  2503. piece that you're working on and you publish it you say this is the number
  2504. publish it you say this is the number that i came up with
  2505. that i came up with are people going to be critical of that
  2506. are people going to be critical of that if you're not like you know what i mean
  2507. if you're not like you know what i mean if you're not citing somewhere else well
  2508. if you're not citing somewhere else well i mean like you're definitely you're
  2509. i mean like you're definitely you're climbing out on a limb every time you do
  2510. climbing out on a limb every time you do it right and i think a lot of in
  2511. it right and i think a lot of in journalism we're often taking numbers
  2512. journalism we're often taking numbers from other people and kind of passing
  2513. from other people and kind of passing them along but you know
  2514. them along but you know the most i think often impactful and
  2515. the most i think often impactful and original data journalism stories are
  2516. original data journalism stories are calculating their own findings right
  2517. calculating their own findings right yeah but i think but you know you have
  2518. yeah but i think but you know you have with that with that comes a higher
  2519. with that with that comes a higher degree of rigor and responsibility
  2520. degree of rigor and responsibility that's necessary yeah and so for me it's
  2521. that's necessary yeah and so for me it's really like i was saying understanding
  2522. really like i was saying understanding every step you're making in the process
  2523. every step you're making in the process uh making sure you fully grasp and
  2524. uh making sure you fully grasp and understand your data what is each row
  2525. understand your data what is each row what's in the data set what's left out
  2526. what's in the data set what's left out of the data set there's stuff we didn't
  2527. of the data set there's stuff we didn't cover like maybe we need to exclude some
  2528. cover like maybe we need to exclude some records right or maybe you need to like
  2529. records right or maybe you need to like really carefully frame your findings so
  2530. really carefully frame your findings so that you don't overstate it or say it
  2531. that you don't overstate it or say it kind of wrong you know what i mean yeah
  2532. kind of wrong you know what i mean yeah refinement process is hard especially
  2533. refinement process is hard especially when you're starting but once you get
  2534. when you're starting but once you get the hang of it it's not so bad i mean
  2535. the hang of it it's not so bad i mean obviously i think you if you're you know
  2536. obviously i think you if you're you know a lot of data journalism will get into
  2537. a lot of data journalism will get into the investigative space you know and
  2538. the investigative space you know and once you're there
  2539. once you're there um you know i think you really want to
  2540. um you know i think you really want to go above and beyond to give people an
  2541. go above and beyond to give people an opportunity to respond you know i've
  2542. opportunity to respond you know i've done you know not like i'm mr data
  2543. done you know not like i'm mr data journalism or whatever but like i've
  2544. journalism or whatever but like i've done stories that have gotten people
  2545. done stories that have gotten people fired
  2546. fired or
  2547. or put in the newspaper that uh uh
  2548. put in the newspaper that uh uh something is dangerous or you know like
  2549. something is dangerous or you know like i did a story like this is the most
  2550. i did a story like this is the most dangerous helicopter kind of thing
  2551. dangerous helicopter kind of thing and like and in those circumstances
  2552. and like and in those circumstances i always make sure that the subject the
  2553. i always make sure that the subject the people i'm writing about see everything
  2554. people i'm writing about see everything i'm writing about before it's published
  2555. i'm writing about before it's published and they have an opportunity to respond
  2556. and they have an opportunity to respond to
  2557. to um every basically the analysis itself
  2558. um every basically the analysis itself so in the case of the helicopter company
  2559. so in the case of the helicopter company where we're going to publish a story
  2560. where we're going to publish a story that is effectively
  2561. that is effectively ben says this is the most dangerous
  2562. ben says this is the most dangerous helicopter in the world
  2563. helicopter in the world the actual computer code that i wrote
  2564. the actual computer code that i wrote um i gave to the helicopter company
  2565. um i gave to the helicopter company ahead of time here's how i calculated
  2566. ahead of time here's how i calculated this i wrote out effectively a memo that
  2567. this i wrote out effectively a memo that was sent to them an email saying these
  2568. was sent to them an email saying these will be our key claims
  2569. will be our key claims and here's a bullet point summary of how
  2570. and here's a bullet point summary of how i arrived at those claims and you know
  2571. i arrived at those claims and you know in that case what we did is we modeled
  2572. in that case what we did is we modeled our study on one that had been done by
  2573. our study on one that had been done by the faa in the past so you know one way
  2574. the faa in the past so you know one way in investigations to deal with this
  2575. in investigations to deal with this issue is to look for outside standards
  2576. issue is to look for outside standards and outside methods that you're not just
  2577. and outside methods that you're not just like inventing you know so yeah that
  2578. like inventing you know so yeah that makes sense my method of calculating how
  2579. makes sense my method of calculating how the helicopters were dangerous was an
  2580. the helicopters were dangerous was an accident rate that was something that
  2581. accident rate that was something that the faa itself had done like 30 years
  2582. the faa itself had done like 30 years ago and so i recreated effectively
  2583. ago and so i recreated effectively something that had already been done by
  2584. something that had already been done by the agency and then in that process
  2585. the agency and then in that process i went to the agency i had an interview
  2586. i went to the agency i had an interview with them and i said i did this analysis
  2587. with them and i said i did this analysis i was on the phone with their data guy
  2588. i was on the phone with their data guy and i filtered this and i grouped that
  2589. and i filtered this and i grouped that and i joined this
  2590. and i joined this right is there anything i'm getting
  2591. right is there anything i'm getting wrong or missing and you give them an
  2592. wrong or missing and you give them an opportunity to tell you you're wrong so
  2593. opportunity to tell you you're wrong so you did all that stuff behind the scenes
  2594. you did all that stuff behind the scenes before the story came out
  2595. before the story came out and so recreating experiments or things
  2596. and so recreating experiments or things that other people have already done that
  2597. that other people have already done that have sort of like some justification is
  2598. have sort of like some justification is good also using standards that are sort
  2599. good also using standards that are sort of objective or defined by the
  2600. of objective or defined by the um
  2601. um defined by
  2602. defined by the subject themselves right so i've
  2603. the subject themselves right so i've done stories about the 911 system in la
  2604. done stories about the 911 system in la and like you can calculate the stats and
  2605. and like you can calculate the stats and say it's slow but it's like slow
  2606. say it's slow but it's like slow according to who
  2607. according to who yeah according to you yeah no well
  2608. yeah according to you yeah no well actually there's like a group of fire
  2609. actually there's like a group of fire chiefs who meet and set the standards
  2610. chiefs who meet and set the standards and i'm judging you against that
  2611. and i'm judging you against that right and so finding an outside standard
  2612. right and so finding an outside standard is often really key
  2613. is often really key to all investigative stories like did
  2614. to all investigative stories like did someone break the law or not would be a
  2615. someone break the law or not would be a classic example right
  2616. classic example right but can be really helpful when defining
  2617. but can be really helpful when defining your data methodology of kind of what
  2618. your data methodology of kind of what your target is right and i think that
  2619. your target is right and i think that these complaint stories kind of have an
  2620. these complaint stories kind of have an issue there because you can say well
  2621. issue there because you can say well only a small number of complaints
  2622. only a small number of complaints are um upheld right you can put that in
  2623. are um upheld right you can put that in the paper it's x percent but like is
  2624. the paper it's x percent but like is that how do we know that's good or bad
  2625. that how do we know that's good or bad you really don't right and that and that
  2626. you really don't right and that and that is i think
  2627. is i think one reason why the stories stories have
  2628. one reason why the stories stories have great impact they're great stories i'm
  2629. great impact they're great stories i'm jealous of them but they're missing that
  2630. jealous of them but they're missing that standard piece that really helps you hit
  2631. standard piece that really helps you hit in a lot of cases you know
  2632. yeah yeah context context context context so important
  2633. context so important um nadia i think you have a question
  2634. um nadia i think you have a question yeah hi thank you so much for um
  2635. yeah hi thank you so much for um speaking with us today um i did have a
  2636. speaking with us today um i did have a question so
  2637. question so typically i
  2638. typically i don't do math and i don't really do data
  2639. don't do math and i don't really do data um but i'm wondering like kind of what
  2640. um but i'm wondering like kind of what you were saying in that like last part
  2641. you were saying in that like last part of your answer to the question about how
  2642. of your answer to the question about how kind of data journalism and also but
  2643. kind of data journalism and also but like data can kind of like impact and
  2644. like data can kind of like impact and strengthen like a story so how would you
  2645. strengthen like a story so how would you say
  2646. say would be a good way to insert like a
  2647. would be a good way to insert like a little bit of data journalism in a story
  2648. little bit of data journalism in a story to kind of strengthen it without take
  2649. to kind of strengthen it without take without making it like the focus of a
  2650. without making it like the focus of a story yeah i mean i mean to me looking
  2651. story yeah i mean i mean to me looking to it for context and just like a little
  2652. to it for context and just like a little bit of background is where it's most
  2653. bit of background is where it's most commonly used like hey i'm writing the
  2654. commonly used like hey i'm writing the story about the chief of police saying
  2655. story about the chief of police saying homicides are up and it's a big problem
  2656. homicides are up and it's a big problem in chicago and it definitely is and i
  2657. in chicago and it definitely is and i wouldn't want to minimize that but that
  2658. wouldn't want to minimize that but that story would benefit from context of like
  2659. story would benefit from context of like well okay it's up over this year but how
  2660. well okay it's up over this year but how does it compare to 20 years ago or like
  2661. does it compare to 20 years ago or like what's the general trend it may be a
  2662. what's the general trend it may be a chart right and so i think like
  2663. chart right and so i think like background and context is like the most
  2664. background and context is like the most common thing you know so like often the
  2665. common thing you know so like often the thing that's in the news is a really
  2666. thing that's in the news is a really short time frame and just like
  2667. short time frame and just like stretching that out is good
  2668. stretching that out is good um i think you know understanding uh
  2669. um i think you know understanding uh something sense of proportion you know
  2670. something sense of proportion you know what i mean so like oh hospitalizations
  2671. what i mean so like oh hospitalizations are really up for coven among young
  2672. are really up for coven among young people but how many young people are
  2673. people but how many young people are actually being hospitalized like the
  2674. actually being hospitalized like the actual number right and so i think uh
  2675. actual number right and so i think uh context over time and kind of contest of
  2676. context over time and kind of contest of like in in the grand scheme of things
  2677. like in in the grand scheme of things you know what i mean are probably the
  2678. you know what i mean are probably the two most common data things if i had to
  2679. two most common data things if i had to i guess guess right um i also think
  2680. i guess guess right um i also think let's say you cover education or you
  2681. let's say you cover education or you cover the hospital system or you cover
  2682. cover the hospital system or you cover whatever i think you could put on a sort
  2683. whatever i think you could put on a sort of metaphorical set of database glasses
  2684. of metaphorical set of database glasses and kind of look at your beat and say
  2685. and kind of look at your beat and say well what's the data gathered on this
  2686. well what's the data gathered on this beat that i could maybe use to bring
  2687. beat that i could maybe use to bring some accountability or context and
  2688. some accountability or context and perspective to a story education's a
  2689. perspective to a story education's a classic example there's just so much
  2690. classic example there's just so much education data of all different shapes
  2691. education data of all different shapes and sizes that i think probably every
  2692. and sizes that i think probably every education reporter in america does some
  2693. education reporter in america does some data right it's like kind of whether
  2694. data right it's like kind of whether it's test scores or
  2695. it's test scores or dropped enrollment after covid or you
  2696. dropped enrollment after covid or you name it right you know there's just a
  2697. name it right you know there's just a lot of data inequity right is also a
  2698. lot of data inequity right is also a classic data frame
  2699. classic data frame and
  2700. and and i think i think looking at your beat
  2701. and i think i think looking at your beat and saying well what's the data gathered
  2702. and saying well what's the data gathered by this
  2703. by this this thing i'm covering or what are the
  2704. this thing i'm covering or what are the goals and standards that they claim that
  2705. goals and standards that they claim that they're reaching for or upholding and
  2706. they're reaching for or upholding and can i use data to measure whether that's
  2707. can i use data to measure whether that's really the case or not
  2708. anyone else have any questions you can either drop them in the chat or go ahead
  2709. either drop them in the chat or go ahead and
  2710. yeah i have one um so you've been doing this for a while you've been doing it
  2711. this for a while you've been doing it for a minute um
  2712. for a minute um and i mean i don't know what depaul's
  2713. and i mean i don't know what depaul's program journalism program was like 20
  2714. program journalism program was like 20 years ago but i'm guessing it didn't
  2715. years ago but i'm guessing it didn't have a super strong data journalism
  2716. have a super strong data journalism program so i guess how have you learned
  2717. program so i guess how have you learned and picked up all of these different
  2718. and picked up all of these different data skills over the past two decades
  2719. data skills over the past two decades have you like just gotten a thousand
  2720. have you like just gotten a thousand degrees
  2721. degrees from people in newsrooms like what's
  2722. from people in newsrooms like what's what's your secret sure
  2723. what's your secret sure um
  2724. um you're right yeah so when i went to
  2725. you're right yeah so when i went to depaul there was it was just the college
  2726. depaul there was it was just the college of communication i actually answered
  2727. of communication i actually answered phones at the front desk
  2728. phones at the front desk uh that was my gig
  2729. uh that was my gig um
  2730. um there was no data journalism program
  2731. there was no data journalism program no
  2732. no uh yes this was on the fifth floor of
  2733. uh yes this was on the fifth floor of sac i don't think it's still there right
  2734. sac i don't think it's still there right i don't know but that's where that's
  2735. i don't know but that's where that's where the department used to be up at
  2736. where the department used to be up at the top and
  2737. the top and use the whole loop thing now and it's
  2738. use the whole loop thing now and it's you know but um
  2739. you know but um no they're real very few places that any
  2740. no they're real very few places that any data journalism programs then data
  2741. data journalism programs then data journalism in the united states really
  2742. journalism in the united states really was kind of a grassroots movement that
  2743. was kind of a grassroots movement that grew out of the 60s and 70s and really
  2744. grew out of the 60s and 70s and really not unlike
  2745. not unlike hackers who made computers in silicon
  2746. hackers who made computers in silicon valley right there was sort of just like
  2747. valley right there was sort of just like a a niche of people who saw the
  2748. a a niche of people who saw the potential of the computer and began to
  2749. potential of the computer and began to work it into their profession and in
  2750. work it into their profession and in journalism that was really the it was
  2751. journalism that was really the it was really an investigative tradition to
  2752. really an investigative tradition to start with it's like we want to look at
  2753. start with it's like we want to look at the census data using a big crazy
  2754. the census data using a big crazy computer or we want to survey our
  2755. computer or we want to survey our readers about what they think about the
  2756. readers about what they think about the detroit riots in 1968 that was a really
  2757. detroit riots in 1968 that was a really big early data journalism story or we
  2758. big early data journalism story or we want to investigate the county
  2759. want to investigate the county courthouse for racial inequities right
  2760. courthouse for racial inequities right these and and a sort of small tribe of
  2761. these and and a sort of small tribe of investigative journalists began
  2762. investigative journalists began developing those skills in the 60s 70s
  2763. developing those skills in the 60s 70s 80s and then they formed actually a sort
  2764. 80s and then they formed actually a sort of
  2765. of a non-profit group that's mission was to
  2766. a non-profit group that's mission was to train journalists in what was then
  2767. train journalists in what was then called computer assisted reporting
  2768. called computer assisted reporting and that became a group at the
  2769. and that became a group at the university of missouri which is the
  2770. university of missouri which is the national institute for computer assisted
  2771. national institute for computer assisted reporting which is kind of a ridiculous
  2772. reporting which is kind of a ridiculous name today you know it's like
  2773. name today you know it's like everyone is computer assisted in
  2774. everyone is computer assisted in everything they do we don't call it
  2775. everything they do we don't call it computer assisted photography right or
  2776. computer assisted photography right or whatever uh but um so that's data
  2777. whatever uh but um so that's data journalism has kind of replaced that as
  2778. journalism has kind of replaced that as a term but um after i finished it depaul
  2779. a term but um after i finished it depaul i went to the university of missouri to
  2780. i went to the university of missouri to graduate school and i was a graduate
  2781. graduate school and i was a graduate assistant at this place nikar and as a
  2782. assistant at this place nikar and as a graduate assistant working there i was
  2783. graduate assistant working there i was able to work on stories with newsrooms
  2784. able to work on stories with newsrooms that partnered with it and spend a year
  2785. that partnered with it and spend a year and a half at mizzou really focusing and
  2786. and a half at mizzou really focusing and that's really where i began to learn how
  2787. that's really where i began to learn how to code and i really began to learn and
  2788. to code and i really began to learn and i met a lot of people who did this field
  2789. i met a lot of people who did this field now since i graduated it's really become
  2790. now since i graduated it's really become much more institutionalized paul has a
  2791. much more institutionalized paul has a class there's many many graduate
  2792. class there's many many graduate programs that explicitly focus on data
  2793. programs that explicitly focus on data journalism right now that just didn't
  2794. journalism right now that just didn't exist then and i think that's great and
  2795. exist then and i think that's great and i think that people are getting a lot
  2796. i think that people are getting a lot more education on it but i really i was
  2797. more education on it but i really i was lucky to benefit from like the one
  2798. lucky to benefit from like the one program that existed at the time that
  2799. program that existed at the time that kind of got me into it
  2800. kind of got me into it and graduate school was great for me you
  2801. and graduate school was great for me you know um i
  2802. know um i mizzou was great for me um you know it's
  2803. mizzou was great for me um you know it's got a great program in it the tradition
  2804. got a great program in it the tradition is there um you know i would just say
  2805. is there um you know i would just say it's it's compared to other graduate
  2806. it's it's compared to other graduate schools very very inexpensive
  2807. schools very very inexpensive um you can see the chart in the wall
  2808. um you can see the chart in the wall street journal about this mizzou is
  2809. street journal about this mizzou is probably the least expensive journalism
  2810. probably the least expensive journalism graduate school
  2811. graduate school in its class you know what i mean by a
  2812. in its class you know what i mean by a long margin so i was able to go in an
  2813. long margin so i was able to go in an inexpensive way and it really worked out
  2814. inexpensive way and it really worked out for me and then i just kind of got jobs
  2815. for me and then i just kind of got jobs and learned from people i work with i
  2816. and learned from people i work with i was lucky to start off in non-profit
  2817. was lucky to start off in non-profit news at the center for public integrity
  2818. news at the center for public integrity which is something that's also changed
  2819. which is something that's also changed since then but i really do think
  2820. since then but i really do think nonprofit news is a great place to start
  2821. nonprofit news is a great place to start out because um the newsrooms tend to be
  2822. out because um the newsrooms tend to be smaller they tend to be more projects
  2823. smaller they tend to be more projects focused so you can like there's not
  2824. focused so you can like there's not quite so much of a hurry so they're not
  2825. quite so much of a hurry so they're not great places to learn how to like write
  2826. great places to learn how to like write right right you know what i mean because
  2827. right right you know what i mean because they oftentimes don't move as quickly
  2828. they oftentimes don't move as quickly but they are great places to learn how
  2829. but they are great places to learn how to do projects to learn how to do
  2830. to do projects to learn how to do investigations to learn how to like
  2831. investigations to learn how to like um put together something that goes
  2832. um put together something that goes beyond the average story so i would
  2833. beyond the average story so i would really recommend them as a place to look
  2834. really recommend them as a place to look for a first job
  2835. for a first job um or whatever because you can be kind
  2836. um or whatever because you can be kind of shielded from some of the
  2837. of shielded from some of the insanity of a larger newsroom oftentimes
  2838. insanity of a larger newsroom oftentimes but different people want different
  2839. but different people want different things in their career but for a data
  2840. things in their career but for a data person it's a great place to start it
  2841. person it's a great place to start it was for me
  2842. great thank you [Music]
  2843. well i have one um so ben what would you recommend for people today who
  2844. recommend for people today who maybe are just now getting introduced or
  2845. maybe are just now getting introduced or this might be their first introduction
  2846. this might be their first introduction into data journalism and spreadsheets
  2847. into data journalism and spreadsheets where do you think they should go to
  2848. where do you think they should go to continue refining the skills that they
  2849. continue refining the skills that they learned today what would be the next
  2850. learned today what would be the next step i think putting on if you're
  2851. step i think putting on if you're covering something like in 14 east or
  2852. covering something like in 14 east or the depaul or somewhere else i think
  2853. the depaul or somewhere else i think really putting on the database glasses
  2854. really putting on the database glasses and saying well i'm covering this topic
  2855. and saying well i'm covering this topic like can i come up with a story where is
  2856. like can i come up with a story where is there some data on this beat that i can
  2857. there some data on this beat that i can like try to do something with and that
  2858. like try to do something with and that could just be you can look at it as just
  2859. could just be you can look at it as just even a breaking news story like some new
  2860. even a breaking news story like some new data gets released what does it say i
  2861. data gets released what does it say i think covet data right now is a great
  2862. think covet data right now is a great opportunity to do this you know because
  2863. opportunity to do this you know because it's just
  2864. it's just you know obviously not the crazy story
  2865. you know obviously not the crazy story it was two years ago but still a really
  2866. it was two years ago but still a really big story and there's probably an
  2867. big story and there's probably an opportunity to find some angle on it if
  2868. opportunity to find some angle on it if you begin to
  2869. you begin to to look at it it's finding that data on
  2870. to look at it it's finding that data on your bead or something coming and then
  2871. your bead or something coming and then really treating it like a source and
  2872. really treating it like a source and like challenging yourself with to come
  2873. like challenging yourself with to come up with questions to answer that ask the
  2874. up with questions to answer that ask the data and then using some of these basic
  2875. data and then using some of these basic skills we covered to try to answer them
  2876. skills we covered to try to answer them and even if you don't have writing a
  2877. and even if you don't have writing a story just that practice of like finding
  2878. story just that practice of like finding the data on the beat
  2879. the data on the beat asking a question answering it is really
  2880. asking a question answering it is really just how you can start to strengthen
  2881. just how you can start to strengthen your muscles and kind of get more
  2882. your muscles and kind of get more comfortable
  2883. comfortable with this kind of approach
  2884. i actually have one more question and i hope it's not too personal
  2885. hope it's not too personal um
  2886. um where's pale wire from what what was the
  2887. where's pale wire from what what was the inspiration for that
  2888. inspiration for that this is my online handle so like
  2889. this is my online handle so like yeah so like i'm old enough you know i'm
  2890. yeah so like i'm old enough you know i'm gray-haired enough that like i i joined
  2891. gray-haired enough that like i i joined the internet in the early 1990s
  2892. the internet in the early 1990s when it was still
  2893. when it was still it was maybe even considered cool as
  2894. it was maybe even considered cool as weird as that it sound to have like a
  2895. weird as that it sound to have like a handle or a username that you kept
  2896. handle or a username that you kept and i think gradually over time it's
  2897. and i think gradually over time it's probably been a good development that
  2898. probably been a good development that people just use their names you know um
  2899. people just use their names you know um but i've kind of just always been
  2900. but i've kind of just always been attached to mine because i feel like
  2901. attached to mine because i feel like it's kind of a marker of where i came
  2902. it's kind of a marker of where i came from and my whatever and it's also a
  2903. from and my whatever and it's also a little bit of an internet brand but it
  2904. little bit of an internet brand but it actually it's pretty pretentious you
  2905. actually it's pretty pretentious you know it comes from william shakespeare
  2906. know it comes from william shakespeare um so uh timon of athens is a you know
  2907. um so uh timon of athens is a you know shakespeare play that has a soliloquy
  2908. shakespeare play that has a soliloquy about a
  2909. about a the moon and it's uh the moon has a pale
  2910. the moon and it's uh the moon has a pale fire that and um the silicone is called
  2911. fire that and um the silicone is called each thing's a thief you can find it and
  2912. each thing's a thief you can find it and it's sort of commonly played upon by
  2913. it's sort of commonly played upon by vladimir nabokov and others through the
  2914. vladimir nabokov and others through the years for different things they do and
  2915. years for different things they do and and um i always i've always i thought
  2916. and um i always i've always i thought that this locally has a certain
  2917. that this locally has a certain resonance or
  2918. resonance or you could see its connection to
  2919. you could see its connection to journalism um if you read it
  2920. journalism um if you read it and uh i just sounded cool when i was 21
  2921. and uh i just sounded cool when i was 21 or whatever and so you know
  2922. so that's awesome i love that
  2923. that's awesome i love that yeah it's pretty nerdy my my wife's not
  2924. yeah it's pretty nerdy my my wife's not a big fan
  2925. a big fan but i do think you know this and this
  2926. but i do think you know this and this has been this has been a big thing in
  2927. has been this has been a big thing in journalism twitter maybe you guys are
  2928. journalism twitter maybe you guys are getting pulled into that world i'm sorry
  2929. getting pulled into that world i'm sorry about you know personal branding as
  2930. about you know personal branding as people call it you know and whether
  2931. people call it you know and whether that's a good thing or a bad thing and i
  2932. that's a good thing or a bad thing and i don't think there's really a simple
  2933. don't think there's really a simple answer but i do think it's true that you
  2934. answer but i do think it's true that you know if you're not someone who gets
  2935. know if you're not someone who gets lucky to get a big job right away or who
  2936. lucky to get a big job right away or who is like you know rocketed to the top
  2937. is like you know rocketed to the top making it into journalism you kind of
  2938. making it into journalism you kind of have to carve out your niche of like who
  2939. have to carve out your niche of like who you are and i mean part of that is
  2940. you are and i mean part of that is branding you might not want to call it
  2941. branding you might not want to call it branding you might not want to think
  2942. branding you might not want to think about it that way but i i you know who
  2943. about it that way but i i you know who are you as a journalist what do you do
  2944. are you as a journalist what do you do and so for me like that decision kind of
  2945. and so for me like that decision kind of coming out of mizzou and depaul was like
  2946. coming out of mizzou and depaul was like i'm a data guy and i'm an internet guy
  2947. i'm a data guy and i'm an internet guy and that led me to do more web
  2948. and that led me to do more web development and stuff that some of my
  2949. development and stuff that some of my peers didn't want to do but i saw as
  2950. peers didn't want to do but i saw as like a path forward in my career as
  2951. like a path forward in my career as opportunity
  2952. opportunity and then part of that was like keeping
  2953. and then part of that was like keeping my dumb internet handle right because i
  2954. my dumb internet handle right because i wanted to kind of just communicate
  2955. wanted to kind of just communicate it sounds stupid to say today but like
  2956. it sounds stupid to say today but like in 2005 to say like i am an internet
  2957. in 2005 to say like i am an internet journalist
  2958. journalist was like you were a weirdo you know what
  2959. was like you were a weirdo you know what i mean and like
  2960. i mean and like and and so there's part of me that like
  2961. and and so there's part of me that like that worked for me and there's also part
  2962. that worked for me and there's also part of me that feels like that's kind of my
  2963. of me that feels like that's kind of my roots in a way and i don't want to give
  2964. roots in a way and i don't want to give it up even though at this point it's
  2965. it up even though at this point it's pretty uncool
  2966. pretty uncool but i think generally having your like
  2967. but i think generally having your like thing of like this is my thing and your
  2968. thing of like this is my thing and your thing doesn't have to be a dumb internet
  2969. thing doesn't have to be a dumb internet handle it doesn't have to be acting like
  2970. handle it doesn't have to be acting like a fool and showing your ass on
  2971. a fool and showing your ass on twitter you know what i mean
  2972. twitter you know what i mean but
  2973. but it you kind of want to figure out what
  2974. it you kind of want to figure out what you know what is it your thing and you
  2975. you know what is it your thing and you don't have to know right away
  2976. don't have to know right away and like part of that might be redefined
  2977. and like part of that might be redefined over time but kind of saying this is
  2978. over time but kind of saying this is like kind of what i do
  2979. like kind of what i do and making sure you have your little tag
  2980. and making sure you have your little tag line that clearly communicates that and
  2981. line that clearly communicates that and your website that like clearly
  2982. your website that like clearly communicates that
  2983. communicates that and then you kind of beat that drum and
  2984. and then you kind of beat that drum and sadly self-promote because you know
  2985. sorry you know like it's just it's it's part of kind of making it not like i'm
  2986. part of kind of making it not like i'm any huge success story but like
  2987. any huge success story but like it does help people know what i do and
  2988. it does help people know what i do and they're like what's pale wire i'm like
  2989. they're like what's pale wire i'm like oh that's me i'm the nerd you know
  2990. oh that's me i'm the nerd you know and oh yeah the nerd yeah
  2991. well thank you so much for that explanation um if we don't have any
  2992. explanation um if we don't have any other questions we can go ahead and wrap
  2993. other questions we can go ahead and wrap up um perfect right on the dot right at
  2994. up um perfect right on the dot right at nine o'clock uh ben thank you so much
  2995. nine o'clock uh ben thank you so much for being with us tonight i know spj
  2996. for being with us tonight i know spj depaul and fort denise are so grateful
  2997. depaul and fort denise are so grateful for your time
  2998. for your time and one more thing before we go we do
  2999. and one more thing before we go we do want to plug that this workshop was free
  3000. want to plug that this workshop was free and open to the public
  3001. and open to the public but um we are a student journalism
  3002. but um we are a student journalism newsroom and we need funding so
  3003. newsroom and we need funding so if you got anything out of this workshop
  3004. if you got anything out of this workshop tonight and you have the means we'd
  3005. tonight and you have the means we'd really encourage you to donate to our
  3006. really encourage you to donate to our student newsrooms fundraiser um so we
  3007. student newsrooms fundraiser um so we can fund more engagement events and
  3008. can fund more engagement events and workshops like this in the future and
  3009. workshops like this in the future and grace has dropped the link grace is on
  3010. grace has dropped the link grace is on it grace dropped the link to the
  3011. it grace dropped the link to the fundraiser in the chat so
  3012. fundraiser in the chat so anything helps if you want to donate um
  3013. anything helps if you want to donate um but other than that thank you so much
  3014. but other than that thank you so much ben for being with us tonight
  3015. ben for being with us tonight thank you for having me and thank you
  3016. thank you for having me and thank you for staying up late on a school night to
  3017. for staying up late on a school night to get nerdy i mean i really appreciate it
  3018. get nerdy i mean i really appreciate it also i would just add if anybody wants
  3019. also i would just add if anybody wants to talk about a story or has questions
  3020. to talk about a story or has questions or just wants whatever um i mean please
  3021. or just wants whatever um i mean please feel free to reach out i'm going to put
  3022. feel free to reach out i'm going to put my email my personal email into the chat
  3023. my email my personal email into the chat um you know you know i'm not uh
  3024. um you know you know i'm not uh you know i'm happy i've got time i'm
  3025. you know i'm happy i've got time i'm happy to talk just feel free to reach
  3026. happy to talk just feel free to reach out and google me probably find my phone
  3027. out and google me probably find my phone number too if you want it you know it's
  3028. number too if you want it you know it's like so
  3029. like so um
  3030. um thank you again
  3031. thank you again go demons

Downloads

Recording video · Timestamped transcript