Part 1: Introduction to interactive databases

The team at ProPublica started with a simple idea: Collect all the payments that pharmaceutical companies make to doctors. Then build a searchable database that lets the public look up how much money any physician in America has received.

Pulling it off was more complicated. Dan Nguyen had to find and download countless payment disclosures from a baffling array of different websites, then spend hours standardizing the inconsistent data. It was months of work before he and his teammates could even begin to think about posting it online.

When "Dollar for Docs" finally launched in 2010, the response was overwhelming. Readers searched it tens of millions of times. Dozens of newsrooms used the data to report their own stories. Law enforcement used it to investigate drug companies.

Dollars for Docs

https://projects.propublica.org/docdollars/

The project was so influential that Congress eventually required drug makers to disclose their payments to a unified federal database, copying the model that ProPublica pioneered.

Examined from a technical perspective, "Dollar for Docs" is an example of a distinct type of news application. You start with a public dataset. You pull it into a web application. You give readers tools to explore it themselves. The formula works for government bailouts, preventive power outages, money given to political candidates, complaints made against police officers or anything else that people want to know about.

Regardless of the topic, the underlying architecture is the same. You have a dataset drawn from outside your code. You write a script to fetch and reshape it. You pull it into your page using a framework's data loading pattern. You render individual records with loops and templates. You use inputs, like the ones you covered last week, to make the page interactive.

You'll cover all of those fundamentals today, step by step, using a public dataset published by New York City.

Part 2: Fetching data with Node.js

Up to now, all of the data in your projects has been typed by hand. Last week, you hardcoded streaming prices and tip amounts directly into your components. Real projects are powered by real data — from spreadsheets, databases, PDFs and APIs — that get pulled in before the page is built.

Today you're going to learn how to bring external data into a Svelte project, where you can use it however you want. You'll use a dataset you see evidence of every day in New York City: The inspection grades posted in the window of food establishments all across the five boroughs.

The New York City Department of Health and Mental Hygiene posts an updated dataset nearly every day on the city's open data portal, where you can download all inspections over the past three years of restaurants that remain open.

DOHMH New York City Restaurant Inspection Results

https://data.cityofnewyork.us/Health/DOHMH-New-York-City-Restaurant-Inspection-Results/43nn-pn8j/about_data

The full dataset contains nearly 300,000 records, each representing an administrative action taken by the city, covering far more than just the final letter grade.

Rather than download the whole thing, you'll write a small Node.js script that talks to the city's data server and asks for exactly the records we want. This is a common pattern in data journalism — you write a script that fetches, filters and shapes your data before it ever touches your project.

Let's start by creating a new repository from our template. Open your browser and navigate to github.com/palewire/cuny-jour-static-site-template.

cuny-jour-static-site-template

https://github.com/palewire/cuny-jour-static-site-template

Click the green "Use this template" button and name your new repository my-first-data-components. Make sure "Public" is selected, then click "Create repository."

Clone it to your computer, open it in Visual Studio Code. Run npm install in the terminal.

Next, create a new file in the root of your project called fetch-data.js. Paste in the following code. It's a Node.js script that will hit the city's data API, ask for the last 50,000 records and then save the most recent grade for each restaurant to a JSON file in our project.

// Node's built-in file system and path modules — lets us write files to disk
import fs from "fs";
import path from "node:path";

// The NYC Open Data API endpoint for restaurant inspections
const apiUrl = "https://data.cityofnewyork.us/resource/43nn-pn8j.json";

// Query parameters sent to the API using the Socrata Query Language (SoQL)
const params = new URLSearchParams({
  $limit: 50000, // max number of rows to return
  $where: "grade IN('A', 'B', 'C') AND inspection_date >= '2025-01-01'", // only graded inspections from 2025 onward
  $select: "camis,dba,boro,cuisine_description,inspection_date,grade,score", // only fetch the columns we need
  $order: "camis ASC, inspection_date DESC", // sort by restaurant ID, newest inspection first
});

// Combine the base URL and query string into one complete request URL
const url = `${apiUrl}?${params.toString()}`;

console.log(`Fetching data from NYC Open Data...`);

// Ensure the data directory exists so we can write the output file without a separate mkdir command
const dataDir = path.join("src", "lib", "data");
fs.mkdirSync(dataDir, { recursive: true });

// fetch() makes an HTTP GET request to the API and returns a Promise
fetch(url)
  // Parse the response body as JSON — also returns a Promise
  .then((response) => response.json())
  .then((data) => {
    console.log(`Fetched ${data.length} records from the API.`);

    // A Set lets us track which restaurant IDs (camis) we've already seen,
    // so we can keep only the most recent inspection per restaurant
    const seen = new Set();
    const restaurants = data.filter((r) => {
      // If we've seen this camis before, skip it
      if (seen.has(r.camis)) return false;
      // Otherwise, mark it as seen and keep it
      seen.add(r.camis);
      return true;
    });

    // Sort by inspection_date descending so the newest inspections appear first
    restaurants.sort(
      (a, b) => new Date(b.inspection_date) - new Date(a.inspection_date),
    );

    console.log(`Filtered to ${restaurants.length} unique restaurants.`);

    // Write the cleaned data to a JSON file so our app can use it
    // JSON.stringify with null, 2 formats the output with 2-space indentation
    const outputPath = path.join(dataDir, "restaurants.json");
    fs.writeFileSync(outputPath, JSON.stringify(restaurants, null, 2));
    console.log("Saved to src/lib/data/restaurants.json");
  })
  // If anything goes wrong (network error, bad JSON, etc.), log it
  .catch((error) => {
    console.error("Error fetching data:", error);
  });

You don't need to understand everything it does here. The important thing to understand is that as a page designer, your job is to take a data file from the outside world and bring it to life. In some cases, you'll get that data from a colleague who has already cleaned it up for you. In other cases, you'll have to figure it out on your own. In a working newsroom, it can go either way, depending on the project.

Now run the script — it will create src/lib/data for you.

node fetch-data.js

You should see a message reporting how many raw records were downloaded and how many unique restaurants remain. Open src/lib/data/restaurants.json in Visual Studio Code and take a look. You'll see an array of objects, each representing one restaurant with its most recent letter grade. The most recent inspections should be at the top.

  {
    "camis": "40369165",
    "dba": "REIF'S TAVERN",
    "boro": "Manhattan",
    "cuisine_description": "American",
    "inspection_date": "2026-02-25T00:00:00.000",
    "grade": "A",
    "score": "9"
  },
  {
    "camis": "40385852",
    "dba": "SAM'S PLACE",
    "boro": "Manhattan",
    "cuisine_description": "Italian",
    "inspection_date": "2026-02-25T00:00:00.000",
    "grade": "A",
    "score": "10"
  },

Scroll to the bottom and you should see inspections from early January 2025, which is the cutoff we set in our script.

  {
    "camis": "50015659",
    "dba": "HARRY'S NEW YORK BAR",
    "boro": "Manhattan",
    "cuisine_description": "American",
    "inspection_date": "2025-01-02T00:00:00.000",
    "grade": "A",
    "score": "12"
  }

Loading data the SvelteKit way

You have your data file. Now you need to get it into your page.

SvelteKit has an official pattern for getting data into your pages. It's called a load function, and it lives in a special file called +page.js that sits right next to your +page.svelte.

Before your page component renders, SvelteKit runs the load function to prepare whatever data the page needs. The page then receives that data as a property. This separates getting the data from displaying it, which keeps your component focused on presentation.

This is the pattern used by complicated news applications with multiple data sources, authentication, pagination and more. It's also the pattern used by our simple project, because it's important to learn it from the start.

Create a new file at src/routes/+page.js. This file must be named exactly +page.js — SvelteKit looks for this specific filename.

import restaurants from '$lib/data/restaurants.json';

export function load() {
  return {
    restaurants
  };
}

This is a simple example of a +page.js. The first line imports our JSON data file, as it would any other Node.js library.

The load function is the special function that SvelteKit calls before rendering the page. Whatever object you return from it becomes available to your page component.

Now open src/routes/+page.svelte. At the bottom of the script block, we need to receive the data that the load function prepared. Add this line, which is just the like one we used to receive properties in our components:

let { data } = $props();

The data property is another SvelteKit convention. It automatically contains whatever your +page.js load function returned. So data.restaurants is our array of inspection records.

Let's verify it works. Add a $inspect() call right after the props line. It's the special Svelte function that logs a variable to the browser console.

$inspect(data.restaurants);

Start the development server with npm run dev and open your browser to localhost:5173. Open the browser's developer tools — you can do this by right-clicking anywhere on the page and selecting "Inspect," then clicking the "Console" tab.

You should see your data array logged in the console. Click the arrow to expand it and explore individual records. This is a useful trick for understanding any dataset — log it, expand it, poke at it.

Once you've confirmed it's working, you can remove the $inspect() line. We won't need it anymore.

Part 3: Looping through lists with {#each}

You have data flowing into your page. Now what? The whole point is to show it to readers. And since we have hundreds or thousands of records, we need a way to render them all without writing each one by hand.

That's where Svelte's {#each} block comes in. It loops over an array and renders one row at a time. It's the tool you'll reach for anytime you need to display a list, a table, a grid of cards, or any other repeating pattern.

Inside the ArticleBody in +page.svelte, add an HTML table that uses an {#each} block to repeat the same pattern for each row in data.restaurants.

<table>
  <thead>
    <tr>
      <th>Restaurant</th>
      <th>Borough</th>
      <th>Cuisine</th>
      <th>Grade</th>
    </tr>
  </thead>
  <tbody>
    {#each data.restaurants as row}
      <tr>
        <td>{row.dba}</td>
        <td>{row.boro}</td>
        <td>{row.cuisine_description}</td>
        <td>{row.grade}</td>
      </tr>
    {/each}
  </tbody>
</table>

Inside the block, we use the familiar curly-brace syntax to insert each field. The row.dba expression pulls the business name. The row.boro gets the borough. And so on.

Save and check your browser. You'll see all 5,000 records printed out on one page. Way more than we'd ever want to show. You get the point. One loop was able to print out the entire dataset.

Creating a RestaurantTable component

Let's clean things up by extracting your table into a reusable component, just like you've done in previous weeks.

Create a new file at src/lib/components/RestaurantTable.svelte. Add this:

<script>
  let { restaurantList } = $props();
</script>

<table>
  <thead>
    <tr>
      <th>Restaurant</th>
      <th>Borough</th>
      <th>Cuisine</th>
      <th>Grade</th>
    </tr>
  </thead>
  <tbody>
    {#each restaurantList as row}
      <tr>
        <td>{row.dba}</td>
        <td>{row.boro}</td>
        <td>{row.cuisine_description}</td>
        <td>{row.grade}</td>
      </tr>
    {/each}
  </tbody>
</table>

<style>
  table {
    width: 100%;
    border-collapse: collapse;
    font-size: 0.875rem;
  }

  th {
    text-align: left;
    padding: 0.5rem;
    border-bottom: 2px solid var(--color-dark-gray);
    font-weight: bold;
    text-transform: uppercase;
    font-size: 0.75rem;
    letter-spacing: 0.05em;
  }

  td {
    padding: 0.5rem;
    border-bottom: 1px solid #eee;
  }

  tr:hover {
    background-color: var(--color-light-gray);
  }
</style>

Notice how it's the same code as in our +page.svelte, but now it's wrapped in a component with a restaurantList property, that can be passed in from the outside.

Back in src/routes/+page.svelte, import the component.

import RestaurantTable from '$lib/components/RestaurantTable.svelte';

Now delete the table you wrote in +page.svelte and replace it with the new component, passing in the data as a property.

<RestaurantTable data={data.restaurants} />

Save and reload. Similar result, a little cleaner though.

Part 4: Filtering with reactive state

We have a table. But it shows every restaurant in the dataset at once. What if a reader only cares about restaurants in their borough? Or wants to see just the places that got a C?

You'll take what you learned last week and add dropdown menus that let readers filter the data.

Adding a borough filter

Start by adding a $state variable to your script block in +page.svelte for the selected borough. We'll default it to an empty string, which will represent "show all boroughs."

let { data } = $props();
$inspect(data.restaurants);

let selectedBorough = $state("");

Now we need to create a filteredRestaurants variable that filters the data based on the selection. Add this below your props:

let { data } = $props();
$inspect(data.restaurants);

let selectedBorough = $state("");

let filteredRestaurants = $derived(
  selectedBorough === ''
    ? data.restaurants
    : data.restaurants.filter(r => r.boro === selectedBorough)
);

This reads: if selectedBorough is empty, use the full array. Otherwise, keep only restaurants where the boro field matches the selection.

Because filteredRestaurants is $derived, the table will update automatically whenever selectedBorough changes.

Now add a <select> dropdown to your page, above the table. We'll use bind:value to connect it to our state variable, just like we did with the tip calculator last week.

<div class="filters">
  <label for="borough">Borough</label>
  <select id="borough" bind:value={selectedBorough}>
    <option value="">All boroughs</option>
    <option value="Manhattan">Manhattan</option>
    <option value="Brooklyn">Brooklyn</option>
    <option value="Queens">Queens</option>
    <option value="Bronx">Bronx</option>
    <option value="Staten Island">Staten Island</option>
  </select>
</div>

And update the table to use our filtered filteredRestaurants variable:

<RestaurantTable data={filteredRestaurants} />

Save and check your browser. Select "Brooklyn" from the dropdown.

The table shrinks to show only Brooklyn restaurants. Switch to "Queens" and it all updates again. Select "All boroughs" and you're back to the full dataset.

Adding a cuisine filter

Let's add a second dropdown for cuisine type. This works the same way, but since there are dozens of cuisine types in the data, we'll generate the options from the dataset itself.

Add another state variable in the script section at the top +page.svelte for the selected cuisine:

let { data } = $props();
$inspect(data.restaurants);

let selectedBorough = $state("");
let selectedCuisine = $state("");

let filteredRestaurants = $derived(
  selectedBorough === ''
    ? data.restaurants
    : data.restaurants.filter(r => r.boro === selectedBorough)
);

We'll need a list of all the unique cuisine types in the data. Add a derived value that extracts them:

let { data } = $props();
$inspect(data.restaurants);

let selectedBorough = $state("");
let selectedCuisine = $state("");

let cuisines = $derived(
  [...new Set(data.restaurants.map(r => r.cuisine_description))].sort()
);

let filteredRestaurants = $derived(
  selectedBorough === ''
    ? data.restaurants
    : data.restaurants.filter(r => r.boro === selectedBorough)
);

This line does a few things at once. The .map() method pulls the cuisine_description from every record. Wrapping it in new Set() eliminates duplicates, since a Set can only contain unique values. The [...] spread syntax converts the Set back into an array. And .sort() puts them in alphabetical order.

Notice we derive the cuisine list from data.restaurants — the full, unfiltered dataset — so the dropdown always shows all available cuisines regardless of the current borough selection.

Now update the filteredRestaurants variable to account for both dropdowns:

let { data } = $props();
$inspect(data.restaurants);

let selectedBorough = $state("");
let selectedCuisine = $state("");

let cuisines = $derived(
  [...new Set(data.restaurants.map(r => r.cuisine_description))].sort()
);

let filteredRestaurants = $derived(
  data.restaurants.filter(r => {
    if (selectedBorough !== '' && r.boro !== selectedBorough) return false;
    if (selectedCuisine !== '' && r.cuisine_description !== selectedCuisine) return false;
    return true;
  })
);

This filter checks each restaurant against both criteria. If a borough is selected and the restaurant doesn't match, it's excluded. Same for cuisine. If both are empty, everything passes through. If both are set, only restaurants matching both survive.

Add a second dropdown next to the first:

<div class="filters">
  <label for="borough">Borough</label>
  <select id="borough" bind:value={selectedBorough}>
    <option value="">All boroughs</option>
    <option value="Manhattan">Manhattan</option>
    <option value="Brooklyn">Brooklyn</option>
    <option value="Queens">Queens</option>
    <option value="Bronx">Bronx</option>
    <option value="Staten Island">Staten Island</option>
  </select>

  <label for="cuisine">Cuisine</label>
  <select id="cuisine" bind:value={selectedCuisine}>
    <option value="">All cuisines</option>
    {#each cuisines as cuisine}
      <option value={cuisine}>{cuisine}</option>
    {/each}
  </select>
</div>

Notice we're using {#each} again — this time to generate <option> tags from our list of cuisines. The same tool that renders table rows can render any repeating element.

Save and check your browser. Try different combinations — Mexican restaurants in the Bronx, Italian restaurants in Manhattan, pizza places across all boroughs. The table updates with each selection.

Limiting results

After filtering, you may still have hundreds of matching restaurants. Rendering thousands of DOM nodes at once is slow and overwhelming. A simple fix: use JavaScript's .slice() method to cap how many rows the table shows.

Add a new derived variable below filteredRestaurants:

let { data } = $props();
$inspect(data.restaurants);

let selectedBorough = $state("");
let selectedCuisine = $state("");

let cuisines = $derived(
  [...new Set(data.restaurants.map(r => r.cuisine_description))].sort()
);

let filteredRestaurants = $derived(
  data.restaurants.filter(r => {
    if (selectedBorough !== '' && r.boro !== selectedBorough) return false;
    if (selectedCuisine !== '' && r.cuisine_description !== selectedCuisine) return false;
    return true;
  })
);

let displayed = $derived(filteredRestaurants.slice(0, 100));

This creates a new array containing only the first 100 items from the filtered set. Now update the table to use displayed instead of filteredRestaurants:

<RestaurantTable data={displayed} />

Save and check your browser. The table is capped at 100 rows no matter how many records match the current filters. Notice you kept filteredRestaurants pointing to the full filtered set.

We can use that to show readers how many total results exist. Add a short line of text above the table:

<p class="count">Showing {displayed.length} of {filteredRestaurants.length} restaurants</p>
<RestaurantTable data={displayed} />

Now when you filter, the count updates automatically — no extra work needed, because both variables are $derived from filteredRestaurants.

It works, but it's bare. Let's add a little CSS to the bottom of +page.svelte to make the filters look nicer and give the table some breathing room.

<style>
  .filters {
    display: flex;
    flex-wrap: wrap;
    gap: 1rem;
    align-items: flex-end;
    padding: 1rem;
    background: #f8f8f8;
    border: 1px solid #e0e0e0;
    border-radius: 4px;
    margin-bottom: 0.75rem;
  }

  label {
    display: block;
    font-size: 0.75rem;
    font-weight: bold;
    text-transform: uppercase;
    letter-spacing: 0.05em;
    margin-bottom: 0.25rem;
  }

  select {
    display: block;
    padding: 0.375rem 0.5rem;
    font-size: 0.875rem;
    border: 1px solid #ccc;
    border-radius: 3px;
  }

  .count {
    font-size: 0.8rem;
    color: #888;
    margin: 0 0 0.5rem;
  }
</style>

Part 5: Adding a search input

Dropdowns work well when the options are finite and known in advance. But what if a reader wants to find a specific restaurant by name? For that, you need a text input.

The good news: it uses the exact same bind:value pattern you already know. The only new thing is the JavaScript to match text.

Add a state variable for the search query at the stop of +page.svelte:

let { data } = $props();
$inspect(data.restaurants);

let selectedBorough = $state("");
let selectedCuisine = $state("");
let searchQuery = $state("");

let cuisines = $derived(
  [...new Set(data.restaurants.map(r => r.cuisine_description))].sort()
);

let filteredRestaurants = $derived(
  data.restaurants.filter(r => {
    if (selectedBorough !== '' && r.boro !== selectedBorough) return false;
    if (selectedCuisine !== '' && r.cuisine_description !== selectedCuisine) return false;
    return true;
  })
);

let displayed = $derived(filteredRestaurants.slice(0, 100));

Update the filteredRestaurants filter to add a third condition. If searchQuery is set, keep only restaurants whose name contains the query:

let filteredRestaurants = $derived(
  data.restaurants.filter(r => {
    if (selectedBorough !== '' && r.boro !== selectedBorough) return false;
    if (selectedCuisine !== '' && r.cuisine_description !== selectedCuisine) return false;
    if (searchQuery !== '' && !r.dba.toLowerCase().includes(searchQuery.toLowerCase())) return false;
    return true;
  })
);

The .toLowerCase().includes() combination does the matching. Converting both strings to lowercase before comparing means "Pizza", "pizza" and "PIZZA" all return the same results.

Add a text input inside your .filters div, next to the existing dropdowns:

<div class="filters">
  <label for="borough">Borough</label>
  <select id="borough" bind:value={selectedBorough}>
    <option value="">All boroughs</option>
    <option value="Manhattan">Manhattan</option>
    <option value="Brooklyn">Brooklyn</option>
    <option value="Queens">Queens</option>
    <option value="Bronx">Bronx</option>
    <option value="Staten Island">Staten Island</option>
  </select>

  <label for="cuisine">Cuisine</label>
  <select id="cuisine" bind:value={selectedCuisine}>
    <option value="">All cuisines</option>
    {#each cuisines as cuisine}
      <option value={cuisine}>{cuisine}</option>
    {/each}
  </select>

  <div>
    <label for="search">Search by name</label>
    <input id="search" type="text" bind:value={searchQuery} placeholder="e.g. Pizza" />
  </div>
</div>

Add a style rule for the input alongside the existing select styles:

input[type="text"] {
  display: block;
  padding: 0.375rem 0.5rem;
  font-size: 0.875rem;
  border: 1px solid #ccc;
  border-radius: 3px;
}

Save and check your browser. Type "pizza" into the search box and watch the table update as you type. Combine it with a borough filter — "pizza" in Brooklyn — and both conditions apply at once. The count line updates too.

Homework

Task 1: Add a grade filter

Add a third dropdown to filter by letter grade. You should be able to select "A" and see only A-rated restaurants, or "B" to see only B-rated ones, or "All grades" to see everything again. Send me the link to a working GitHub Pages url with the filter.

Task 2: Create a summary dashboard

Above the table, add a row of BigNumber components inside a Dashboard parent component that summarize the current filter results. Your dashboard should show:

• The number that received an A
• The number that received a B
• The number that received a C

These counts should update reactively whenever the user changes any filter — borough, cuisine, search query or grade. Use $derived() to calculate them from your filteredRestaurants array. You'll need to import the BigNumber and Dashboard components we built in Week 3.

I've added them to our class template's shared component library, so you can now import them from a fresh clone of the template like this:

import { BigNumber } from '$lib/components/BigNumber.svelte';
import { Dashboard } from '$lib/components/Dashboard.svelte';

If you don't remember how they work, revisit your code from that week or ask Copilot to remind you. There's a hint in how we calculated the count of displayed restaurants above our table in tonight's lesson.

As with task one, send me the link to a working GitHub Pages url with your dashboard.

Task 3: Pitch a data application

Before you build something, you have to know why it's worth building. This week's assignment is research, not code.

Browse the NYC Open Data portal and find a dataset you think could support a useful application for New York City readers.

Prepare a short pitch for the thing you'd hypothetically build. You'll deliver your idea in two minutes or less at the start of next week's class, so keep it concise. Your pitch should answer:

• What dataset would you use?
• What would the headline of your page be?
• How would you present the data in a way that's interesting?

Don't worry about having to actually make the idea. We're just practicing the process of finding and pitching ideas. The more creative, the better.