To that end, here’s an even more interesting question around distillation: why exactly is it bad? After all, what are large language models but the distillation of all of the knowledge on the open Internet, scraped by the frontier labs and distilled into the models that are themselves being distilled? Who is exactly being wronged here?

In fact, this paradox is the solution. I believe that open weight models are good for innovation (and, per the above, I think that labs on the frontier will be fine), but it’s a problem to be dependent on China. The U.S. should pass a law that (1) makes explicit that collecting data for training models is fair use, and (2) bars terms of service that forbid distillation, for U.S. companies at a minimum. Stopping distillation — which is literally just querying the API — is nearly impossible; the U.S. should go the other way and lean into a new copyright policy that both indemnifies the labs and also guarantees that what they learned fuels further innovation for everyone else.

Stratechery on Chinese AI models. Good points all around on R&D costs versus COGs, distillation and open source in general.

NYC has been using aerial photography for over 100 years to catalog and measure the city. Thanks to some superb data preservation, you can explore some amazing aerial photography data going back to 1924.

BPD covers NYC's building footprints dataset updated weekly and covering 1M+ structures. Garages make up about a fifth of NYC buildings.

The cost of DNA sequencing has fallen faster than Moore's Law. Since 2001, the National Human Genome Research Institute (NHGRI) has tracked costs at its funded sequencing centers — from $95 million per genome in 2001 to around $500 today.

Genome sequencing costs fell 190,000-fold since 2001. Data here stops at 2023 and I can't seem to find reliable sources since then.

As a new parent, my mind immediately went to movie-viewing volume. With a four-month-old, I am not exactly plugged into movie culture—or really any culture unrelated to said four-month-old. So I wanted to know whether this “parent tax” is universal: Do we watch fewer films as our responsibilities grow?

The answer, unsurprisingly, is yes. In the MovieLens dataset, average rating volume peaks among users ages 25 to 34, then declines with each passing decade.

MovieLens data shows movie taste shifts with age, driven by nostalgia bias via Don't Worry About the Vase