The Five Data Files Behind Our Terms Page: 12 MB of JSON, Five ETags and Five 304s

I work on OnePassApps, so this is our own site, and every figure below was measured from outside the browser with plain HTTPS requests on 8 November 2026. No session, no cookies, no JavaScript. Our terms page points at five static files that nothing else on the site links to: four JSON datasets and one Markdown README. They sit on the same origin as the apps, they are readable by anyone, and each one carries an ETag and answers a conditional request with a real 304 Not Modified - while every one of our 56 HTML pages carries no validator at all. We wrote about the three text files a crawler reads on 7 November: /robots.txt, /llms.txt and /sitemap.xml all answer a conditional request with a real 304. This is the layer nobody had measured: the five data files behind the terms page. Only one page links them I fetched all 56 URLs in our own sitemap.xml and collected every href on every page. Exactly one page links any of the five files: /terms. Not the home page, not the app pages, not the guide pages, not the seven Studio shells. That is a deliberate shape rather than an accident of crawling. The terms page is where the licences and the attribution live, so the credits are linked from the legal document and nowhere else. The link labels are the ones a reader actually sees: "Provider credits and permission links", "bundled Wikipedia selection", "download the image credits", "bundled rating selection" and "Local speech model credits and software licences". The five files File Bytes Records Gzipped ETag Conditional /movie-data/provider-media.json 10,188,207 7,839 852,744 f768848ba7b6... 304 /movie-data/descriptions.json 1,463,297 646 404,285 5853ad82602c... 304 /movie-data/ratings.json 344,862 2 inputs 26,001 1d85c076db1f... 304 /movie-data/images.json 16,195 15 2,974 fb4110bee954... 304 /third-party/newsflow-speech/README.md 5,983 - 2,728 3edf9426e434... 304 Together that is 12,018,544 bytes uncompressed and 1,288,732 bytes over the wire with compression on. The provider-media file alone is 10,188,207 bytes, and 85% of the whole set. The full validators, if you want to check them yourself: 5853ad82602c2b2f611f41ab3722b280 for descriptions, f768848ba7b6b0e8fe56368929416acd for provider-media, 1d85c076db1fcd4c19e2d32ff9170a49 for ratings, fb4110bee9548c84b46fa69f4f38c9cf for images and 3edf9426e43430f47aba2994926d6e2e for the README. The validator layer is the opposite of the pages Every HTML page we serve carries cache-control: public, max-age=0, must-revalidate and no ETag and no Last-Modified, so a conditional request can never succeed and a repeat visit re-downloads the document. We measured that on all 56 URLs. The five data files behave the other way round. Each one carries a quoted 32-character hex ETag, each one answers If-None-Match with a real 304 and no body at all, and the ETag does not change when a query string is added - /movie-data/ratings.json?utm_source=devto returns the same 1d85c076db1fcd4c19e2d32ff9170a49 as the bare path, with the same 344,862 bytes. So the site has two validator regimes, and they split along a line that is not "pages versus files". The 3 plain-text files a crawler reads and these 5 data files both carry validators and both answer 304; the 56 HTML pages carry none. Our earlier post measured the cache tiers - the 56 pages with no validator and the 3 text files with one. The five data files are a group that post did not name. The ETag is not a checksum of what you receive. We hashed the bytes the server sent us, both uncompressed and compressed, and none of the five ETag values matches the MD5 of either. The validator is an opaque fingerprint the origin keeps; you can compare it, but you cannot recompute it from the payload. What is inside each file provider-media.json is the big one: 7,839 media records, every one of them carrying an imageUrl, across 23 fields. The provider split is kinocheck at 6,516 and tvmaze at 1,323. Its own header says sourceCount: 121,750 - the number of source rows considered when the file was built, against 7,839 rows kept. descriptions.json holds 646 Wikipedia plot selections, every one carrying a wikidataId. The file states its own licence as CC-BY-SA-4.0 and the timestamp it was fetched: 2026-10-06T10:03:47.422Z. images.json is the small curated set: 15 images, broken down as cast-photo 9, poster 5 and still 1. ratings.json carries the community-vote provenance rather than the votes themselves. Its version is omdb-org-community-v1, its snapshotAt is 2026-10-06T10:13:30.003060Z, and it names the two source downloads with their SHA-256 sums so the snapshot can be checked against upstream. It also carries its own catalogueCount of 652, which is 6 more than the 646 entries in the descriptions file - two different counters, measured from two different files, and we have not reconciled them. The README is plain Markdown: 5,983 bytes explaining which speech model and voice the local narration uses and what the generated audio does and does not contain. The robots layer says the opposite of the headers Each of the five files carries x-robots-tag: noindex, so a crawler that follows a link is told not to index it. Our robots.txt does not block the path - it disallows /api, /account, /sign-in and /sign-up, and nothing else. So the two layers disagree on purpose: the directory is open to crawlers that follow a link from the terms page, and each file asks not to be indexed on its own. That is the right shape for a licensed dataset you want readable and attributable but not competing with your own pages in search. Our llms.txt does not name any of the five. Limits Every figure is one fetch per URL on 8 November 2026, not a sample over time. The ETag values are what the origin served today; they will change whenever the file changes. The record counts are what each file contains today; the files are rebuilt periodically. I did not fetch the two upstream omdb.org downloads to check their SHA-256 sums against upstream - I only read the sums the file publishes. I did not attempt to read the app's private movie data or anything behind a sign-in. If you want to see the same thing yourself, fetch https://app.onepassapps.com/movie-data/ratings.json with a browser User-Agent and look at the ETag header, then send that value back as If-None-Match and you will get a 304 with an empty body. The legal page that links all five is our terms of use, and the app the ratings describe is MovieMatcherAI. I work on OnePassApps. Everything above is measured on our own site, and I have tried to be exact about which parts are the server's own statements and which parts are our measurements.