How this site checks itself

That nobody typed it. When a page here says a subject appears a certain number of times, that number is produced while the page is being built, by counting the entries that match. It is not a remembered figure, not a figure from a previous version of the page, and not a figure somebody checked once and left in place.

The practical difference shows up when something changes. A typed number stays correct until it silently stops being correct, and nothing announces the moment. A computed one moves when its source moves. If the record grows, every page reflecting it grows too, without anybody editing a sentence.

That is drab plumbing, and it is the entire argument. Most editorial boasts about rigour describe intentions. This one describes a dependency graph, a wiring diagram of obligations.

Abstractions about dependency graphs are easy to assert. Here is one number, followed from the page a reader sees back to the file it rests on.

A subject page states how many archived items concern its topic. The build loads the record from disk, tests each item's headline and summary against a pattern for the subject, tallies the matches, and hands that tally to the template. Nothing in between stores a value anywhere a person could reach in and adjust.

Break it deliberately and the dependence shows. Rename the record file and the import fails, the build aborts, and yesterday's page stays deployed rather than a fresh one arriving with a stale count. Delete half the items and every page touching that subject moves together in the same run, because none holds a figure of its own to fall behind.

The pattern matching is the weak link and deserves saying so plainly. Deciding that an item concerns ransomware by testing its headline for the word is crude — it misses pieces that describe an incident without naming the category, and it catches passing mentions. That imprecision is a property of the classification, not of the arithmetic, and pages leaning on such tallies name the pattern that produced them.

So the guarantee is narrower than it first sounds, and worth stating at its real size. The number shown is exactly what the stated rule produces over the stated record. Whether that rule captures the concept a reader has in mind is a separate question, and one no build step can settle.

14,951 entries so far, collected from 969 named publications since August 7, 2014. It is still being added to, so every figure on this site is a figure as at the last build rather than a final one.

An entry holds a headline, a date, an attribution to the publication that produced it, a set of subject tags and, for 6,634 of them, a written summary. It is a record of what the trade press published, which is a different thing from a record of what happened. The distinction runs through every page built on it and is stated on each one.

Three consequences follow and none is a defect exactly. Coverage volume reflects editorial attention rather than incidence, so a subject can be enormous here because it was interesting to write about. Attribution is to the publication rather than to any primary source, so a claim repeated by forty outlets appears forty times without becoming more true.

The third is subtler and shapes every count on the site. An entry credits a domain, not a masthead, and the gap between those widens once anything is tallied. Syndicated material carries whichever domain republished it, so one wire story travelling through six outlets deposits six separate rows under six separate names. Aggregators sit alongside originators with no mark distinguishing them. Nothing collapses duplicates and nothing traces an item upstream to whoever first reported it.

The distortion is systematic rather than random, which makes it predictable and therefore usable. Heavily syndicated subjects swell; material that stayed with one outlet does not. A count of how often something was written about sits closer to a count of how far it travelled, and those two quantities diverge furthest wherever a story was cheap to reproduce and expensive to originate.

A slower decay affects the far side of every credit. Addresses published a decade ago resolve unevenly today: some redirect into a successor title, some land on a parked domain, some return nothing at all. An entry outlives whatever it points at, because the headline, date and credit are held here rather than fetched on demand. What does not outlive it is the ability to check an item against the original, and for the oldest material that avenue has quietly shut.

Because a number means little without them, and because a compendium that hides its holes invites precisely the over-reading that breeds bad conclusions.

Take the thin months of 2016. Any subject total for that year is depressed by them, and a reader comparing 2016 against 2017 without knowing would conclude that attention rose when the collecting merely resumed. The defect does not make the totals wrong. It makes one particular comparison wrong, which is worse, because the comparison looks sound.

There is a commercial argument too, and it is not in tension with the editorial one. A buyer evaluating this record will find its defects within an afternoon of serious work. Finding them listed is reassuring. Finding them unlisted, after being told the material is rigorous, ends the conversation and deserves to.

The habit generalises. Anything measured here that cannot support a conclusion gets a sentence saying so, and those sentences are on the pages rather than in a note at the end.

The assertion is dropped. No gentler route exists, and that is by design rather than restraint.

The failure mode is a broken deployment, which somebody notices within minutes, rather than a plausible sentence, which nobody notices ever.

This shapes what gets written more than any editorial rule, and the questions it rules out tend to be the appealing ones. Did defensive spending rise after a particular breach? Did firms named in coverage fare worse afterwards than firms absent from it? Did a regulation alter behaviour, or alter only what got written about behaviour? Every one of those needs something this collection lacks — budgets, outcomes, or any observation of the world beyond what was printed about it.

So the writing goes where the evidence sits rather than where the curiosity does. That is a genuine cost, and pretending otherwise would be its own kind of overclaiming: the most interesting questions about this field are mostly not answerable from a pile of headlines, however carefully the pile is counted.

The newest archived item is dated November 30, 2021. Filing lags publication, as it does in any collection, so a stretch of recent weeks always sits beyond what the material can yet answer for.

That trailing edge is easy to read past, because an archive gives no signal at its brink. The most recent weeks look thin in a way that resembles a subject going quiet, and a subject that has only just emerged looks like one that does not exist. Neither reading is safe near the edge, and the distance moves with every addition.

Where the material is genuinely historical the lag costs nothing, because a completed period is what those pages examine. Elsewhere the width of the band even varies — publishers file at different speeds, regulators sit on notices, holidays slow everything — so a recent count deserves lighter weight than an old one until the edge has moved past it.

None was designed in advance. Each exists because a particular kind of error got through first, which is the ordinary way such things accumulate and is worth admitting.

The one comparing a page against its plain-text twin came first, because generating two versions from one source invites them to diverge the moment somebody edits the wrong one. The orphan check arrived the day an index page was created that nothing pointed at: it existed, it was correct, and no reader could have reached it. The check found it within minutes of being written, which settled its usefulness immediately.

The check for words fused together exists because that defect reached a published page unnoticed. A template that closes a link at the end of a line and continues on the next will silently drop the space between them. On screen it is glaring. While writing it is invisible, because the source has a line break exactly where the space should be.

The editorial check grew last and is the fussiest, because thin writing is harder to detect mechanically than a broken link. Counting words catches padding. Measuring how much of the vocabulary repeats catches a page that reaches its length by restating itself. A list of stock phrases catches the register of automatically generated prose. Measuring the spread of sentence lengths catches text where every sentence runs to the same size, which almost nothing written by a person does.

Each of those rules has produced a false alarm and been narrowed in response. The phrase list originally rejected an ordinary English word used correctly, and it now demands the surrounding construction that marks the promotional sense. A check nobody trusts gets switched off, so keeping them precise is a condition of them surviving.

Not a guarantee of accuracy. Seven gates passing means a page is buttressed and structurally sound. It does not mean the conclusion is right.

Not an audit. No external auditor has vetted this. The checks are run by the same build that publishes the pages, which makes them a habit rather than a warranty. Described accurately, they are a system that makes certain mistakes impossible and others merely visible.

Not a claim about the news entries. The archived items carry whatever text their original publisher supplied, attributed to them. The standard described here governs the pages written for this site.

Not finished. Checks get added when a species of error turns out to be possible. The one that catches words glued together by a template exists because that defect reached publication before anybody saw it.