2017 in the archive: when attention and damage came apart

The year opens on January 2, 2017. Within that first month, 17 entries are forecasts: predictions, trend lists and outlooks for the twelve months ahead.

Read now, the interesting thing about them is not that they were wrong. Several were perfectly reasonable, and some of what they described did happen. It is that the three events which came to define 2017 are not the kind of thing a forecast produces. A cache of exploits leaking out of an intelligence agency, a worm assembled from one of them, and a credit bureau losing the records of 143 million people are not trends. They are occurrences, and they have dates.

Forecasting in this field predicts categories: more of this, a rise in that, attackers increasingly doing the other. Categories are usually right, for the unsatisfying reason that the volume of nearly everything rises. What a category cannot supply is which particular week will end with a hospital turning patients away, and that is the information anybody actually planning would want to have.

None of which is an argument against the genre. A trend list read in January can direct a year of attention usefully, and several of these did. It is an argument for reading them as what they are — statements about direction rather than about events — while noticing that events are what a year gets remembered for.

On April 11, 2017 this archive filed an entry about a group publishing exploits taken from a national intelligence agency. It reads, at this distance, like an ordinary item in an ordinary week — one more disclosure in a field that produces them continuously.

34 days later the first entry about the attack that used one of those exploits appears. In between, a patch existed and had been published. The interval is not an inference from outside sources; it is two dated records in the same collection, and anybody reading this archive week by week in 2017 had both of them.

There is a version of this observation that is too pleased with itself, in which the lesson is that people should have listened. The more useful reading is that they did listen, in the sense that the information was published, circulated and filed — and that publishing, circulating and filing is not the same activity as patching several hundred thousand machines belonging to organisations that had reasons, some of them good, for not having done it yet.

The reasons were largely not negligence either. Among the machines still unpatched six weeks later were medical devices whose configuration a manufacturer had certified and a hospital could not lawfully alter, industrial controllers with no maintenance window short of a full shutdown, and desktops running software whose supplier had abandoned it years earlier while the workflow depending on it carried on regardless. Patching is discussed as an administrative act and is frequently a procurement, regulatory and clinical one. That is why the interval between a fix existing and a fix being applied runs to months across most of this industry, and why an exploit published in April was still worth something to somebody in May.

The gap between knowing and having acted is where nearly all of security actually happens, and 2017 produced an unusually clean measurement of it.

In September a credit reference agency disclosed that the personal records of roughly 143 million people had been taken — names, dates of birth, national identity numbers, in many cases addresses. The people affected had not chosen to have a relationship with the company. Their data was there because lenders had supplied it.

That month holds 178 entries, below the monthly average of about 197. The subject appears 8 times across the entire year. October, November and December are the three thinnest months in 2017.

Some of that is a limitation of this collection rather than a fact about the world, and that limitation is set out further down. But the shape is hard to explain away entirely, because it is consistent: by the autumn of 2017 this archive was recording fewer things overall, and the largest breach of personal records anybody had yet seen did not reverse the trend for even one month.

A breach of that kind is difficult to write about repeatedly. There is no technique to explain that has not been explained, no product that would obviously have prevented it, and no action available to the affected person beyond monitoring their own credit indefinitely. Coverage needs somewhere to go next, and this story's next step was litigation and testimony, which is a different beat.

Late in the year an entry records that a ride-hailing company had suffered a breach affecting tens of millions of riders and drivers roughly twelve months earlier, had paid the people responsible to delete what they held and say nothing, and had told neither the affected people nor any regulator. It surfaces 2 times in the whole of 2017.

The concealment did not end through detection. It ended because the company changed hands at the top and the incoming management chose to disclose it, which means the mechanism that brought it to light was internal politics rather than anything security had built.

Set beside the 64 entries about a notification regime, this is the sharpest juxtaposition the year offers. Across the same twelve months in which the field produced a steady output about the duty to report, the most instructive breach available had been deliberately unreported, bought into quiet with a payment presented as a bug bounty.

That label repays a moment's attention. Bounty programmes are a legitimate institution with a real purpose: they give somebody who finds a flaw a lawful, rewarded route to handing it over. Applying the term to money paid to people who had already taken the data, in return for their silence, borrows the standing of the institution to describe the very behaviour it was invented to displace. The vocabulary of a good practice is available to anybody, including to whatever the good practice replaced.

Concealment of this kind is rarely a single decision taken by somebody who understands themselves to be doing wrong. It assembles from smaller judgements that are each arguable: the material was recovered, the people involved were paid and signed something, no harm has appeared, and telling everybody would cost enormous sums and considerable trust over an injury that may never arrive. Twelve months of silence is what those reasonable-sounding steps add up to.

The consequence reaches well past the one company, and it undermines a whole genre of statistic. Every count of breaches, every average detection interval, every claim about how promptly organisations own up, is assembled out of breaches that were disclosed. A concealment that succeeds enters neither the numerator nor the denominator. It is invisible to exactly the measurements people cite when arguing about whether the field is improving.

How large that hidden category might be is unknown, and saying so is the honest position. It is a materially different statement from assuming the category is small, and the difference between those two sentences is where a great deal of confident commentary quietly lives.

188 entries from 2017 mention ransomware in their title or summary. 103 carry the tag for it. The classification is 85 entries behind the text — roughly 45% of what was being written about never reached the label.

The tags that did the work instead are broader and older. Here are the eight most used across the year:

• 209Cyber Attack
• 184Data Breach
• 169Software Security
• 120Malware
• 119Cyber security Survey
• 117Cloud
• 116Identity Theft
• 103Ransomware

Cyber Attack and Data Breach lead, which are categories broad enough to absorb almost anything. A taxonomy is designed before the year it has to describe, and this one was built for a period in which ransomware was one nuisance among several rather than the thing that stopped a health service.

The general point survives the specific example. Classification schemes record the concerns of the moment they were written, and they go on quietly shaping what can be counted long after those concerns have moved. Anybody drawing conclusions from tagged data is reading a description of the past through a vocabulary fixed even earlier.

About 8.7% of the year — 206 entries — carries a survey or report tag. Vendor-published research was already a standing component of security writing rather than an occasional intrusion into it, which is worth knowing before treating any aggregate of this material as a picture of events.

The most frequently mentioned subject of 2017 is not an attack, a breach or a regulation. 242 entries mention the cloud — more than the regulation, the year's worm and its largest breach put together.

That is a loose text match rather than a tagged category, so it catches passing mentions alongside entries genuinely about the subject, and it should be discounted accordingly. Discount it heavily and it is still the largest thing in the year. The internet of things follows at 127, phishing at 70, machine learning at 29, and the difficulty of hiring anybody at 16.

What that describes is a background rather than a story: the slow migration of everything into somebody else's datacentre, which generated a continuous supply of writing for years without ever supplying a date. Events are what an archive remembers, because events have dates and can be filed on the day they happen. Migrations do not, and this collection records them as a hum instead of a spike.

The hum turned out to matter at least as much. The arguments now made about shared suppliers and concentrated failure descend directly from the migration that was quietly underway through 2017, while the reading public was occupied with May.

2,368 entries places 2017 second in this archive rather than first. 2015 holds more, at 2,997. The five years around it run:

• 2,9972015
• 2,2772016
• 2,3682017
• 1,7582018
• 1,6872019

The movement worth noticing is what comes after. 2017 sits slightly above the year before it, and then 2018 falls by about 26%.

A drop that size immediately after the loudest year in the collection is not what anybody would predict from the state of the world, because 2018 was not a quieter year for security. It is where volume stops describing events and starts describing the collection itself — how much was being filed, by whom, and with what attention.

That is worth holding on to before the next section, which asks the same question about every other number on this page.

Not directly, and the distinction matters enough to be stated rather than buried in a footnote. This is a count of what one aggregation filed, week by week, during 2017. It is not a census of security coverage, and it is certainly not a measure of how much of anything occurred.

Three limits are worth naming. The selection reflects whoever was choosing, and their interests were not neutral. The sources are concentrated in trade titles, as the previous section shows. And 907 of the 2,368 entries carry a written summary, so the text searches behind these subject counts run against a richer record for some entries than for others — which means the totals for each subject are floors rather than exact figures.

None of that dissolves the finding, because the comparison is internal. The same selection, the same sources and the same summarising produced 64 entries for a regulation and 8 for the largest breach of personal records of the era. Whatever bias shaped the collection shaped both numbers, and a ratio of that size does not come from an uneven summary field.

What the count does measure, honestly described, is contemporaneous attention: what looked worth keeping to somebody reading the field every week at the time, before anybody knew which of these stories would still be cited a decade later. That is a narrower claim than a picture of the year, and it is more interesting than one, because hindsight is exactly what it lacks.