All 7 entries tagged Security

Whatever is Abnormal is Suspicious

Writing about web page http://arstechnica.com/information-technology/2016/03/china-is-building-a-big-data-plaform-for-precrime/

China is investing heavily in its capacity to monitor and evaluate the attitudes and behaviour of the population. On 14 June 2014 the State Council issued a Notice concerning Issuance of the Planning Outline for the Construction of a Social Credit System. The plan envisages that by 2020 every adult Chinese citizen will have a social credit rating.

In a market economy, a person's credit is based on the record of what you earn, spend, borrow, and repay. You gain credit by demonstrating that you can handle money within the law and by honouring your debts. In China, "social credit" is partly financial, but it's also cultural and political. Social credit is gained, not just by handling money honestly and non-corruptly, but also, on a reasonable interpretation of the official language, by knowing the right people and showing the right attitudes in your social and political behaviour.

In other words, just as you can lose financial credit by breaking money rules, you will lose social credit by knowing the wrong people and saying the wrong things. And these things will interact, so that if you know the wrong people or say the wrong things you will put at risk your ability to borrow and to find responsible employment.

What's it all about? Based on its official motivation, the programme

uses encouragement to keep trust and constraints against breaking trust as incentive mechanisms, and its objective is raising the honest mentality and credit levels of the entire society.

Thus, it's explicit that social credit is an incentive mechanism aimed at behaviour change at the level of the population. Every single adult must must understand the norms that China's ruling communist party sets for personal behaviour in economic, cultural, and political life. Break those norms and you lose trust. Lose trust, and there wil be personal consequences. No one will be beyond the system.

Communist regimes have always aimed to classify their subjects for political reliability, but classification was usually crude and error-prone. Stalin's "usual suspects," (described in my new book One Day We Will Live Without Fear) were anyone who from a non-proletarian background, anyone educated under the old regime, anyone of foreign origin or experience of life abroad, anyone with religious beliefs, and so forth. In Mao's China people were classified into "red" and "black."

What the Chinese authorities have in mind today is a classification that is more sophisticated in every way: multi-dimensional, continuously calibrated, and above all comprehensive.

It's not hard to see the benefit for the party leadership. The party authorizes the norms that you should follow, but enforcing those norms throughout society is an unremitting slog. Through comprehensive "social credit" rating of the population, based on big data, the rulers gain a system that sets up clear incentives for every single citizen to conform in every aspect of their lives. If you have the wrong friends or you're indiscreet on social media, you lose the promotion or you are denied the loan you hoped for. So most people will be persuaded to conform.

Plus, the system will also identify the minority that isn't persuaded, and so resists the official incentives, and it marks them out as security risks.

Recently my attention was grabbed by the technology website Ars Technica discussing China's investments in big data collection such as CCTV:

The authorities are watching for deviations from the norm that might indicate someone is involved in suspicious activity (my emphasis).

I knew I had seen this somewhere before. So I looked for it, and here's what I found:

Our communists should be concerned every day to study and know more deeply processes that are essentially anomalous, that is, incorrect, deviating from the general rule of processes and phenomena, and in a timely way to obtain alerts leading to the exposure of persons intending to carry out hostile actions that can lead to serious consequences (my emphasis again).

This was nearly fifty years ago: on 24 April 1968, Lt. Col. Matulionis, an officer of the Soviet Lithuania KGB, was speaking to a meeting on counter-intelligence priorities of the day. (The documentary record is held on microfilm by the Hoover Institution Library & Archives, where I consulted it.)

Communism in Europe and China had common roots. After that, they went different ways. China today looks very different from the Soviet Union. But in respect of what makes a security risk, China's secret policemen have retained exactly the same idea as the Soviet KGB. An ordered society has normal processes. Good citizens follow those norms. When social norms are disrupted, the result is "anomalous, that is, incorrect."

That's where the secret policeman steps in. What is anomalous is incorrect. It arouses suspicion of a crime, and what is suspicious must be investigated for evidence. Who is behind this, and is the hand of the enemy at work?

Mark Harrison : 11 Mar 2016 08:00 |  Tags: China Communism Russia Security |  Comments (0) |  Close comments |  Report a problem

The Soviet Military–Industrial Complex: New Year Insights from Dexter and Rodionov

Writing about web page http://warwick.ac.uk/vpk/

Today sees a new version of the Dexter-Rodionov guide to The Factories, Research and Design Establishments of the Soviet Defence Industry. This is the sixteenth edition; the very first (in which I was co-author) appeared in January 1999. In that time the datset has grown from just over 2,000 entries to nearly 30,000, and the detail from around 100kb to more than 10Mb.

From the start this was a curiosity-driven project. The Soviet military-industrial complex was veiled in secrecy for decades. In 1992 the former Soviet archives were opened up for independent research. Google's Ngram viewer lets you see how the subject broke out into the light of day. The chart shows the relative frequency of the phrase "советский военно-промышленный комплекс" (Soviet military-industrial complex) in Russian-language publications from 1917 to 2010. A few of these would have occurred in items published in Russian outside the Soviet Union; I suspect that explains the first observations from the 1970s and early 1980s.

What were the factories that made Soviet weapons and military equipment? How many and how important were they? Where were they? When were they built? How specialized were they, and how self-sufficient? We just wanted to know.

My co-author of the time, Nikolai Simonov, was showing me some of the lists of secret ("numbered") defence factories in the 1920s and 1930s that he had found in the archives. I knew that Julian Cooper at Birmingham had his own files. We were soon joined by Keith Dexter, an authority on Soviet aviation. We put together what we had and the result was the first edition of the present guide. If you are at all interested in the history of exactly how and when the Soviet defence industry was made secret, I still recommend that you read Julian Cooper's introduction to this first edition.

Soon after that, Keith drew in Ivan Rodionov, another aviation expert, and so it became the Dexter-Rodionov guide.

What's new in version 16, apart from additional detail? The cover page carries the chart below, which shows the growing number of Soviet enterprises engaged in defence production from 1917 through 1991, distributed among the major production branches.

Here are my takeaways (thanks to Dexter and Rodionov for drawing my attention to some of these):

• The breakneck pace of Stalin's rearmament from the mid-1930s is clearly visible. It culminated in the war, and the first spike which is recorded in 1944).
• Also visible is the more moderate but sustained growth of defence plants after the war, including the rapid surpassing of the wartime peak.
• There is a second spike in the number of defence plants in 1964. This was the year in which Khrushchev was outmanoeuvred and replaced by Brezhnev. It suggests an economic issue in the power struggle: was Khrushchev trying to build up defence production at a pace that others considered to be infeasible?
• The changing composition of the defence sector has two striking aspects. One is the vast growth of radioelectronic establishments. By the end, this sector alone accounted for half of the entire Soviet defence industry.
• The other aspect is the tremendous stability of the traditional sectors: armament, armour, and shipbuilding. It would not come as any surprise to a student of the Soviet economy to learn that they could create new sectors (like the nuclear industry or radioelectronics) but even if they wanted they couldn't close the old ones down.

Finally, the chart shows us that by the end there were just over 5,000 plants engaged in defence production. How many is that? In 1987 (according to the Soviet statistical handbook of that year) there were more than half a million state-owned establishments of all kinds in the Soviet economy. So, we are looking at no more than one per cent of the total, and one per cent does not seem like a lot. The explanation is that most defence plants were relatively large. Their share in the whole economy, measured by capital assets or production, was many times greater than their share in the number of plants.

As for the share of defence production in the whole Soviet economy, we are still a long way from being able to pin that down. For any other country the most obvious way to do it would be to work from the expenditure side, by comparing the size of the Soviet military budget with the size of the economy, as opposed to working from the production side, which raises a lot of complicated issues about plant specialization and intermediate production. Alas, in the Soviet case it is no less of a problem to work from the expenditure side, because Soviet defence expenditures were also highly secret. Here I mean true military expenditures, not the officially published figures which were as phoney as a three-dollar bill. In fact, the real figures were so secret that by the end nobody knew what they were! And i mean nobody, literally; I wrote about it here.

The Soviet military-industrial complex continues to throw up many challenges for historical research. The Dexter-Rodionov guide is a terrific place to start looking for both questions and answers.

Mark Harrison : 01 Jan 2015 06:00 |  Tags: Economics History Russia Security War |  Comments (5) |  Close comments |  Report a problem

Back to the USSR: National Security in the Soviet Economy

Writing about web page http://postnauka.ru/video/30259

In Moscow 18 months ago, I made some short videos for Postnauka, a Russian science and social-science video magazine. One of the videos I made was about the security dimension of the Soviet economy. Under the heading of "security" I had in mind both external (mainly military) security and internal political security. The Postnauka people published it on line in August this summer and I think they forgot to tell me so I noticed it only recently. Anyway here is the interview(in English, just over 16 minutes).

Here's a translation of the Russian-language introduction on the Postnauka web page:

What approaches have been used to study the history of the Soviet economy? Why is it difficult to investigate the various influences on the Soviet economy? University of Warwick Professor Mark Harrison explains how to uncover the hidden connections between the agencies of government in the "Serious Science" project established by the Postnauka team.

At first [after the opening of the archives] researchers looked into just two aspects. One was the actual scale of the Soviet military-industrial complex, which was not the whole economy but still a very important part of it and it affected the whole economy, for example, through mobilization planning. So understanding the scale of Soviet rearmament and the military-industrial complex was one aspect, and the other was the effort to better understand the general context of the "great breakthrough" of the first five year plan of the 1920s and the transformation of the Bolshevik party into Stalin's personal power.

I want to mention here another researcher, Vladimir Kontorovich. He drew attention to the need to listen to what the Bolsheviks actually said. In Western economics there is often neglect of what politicians say because we think of our own politicians and their broken promises; we know that politicians lie, so why read what they write as opposed to looking at the outcomes? Kontorovichhas argued that we ought to take seriously what Lenin and Stalin said about their economic goals. In my view if you consider carefully what they wrote about economics you can see some things that emerged from the Bolshevik strategy and the Soviet system of power.

It's difficult for historians to evaluate the role of the security services in economic policy and decision making partly because we have access to the security archives only for the 1930s. Now this situation is changing because a group of independent states that were Soviet republics have chosen a different political path and some of them have opened thei security archives. These are primarily the Baltic countries — Latvia, Lithuania, and Estonia; Ukraine (to some extent) and Georgia have also opened their archives, so now we can find out how the Soviet security service operated in the [Soviet] borderlands. Based on this, we can try to infer how they worked in the Soviet Union as a whole.

To give an example of the kind of research that is now possible, over the last couple of years Inga Zaksauskiene of the History Faculty of Vilnius University and I have been writing a paper entitled "Counter-Intelligence in a Command Economy." Our paper, bassed on research in the documents of the Lithuania KGB held on microfilm at the Hoover Archive, has just been acceped for publication by the Economic History Review. Here's the abstract:

We provide the first thick description of the counter-intelligence function in a command economy of the Soviet type. Based on documentation from Soviet Lithuania, the paper considers the KGB (secret police) as a market regulator, commissioned to prevent the disclosure of secret government business and forestall the disruption of government plans. Where market regulation in open societies is commonly intended to improve market transparency, competition, and fair treatment of consumers and employees, KGB regulation was designed to enforce secrecy, monopoly, and discrimination. One consequence of KGB regulation of the labour market may have been adverse selection for talent. We argue that the Soviet economy was designed to minimize the costs.

And here is a preprint.

Mark Harrison : 22 Dec 2014 09:09 |  Tags: Economics Politics Russia Security |  Comments (0) |  Close comments |  Report a problem

Soviet Censorship: A Success Story

Writing about web page http://www.voxeu.org/article/costing-secrecy

Yesterday VOX published a short column that I wrote about Costing Secrecy. The teaser is as follows:

Democracy often seems bureaucratic with high ‘transaction costs’, while autocracies seem to get things done at lower cost. This column discusses historical research that refutes this. It finds empirical support from Soviet archives for a political security/usability tradeoff. Regimes that are secure from public scrutiny tend to be more costly to operate.

A starting point of my column was that communist rule in the Soviet Union gave rise to one of the most secretive systems of government that has ever been devised. I'm always looking out for ways to illustrate this, and I found a new way recently with the help of Google's Ngram Viewer(thanks to Jamie Harrison). The Ngram Viewer searches the Google Books corpus for words and word combinations and shows their changing frequency over time. The chart below shows the result of searching in the Russian corpus for the word "Главлит" (Glavlit).

Glavlit, the Soviet Union's Chief Administration for Affairs of Literature and Art, was created in 1922 to centralize the censorshop of the media. The background is that the Bolsheviks introduced censorship in November 1917 as one of the first acts of the Revolution (the "Decree on the Press"). During the Civil War that followed, they operated censorship through many agencies at many levels. Glavlit pulled it all together into a single, unified agency. The official title of Glavlit changed a few times over the next 70 years. Still, no one ever called it anything but "Glavlit," even in official government and party documents.

My current research is on secrecy. Censorship and secrecy are not the same. But they are closely connected. Enforcing government secrecy was one of the most important functions of censorship. In addition, Glavlit was a government agency, and its working arrangements were entirely secret, so the censorship had to censor the facts of its own operations.

How effective was Soviet censorship? The frequency with which the chief agency of censorship was mentioned in published works offers a simple measure in one dimension. Here it is:

^{Notes: My guess is that the Google Books Russian corpus must include books published in the Russian language abroad, out of reach of the Soviet censor, as well as within the Soviet Union. For transparency the chart is completely unsmoothed. In the years of the Civil War (1918 to 1920) and World War II (1941 to 1945) fewer books were published, making observations in those years more susceptible to the law of small numbers. You can view and play with the chart here in its home setting.}

There is a simple message. The Soviet censorship agency was openly acknowledged and discussed at the time of its establishment and for a few years afterwards. From the mid-1920s it faded rapidly from sight. By 1931, when Stalin was fully in charge, its disappearance was almost total. For more than half a century Glavlit successfully covered its own tracks. Fifty-six years later, in 1987, Gorbachev launched his policy of "glasnost" (openness). Only then did Glavlit gradually come back into uncensored view. Glavlit was finally abolished in 1991.

In short: Soviet censorship worked.

Mark Harrison : 16 Jan 2014 09:37 |  Tags: Communism Economics History Politics Security |  Comments (0) |  Close comments |  Report a problem

What is the True Mission of the NSA in a Free Society?

Writing about web page http://www.dnaindia.com/analysis/column-what-is-the-true-mission-of-the-nsa-in-a-free-society-1944437

^{This item first appeared under the same title on DNA India on 3 January 2014.}

In the name of counter-terrorism our phone and Internet communications are today under continual government surveillance. Should we worry about the US National Security Agency and Britain’s GCHQ? Yes: Ordinary people have a right to privacy and few means to resist covert surveillance. The privacy of hundreds of millions of people is at risk. Against that, a relatively small number of ordinary people have secret plans that threaten our security. Compared to the threat, the indiscriminate character of surveillance seems disproportionate.

But the critics of surveillance also need a sense of proportion. Many commentators have suggested that more than privacy is at stake. Our liberty is also at risk, they fear: perhaps we are moving towards a police state. Really? A standard of comparison is needed, one that would be best provided by the historical records of a real totalitarian police state.

The Soviet Union was such a police state. In the Soviet Union under Communist rule, the secret police was the KGB (Committee on State Security). Most KGB records remain under lock and key, because Russia today is governed by an ex-KGB elite that has no interest in letting the world see how the KGB upheld Communist rule. A few of the former Soviet states have made a clean break with the Communist past and have opened up their KGB archives. For the last five years I have been working with records from now-independent Lithuania — held on microfilm at the Hoover Institution in California. These are highly revealing about KGB methods of mass surveillance and intervention.

Soviet society was organized to make surveillance easy. Every citizen had an ID card; everyone’s residence was registered with the police. At work, everyone was employed by the State or by government-controlled “collectives”. At home, everyone was a tenant of the government or some collective. The government and the ruling party ran the press and TV; there were no independent media, no independent access to copying or print services, and absolutely no Facebook or Twitter.

With one State postal and telephone service, any letter or call could be intercepted. The KGB ran a network of informers, which was concentrated on key offices, factories, and colleges where young people gathered. The extent of secrecy and surveillance was never debated in any public forum.

No one could leave the country without permission, and the small numbers allowed in and out were basically limited by the KGB’s capacity to watch them individually or in groups. In the 1970s, for example, Soviet Lithuania sent at least 1,000 visitors abroad each year and received at least 10,000 visitors. Forty years later, freed from Communist rule Lithuania would receive more than one million visitors each year just from the European Union. By the standards of a middle-income country today, Soviet citizens were almost unbelievably isolated. Just as important, the cause of their isolation was the Communist regime’s resolve to keep the citizens under continual observation. The first lesson seems to be that a police state will restrict citizens’ travel and communication to a level that it can observe. As humans we love to move around and be in constant touch with each other through social media. In open societies our intelligence agencies respond to this challenge by gathering our data indiscriminately and hoarding it in vast quantities. But they do not compel us to live or work only where they can watch us, and they do not try to prohibit us from communicating through channels they cannot overhear or from travelling to where they cannot see us. On this criterion we are still far from a police state.

After surveillance comes intervention. Intelligence agencies don’t do surveillance for its own sake; they want information on which they can act. Another important difference between us and them is what the authorities do with the products of surveillance. On the basis of the information it received, the KGB intervened directly in the lives of citizens to nudge their behaviour and limit their choices. Suppose they heard that Ivan Ivanovich was behaving suspiciously or voicing undesirable views. The response, at a minimum, was to call Ivan in for an unpleasant and frightening warning. Ivan’s card would also be marked for the future. No Soviet citizen could be promoted to any management position or allowed to travel to any foreign destination without KGB clearance, and Ivan’s chances of either of these were now greatly reduced.

At the moment we have no clear evidence that any of the NSA’s programmes has impinged on the life of any citizen in this way. Nor is it clear how they might do so, other than in the form of private abuse. Again, we seem to be a long way from the working of a real police state.

Still, is there something to worry about? Absolutely. Just as there is no clear evidence that Western intelligence surveillance is taking us into an Orwellian nightmare, there is also a lack of evidence that it is effective at doing what it is supposed to do: combat terrorism and strategic threats to Western security. Western security establishments look overfunded and undermanaged. Potentially, vast resources are being wasted to promote the careers of security empire builders. That should be of huge concern.

What is the true mission of national security in a free society? Surely it is to protect the democracy that allows us (as voters) to toss out the government, to protect our freedom (as private persons) and to be the people we want to be. A question then is: What do we want to be, or how do we want to live? Intrusive mass surveillance in the hands of a bloated security apparatus seems unlikely to protect democracy or freedom. If we seriously want to protect free speech and free association, we should set limits on surveillance and accept some risk that a few bad people will successfully exploit free speech and free association to do bad things to some of us. It’s a tough one, especially for politicians who do not want another 9/11 on their watch.

Mark Harrison : 03 Jan 2014 08:38 |  Tags: Communism Politics Security |  Comments (0) |  Close comments |  Report a problem

Anyway, What's National Security For?

Writing about web page http://isc.independent.gov.uk/files/20131107_ISC_uncorrected_transcript.pdf

Yesterday the House of Commons Intelligence and Security Committee met the chiefs of Britain’s three intelligence agencies. Among other matters, they debated the price we should be willing to pay for national security. I was interested by how this quickly turned into a debate over the meaning of national security itself. There were unexpected differences among legislators and spies; the spies themselves did not speak with one voice. (Here's the uncorrected transcript.)

Hazel Blears, a Labour MP and former local government minister, who is also an ISC committee member, offered up the conventional formula that might be most appealing to an economist:

I wonder if you would agree that in order to have the trust and confidence of the nation, which provides a strong platform for your work, that it is important that we again look at the balance between privacy and security.

She was saying, in other words, that privacy and security are competing objectives of government, and we have to balance them, or trade one against the other. The slope of the "trade-off" is then the price. If we want more security we may end up with less privacy, so the price of security is the amount of privacy foregone. Do we have the right balance? Or, are we paying too much for security in lost privacy? It’s hard to say; we’ll come back to that.

Here’s what was said by Sir Iain Lobban (GCHQ):

I believe a government's first duty is to protect its people. Some ways that it does that I think are necessarily secret. I don't think "secret" means "unaccountable" in any sense, and I think the Foreign Secretary, certainly appointed by an elected government, authorises our operations. There is a Parliamentary Committee which gives us plenty of oversight. There is also the two Commissioners, the Interception of Communications Commissioner, the Intelligence Services Commissioner, who the Chief mentioned earlier.

In these words Lobban said something different from Blears. For him, government has a list of priorities. Security (meaning “to protect its people”) comes first. Everything else comes after. In this perspective there is no balance (or “trade-off”). First, achieve security; privacy comes after. Where it comes (second, third, fourth, etc.) is up to the government and the scrutineers. In case you might think I'm overinterpreting, Lobban went on later to say exactly this:

I don't particularly like talking about the privacy and security balance because I think it is a false choice. I think our job is to provide intelligence around security which enables security in a way which safeguards privacy to the maximum extent possible.

In other words, you can have as much privacy as is left to be had -- after you have ticked security off the list, and security comes first. I don't want to make this sound too bad. Lobban also said other things that, if you believe them (and I have no particular reason not to) are quite reassuring, for example:

[GCHQ] can only look at the content of communications where there are very specific legal thresholds and requirements which have been met. So that is the reality. We don't want to delve into innocent e-mails and phonecalls. I feel I have to say this: I don't employ the type of people who would do. My people are motivated by saving the lives of British forces on the battle field, they are motivated by fighting terrorists/serious criminals, by meeting that foreign intelligence mission as well. If they were asked to snoop, I would not have the workforce. They would leave the building.

Now, here’s the contribution of Andrew Parker (MI5):

I think fundamentally, the raison d'etre of an organisation like MI5 is to protect the sort of country we live in against threats to it. The sort of country we live in is a free society, a democracy, a country where we do prize our individual liberty and privacy. Those values are extremely important to all of the men and women who work in our Agencies, who are members of the public, who live in communities and don't want to live in a surveillance society or a North Korea. They want to live in a country like this. Our job is to keep it that way.

Here Parker took a third line, different from that of either Blears or Lobban. In his view the purpose of security is not to protect persons, or even the people (as Lobban had it) and the price of security is not privacy (as Blears said). Rather, the aim should be to secure “a free society.” Because privacy is one of the characteristics of a free society, he implied, security and privacy are not in conflict; security that infringes on privacy is not security.

To repeat, for Lobban, security and privacy are not in conflict because security comes first. To Parker, security and privacy are not in conflict because privacy is part of a free society and a free society is what must be secured.

Of these three views I have most sympathy, by far, with the third – the "Parker view" that the ultimate mission of national security is to protect the institutions of a free society and democracy. In too many countries the mission of national security has been to protect the incumbent government and repress dissent. Consider the things that distinguish our own society from the settings in which the KGB or Gestapo held sway. Aren’t the most important of these the democracy that allows us (as voters) to toss out the government if we wish and our freedom (as private persons) to be the people we want, say what we believe, and associate with whom we choose?

But this is only the beginning of the problem. Intrusive mass surveillance in the hands of a bloated security apparatus seems unlikely to protect democracy or freedom. If we seriously want to protect free speech and free association, we should uphold limits on surveillance. Inevitably, then, we will incur some risk that a few bad people will successfully exploit free speech and free association to do bad things to some of us. So there is still a trade-off here, but the balance we have to strike as a society is not between security and privacy. It is between two kinds of security: the security of our freedoms and of our physical persons.

Hazel Blears captured this difficult point quite nicely:

You [the intelligence agencies] are currently under some criticism for knowing too much. If there is a terrorist incident, no doubt you will be under criticism for knowing too little. It is a rock and a hard place.

In other words it is questionable whether the mission of national security as safeguarding our way of life, not our persons, is politically viable in the long run, when all the bad (and good) luck has come in. It’s easy to agree beforehand that we should tolerate a few risks. It’s much harder to maintain that after the event, when lives have been lost as a result. At the very least, clear leadership is required. That’s a tough one, especially for politicians and security chiefs who do not want another 9/11 or 7/7 on their watch. In other words freedom carries risks, and may call for a little courage from time to time.

Mark Harrison : 08 Nov 2013 15:31 |  Tags: Politics Security |  Comments (0) |  Close comments |  Report a problem

Needles in the Mega–Haystack: NSA versus KGB

Writing about web page http://www.bbc.co.uk/news/technology-22811580

Widespread concerns about mass surveillance in Western societies have been triggered by two revelations in The Guardian: a court order of the U.S. Foreign Intelligence Surveillance Court giving the FBI unlimited access to the call logs of the Verizon telephone network; and details of the Prism program that gives the U.S. National Security Agency – and maybe others, such as our own GCHQ, access to servers through which foreign communications pass.

Natural questions arise. Are our liberties at risk, along with our privacy? Are we moving in the wrong direction along the spectrum that runs from a free and democratic society to a totalitarian police state?

To help answer such questions, it would seem only sensible to ask how surveillance works in real totalitarian police states. The answer might give us a reality check. That comparison is what I’m going to offer. I’m going to point out some important similarities between what the U.S National Security Agency (and others) are up to and the functions of the secret police under communist rule. I’m also going to show some differences. My conclusion is going to be that we are a long, long way from mass surveillance in the style of the Soviet KGB or China’s Public Security Bureau. But that should not be completely reassuring.

Here are the similarities that look important to me:

• Mass surveillance

American counter-intelligence is in the business of mass surveillance. They’re looking at everyone. Jeremy Bash, chief of staff to former CIA director and defense secretary Leon Panetta, is quoted in the New York Times as saying:

If you’re looking for a needle in the haystack, you need a haystack.

That haystack is the millions and billions of bits of our data that are being gathered. Mass surveillance was also the business of the KGB, as it is the business of the secret police under any dictator. In fact, counter-intelligence everywhere has an unquenchable thirst for personal facts. Every secret policeman knows that the most dangerous enemy is the one you don’t have on file. You can keep tabs on the ones already in the Rolodex – but what about the sleepers, the new recruits, the ones that are out there and completely invisible to you? It’s what you don’t know that can kill you. So, in the interests of staying alive you can never know enough.

• Detection relies on big data

How do you find the enemy you don’t know? By using data and looking for patterns in the data. This is what the KGB did. They looked for several kinds of patterns. They were pioneers of profiling, for example. They figured that many disloyal people had markers in common, although exactly what mattered changed from one period to another. In one period it was your social origins – upper class (which meant the regime had taken your property) or poor. In other periods it was whether you had family members that had fled abroad, or you spoke a foreign language, or you had stayed behind when the war came and tried to live quietly under German occupation. So, the KGB looked for people with those markers. Another thing the KGB looked for was who knew whom or was related to whom. When they put a person under surveillance, they obsessively tracked friends and family members, telephone callers, letter writers, and so on. A third thing was just to look for unusual patterns of activity in the street and at work. To know what was unusual, they had first to know what was usual, and this in itself required data collection on a massive scale. The abnormal would stand out only against the normal. Qualitatively, this isn’t different from what the FBI or the NSA are doing. They too are mainly just looking for anomalies, or patterns of interest in the data.

• The goal is prevention

The ultimate goal of surveillance is prevention. Exactly what is being prevented may vary. Most western intelligence agencies today are trying to prevent another 9/11 or its London equivalent, another 7/7. They are also trying to prevent the public from finding out exactly how they are doing this, because that knowledge might help their targets to pass under the radar. China’s Public Security Bureau has a wider set of goals: to prevent public disorder, to prevent open criticism of China’s leaders and political order, and to prevent everyone from getting the idea that open opposition could ever be normal and go unpunished. The KGB’s goals were pretty similar. To do any of these things you have to be ready to react instantly to signals that something is up. Sometimes you receive a signal, and you can wait and see how it develops. Sometimes you have to react and nip it in the bud even before you know what it is that “it” might be. To prevent the bad stuff you have to review all situations that look as if they have a potential for going bad, and consider all people that look as if they have a potential to become enemies. Identifying the potential enemies is always and everywhere a judgement call.

• Risk of type I errors.

So much in this line of work is a judgement call that errors are inevitable. Some are what statisticians would call Type I errors and some are of the opposite type – Type II. You make a Type I error when you see a pattern in randomness, so for example a person has a random resemblance to a terrorist by having the wrong appearance and being in the wrong place at the wrong time, and suddenly you’ve got them on a plane to Guantanamo Bay. And then a Type II error is when you miss a pattern, or overlook a real spy or terrorist. To explain this another way, when you’re looking for a needle in a haystack, and it’s important to avoid missing it, it’s inevitable that you will turn up lots of things that might be needles because they look quite like needles and in fact you might have even stuck one in the pin cushion before you realized that it’s just a shiny thorn … and now you can’t be bothered to retrieve it. Yes, and that means that where there is scope for error there is also scope for abuse, because secret policemen are not all dedicated professionals; among them will be those that are too lazy, or too ambitious, or too much in love with power to correct a mistake. In most situations Western societies show a preference for Type II errors over Type I errors; we’d rather leave a criminal at liberty than imprison an innocent person. That’s not so hard when we’re talking about shoplifting; it’s harder by orders of magnitude when the criminal at liberty has the potential to behead a bystander or fly a passenger jet into a shopping mall.

Those are the ways in which western counter-intelligence looks very much the same as counter-intelligence under totalitarian rule. But there are also some key differences. Here they are:

• Law governed and openly contested

Most obvious is the existence of a legal framework. It was not always like this but in both Britain and America the intelligence services now operate within the law, subject to both legislative and judicial oversight. The law permits some things and not others. The NSA can find out that X sent an email to Y, but it can’t read your email without a court order that names you and convinces a judge of probable cause. This framework may well look unsatisfactory, and may indeed be unsatisfactory; I’m not a lawyer and don’t pretend to know. At the same time, we also have a free press and intrepid journalists that have strong incentives to find scoops and dig out scandals. As a result, the scope of secrecy and surveillance is law-governed (although imperfectly), open to free discussion (to the extent that we know of it), and contested (vigorously and continually). If you don’t like the law you can take the contest to the polls, and do the hopey-changey thing of tossing out the law makers. Or you can take a personal stand, break the law, and answer for it in the courts like Bradley Manning (although this does not seem to be the path preferred by Julian Assange or Edward Snowden). The contrast with the situation in countries under communist rule could not be more stark. There the KGB responded only to the instructions of the ruling party (and the same no doubt holds in China, Cuba, North Korea, and North Vietnam); there was and is no answerability to the parliament, the courts, or the press. What is more, the merest mention of secrecy and surveillance was completely suppressed; the existence of secrets was a well policed secret.

• A much bigger haystack

America’s haystack is of unimaginably vast dimensions. It’s so big that, according to Edward Luce in the Financial Times, it employs a data-intelligence complex with a staff of nearly a million and a budget of $80 billion. The KGB’s haystack was pretty large in its time. It was put together from many individual straws: agent reports of gossip from canteen queues and student dormitories, surveillance reports, information gathered from microphones, phone taps, opening the mail, and so forth. In 40 years the archive of KGB counter-intelligence in Soviet Lithuania (a country of around 3 million people) accumulated at least a million pages of documents. On that basis, the total paperwork of the entire Soviet KGB archive (for 70 years and a country of 200 million people and more) ought to exceed that of Soviet Lithuania by at least two orders of magnitude. And this was in a society with one landline system and one mail service, without networked computers or mobile phones, where no one even had free access to a photocopier. When even intercity phone calls had to be booked through an operator in a city exchange, it was relatively easy for the KGB to monitor anyone’s personal network. So the size of America’s haystack must be thousands of times larger than this, and probably tens or hundreds of times larger than even China’s haystack. This observation, at first alarming, is testimony to the fact that we live in a free society in which communication is unfettered and of negligible cost by historical standards. We, the citizens, are the ones that make the haystack so large by our abundant use of the freedom to communicate.

• Many fewer needles

The problem of finding needles in this vast haystack is magnified by the fact that western societies do not appear systematically to produce needles – certainly not on the scale of more repressive societies. As the sociologists Inkeles and Bauer (in The Soviet Citizen, 1959) reported from the first wave of the Harvard Interview Project, the Soviet system of repression was apparently based on the assumption that everyone had a reason to hold a grudge against the communist rulers somewhere in their past. A parent had lost property, a brother had been arrested, a husband shot, a cousin’s family resettled in the remote interior. As time passed the salience of such historical events might recede, yet for some reason each new generation of Soviet-educated citizens kept on throwing up new kinds of nonconformity and outright disloyalty that had to be monitored and checked. In contrast western societies are not governed by dictators that have systematically expropriated property and penalized wide social classes and ethnic groups; they also provide multiple channels for citizens to express discontent and resentment and organize for social and political change. Despite this, there are still needles: enemies of openness and tolerance. But they are far fewer in number than the hostile forces that repressive regimes cannot help but produce and reproduce continually.

• More type I errors.

You put a much bigger haystack together with far fewer needles and the implication is unmistakeable. When the haystack is small and needles are many, the chances of making Type I errors are reduced. Under communist rule, if it pricked like a needle and it looked like a needle, there was at least a good chance that it was a needle. Any western intelligence agency trying to find those few needles in today’s mega-haystack has a much reduced chance of coming up with real needles compared with their communist counterpart, and a correspondingly heightened chance of false positives. The fact that so many people are looking for the few needles, that the number of big data analysts must exceed the probable number of real terrorists by a factor of one hundred or even ten thousand, just makes it much, much worse. So you want to make a career as an analyst. How can you distinguish yourself if you never identify a threat? How can you fend off boredom if you never reach the point of saying: “This is someone we should look at more closely”? So you do it, and you make a mistake. Well, it was worth looking into. And that is most unfortunate, because as a society we want to live in safety but we also hate Type I errors. We intensely dislike the idea that an incidental bystander might get investigated, or even detained, because of an intelligence error. So intelligence errors sow cynicism and mistrust.

Now I’ll summarize. NSA versus KGB: Is there good or bad news in the comparison? To me the news looks mostly good. Compared with the KGB, the NSA looks quite benign. But there is also a warning. The warning flows from the observation that there is no limit on what our guardians would like to know about us. The more they know, the better informed they are. But the more resources they have, the greater is the scope for over-ambition, the abuse of power, and the false positives that we rightly fear. How much is enough? The purpose of national security is not to suffocate us with cotton wool. It is to enable us to be the people we would like be and to protect the rule of law that we would like to have. In a free, open society the limits of security are something we, the citizens, should always debate, contest, and, if necessary, push back.

About me: I've spent much of the past five years working with archives of the KGB of Soviet Lithuania held at the Hoover Institution Archive. This work is in a paper I have coming out soon in the Journal of Economic History and in other work in progress or under review.

Mark Harrison : 11 Jun 2013 14:15 |  Tags: Kgb Politics Russia Security |  Comments (0) |  Close comments |  Report a problem