"X Framework" Debug Console
Other results for searched terms
Terms Of Service
Search engine revolutionized
Collection management policy
1641 x 1031
66 x 24
1200 x 675
1000 x 1000
812 x 1225
66 x 24
460 x 276
620 x 413
Innovation management measurement
Innovation management measurement helps companies in understanding the current status of their innovation capabilities and practices.
Biometric data and the Taliban: What are the risks?...
The New Humanitarian
1 month ago
From ethical dilemmas on data security to worst-case scenarios unfolding in real time – the Taliban’s rise to power in Afghanistan is spurring urgent concern about the safety of data that aid groups have collected over 20 years. Data protection experts warn that aid groups must quickly review and safeguard sensitive information on Afghans who have received emergency relief and other services. Humanitarian agencies are among those that have tracked, stored, and shared data linked to millions of Afghans – including precise biometric data like fingerprints or iris scans. Crucially, some of this data has been collected by the now-deposed Afghan government – raising concern that the Taliban have inherited databases and technology that could be used to identify people linked to previous regimes or international forces, or members of persecuted groups who have received aid. “The Taliban have been given the keys to the server room, so to speak,” said Karl Steinacker, a former official with the UN’s refugee agency, UNHCR. He now advises civil society organisations on digital identity. The New Humanitarian spoke with Steinacker and Katja Lindskov Jacobsen, a senior researcher at the University of Copenhagen’s Centre for Military Studies, to unpack the issues. In the interview, parts of which are excerpted below, they discussed the potential risks, why aid groups collect so much data in the first place, and the right to be forgotten. It’s unclear exactly how much data aid agencies have collected and shared over the years, or what the Taliban have access to now, which underscores the need for a swift review, Steinacker said. But aid groups or international donors have had their hands in an enormous range of data through two decades of programming: registration for millions who received food aid or mobile cash transfers; digitised government identity cards linked to biometric data; or iris scans for refugees in neighbouring Pakistan, for example. UNHCR did not respond to a request for comment. Other agencies, including the migration agency (IOM) and the World Food Programme, said they were not able to respond to questions before publication. READ MORE: Who has data? Aid agencies, militaries, and the government have collected a wide range of data in Afghanistan, and even determining who has what can be complicated, experts say. More than two million Afghans have been issued digital versions of national ID cards, known as Tazkira, Steinacker said. These IDs are linked to a biometric database in an interior ministry programme supported by IOM. The agency also helped the Ministry of Refugees and Repatriation with plans for a database tracking the locations of returnees and the internally displaced. The World Food Programme has more than six million people in Afghanistan registered on its beneficiary management programme for food and cash transfers, the agency said in its most recent annual report. A 2017 agency-wide internal audit found a range of problems with the programme, known as SCOPE, from unprotected data to a lack of “formal beneficiary consent”. The annual report for Afghanistan said the agency also provided technical expertise on “beneficiary registration, including biometric information and transfer management”, to a government ministry. Data is also routinely shared among UN agencies and NGOs. The WFP shared SCOPE beneficiary data with UNHCR, for example, while both agencies and UNICEF have a three-way data-sharing agreement on cash aid. Data for people who receive cash transfers may also be stored by banks or mobile phone operators. Anonymous data is no guarantee of privacy. People can be identified by combining answers to different questions, according to protocols on data-sharing used by aid groups in Afghanistan. Even simpler forms of data are a risk: Before leaving Kabul, British embassy staff left behind CVs that compromised several stranded Afghans, The Times reported. Three families were later evacuated. The WFP said it received “informed consent from all beneficiaries” to collect and store data. This may have been particularly difficult in some areas: The Taliban refused to allow biometric registration in areas they controlled, the annual report stated. The Taliban have promised an amnesty, and a spokesperson said there is no “hit list”. But rights groups already report reprisal killings and threats. In early August, the Taliban seized US military biometric devices that might help uncover people who worked with international forces, The Intercept reported. And when Kabul fell, Taliban soldiers searched for files at the national intelligence agency and the communications ministry, The New York Times reported. Today’s risks should underscore wider data privacy questions for the entire aid sector, which has often embraced the benefits of digitised records and biometrics while overlooking the dangers, according to researchers who study data security in aid settings. How long is data stored? Who else has access? Is there adequate consent from people receiving aid – often newly displaced with few other options? Are policies future-proofed to protect against unforeseen risks? The Taliban’s rapid takeover in Afghanistan has brought these and other questions to the forefront again. In June, a Human Rights Watch investigation detailed how biometric data UNHCR collected from Rohingya refugees was shared with the country they fled, Myanmar. “With biometrics, the concern is, you can take a new name, but you can’t really take a new iris,” said Jacobsen, whose research often focuses on humanitarian interventions and technology, including biometrics. Her 2015 study highlighted potential flaws with a first-of-its-kind UNHCR biometrics programme for Afghan refugees in Pakistan. The system used iris scans – stored anonymously – to determine whether returning refugees had already received aid. Jacobsen’s research warned that “false positives” – where a person’s iris is erroneously found in the system – could essentially deny aid. It was a programme Steinacker supported as UNHCR’s head of global registration in 2004. Now, both he and Jacobsen are calling for an urgent review of data in Afghanistan, and for a deeper re-evaluation of the use of biometrics across the aid sector. This conversation has been edited for length and clarity. The New Humanitarian: What should be the immediate priority for aid agencies when it comes to evaluating data security in Afghanistan? How much – and what kinds – of data are we talking about? Karl Steinacker: What is important is that the agencies sit together and assess first what data there is, and where it is. Every big organisation would say: “There’s no need to worry. We have data security in place.” But is that so? What about the data which is in common databases: a child protection database, let’s say, where you can maybe trace single mothers, or victims of sexual violence – things that are quite delicate issues. The other issue is the data used through commercial service providers – cash programmes in particular. Were they cash programmes for very specific vulnerable groups who might be targeted by the Taliban, because they were war veterans, or sexual minorities – whatever it is. But this process has to start somewhere. Someone has to say, “Since we haven’t done what we should have done before we implemented these programmes, let’s now retroactively look at what we have, what can be accessed by the Taliban, and how can we mitigate the problem?” What should have happened is the data protection impact assessment has been done before they started these programmes. But we know from experience that no humanitarian agency does these impact assessments. The New Humanitarian: How much of a debate is there about data collection within humanitarian organisations before programmes begin? Steinacker: There is no debate, in reality. It’s assumed the more data I have, the better it is for the programme. I can better target, I can better report, I can better ask for funds from donors. There is no questioning whether this could create collateral damage, that it could really backfire. Katja Lindskov Jacobsen: Much of this is not as new as we would like to think. It’s new that it’s the Taliban. But UNHCR has, for a long time, had a data-sharing agreement of sorts with the [the US Department of] Homeland Security. The idea was to share data on refugees resettled in the US, but a lot more data has been shared. It’s researchers as well. A lot of actors are interested in biometrics that UNHCR and other agencies have collected – not just in Afghanistan. There are donors that are interested, and also host states. I think we have to think about this whole idea of who to give access to this data and who not to, and whether we can really control that. “Much of this is not as new as we would like to think. It’s new that it’s the Taliban.” Some humanitarian agencies, like the [International Committee of the Red Cross], have decided not to collect biometrics if they can avoid it. It puts into contrast some of the choices UNHCR has made about maintaining data forever, really. I think decisions like those have to be revised as well, given the sensitivity and the question of whether we can really make sure that this data, which is kept forever in enormous databases, is always in safe hands. The New Humanitarian: It obviously sounds alarming if the Taliban now has access to biometric and other data once possessed by Afghanistan’s former government. How would you describe the level of concern? Steinacker: The issue is extremely complex. [It could be] that the [Taliban] uses data to identify collaborators – people they consider traitors, collaborators, and whatnot. That would be my worry with regard to humanitarian data: that they use it, not against everyone who has received assistance, but specific groups: victims of sexual violence, sexual minorities – this kind of thing. Jacobsen: For me, the bigger issue is they could use it whenever someone would require assistance from the government, whether that be schools, hospitals, medical assistance of any kind. Services from the government could be linked to a requirement of registering irises or fingerprints, now they have the devices and the databases to do that. But even for individuals: having that concern would mean they decide not to go to hospital. We’ve seen that with UNHCR programmes [elsewhere], too: that individuals would decide not to register for humanitarian assistance because they didn’t know what was going to happen with their data. Was it being shared with the host government, or with their [home country]? The fear in the population that they have registered their data and it might be used – I think we have to take that fear seriously. Because it might mean that people don’t access hospitals, for example, if this is where the Taliban decides to introduce biometric registration. The New Humanitarian: What should humanitarian agencies do? Let’s start with the UNHCR data in Pakistan: biometric data – iris scans – collected from former Afghan refugees who returned from Pakistan over the last two decades. Steinacker: We would basically say, delete it. The people never consented to it being kept forever. Secondly, if you have a new influx into Pakistan, then it no longer serves any purpose. There is no reason to keep a database with four million anonymous biometric datasets. It makes no sense. “When should humanitarian and other organisations delete data?” Jacobsen: It calls into question for how long such data should be kept. For this specific database, it’s even more pertinent to delete it if it technically also cannot be of much use. But I think the question is much, much bigger. When should humanitarian and other organisations delete data? The New Humanitarian: For the data within Afghanistan, which has been collected by humanitarian agencies and others, what can or should be done with it? Steinacker: They have to do a kind of housecleaning. They have to find out first what data there is. The amazing thing about working in humanitarian agencies is that over the years, you amass data. And with the staff rotation, people who are working there today have no clue what has been collected five years ago or 10 years ago. And this data has been collected under different circumstances at different times. So the first thing that has to happen is an inventory of data: what data is there, what is needed and what is not needed, where are the copies… and is this data potentially damaging to people in the databases? So this operations security process has to take place. Jacobsen: And urgently. Not in a month. The New Humanitarian: Karl, you’re coming from a position where you were partly responsible for the collection of data at UNHCR, and now you’re advocating for the right to be forgotten. How would you describe how your views have changed? Steinacker: I was a fervent advocate of biometrics when it was introduced, because I believed that the systems which were used before were extremely degrading and humiliating to people. The way it was done before: it was like, spraying people with invisible ink, putting people behind fences, and whatnot. So I was certainly somebody who supported biometrics. “Aid agencies have to learn that this kind of data is extremely important. It’s not just statistics.” But we are all learning. Today, we can see certain impacts and side effects. I’m not totally against biometrics. But aid agencies have to learn that this kind of data is extremely important. It’s not just statistics. There is data that is linked to a person – through physical features like biometrics, for instance, but also others – that is so important that it needs special protection and measures, which have to be reviewed all the time because the situation is changing all the time. It’s not enough to pay lip service. I believe that most agencies are paying lip service to data protection and data security issues. They can show you their manuals, they can show you their instructions, they can always say we follow GDPR rules. But then it is self-policing. None of these agencies submits itself to third-party evaluations or oversight. No beneficiary has access to his own data. So it’s just – it’s a mess. And these are the moments when we can see it. ‘The Taliban have been given the keys to the server room.’ Biometric data and the Taliban: What are the risks? Irwin Loy Interview Aid and Policy Human Rights BANGKOK IRIN Asia Afghanistan Aid and Policy The UN’s refugee data shame Editor’s take: Data responsibility starts with you Head to Head: Biometrics and Aid Biometrics: The new frontier of EU migration policy in Niger In Bangladesh, a Rohingya strike highlights growing refugee activism Afghanistan......
Zuck is a lightweight, and 4 more things we learned about Facebook from 'An Ugly Truth'...
3 months ago
Here's the challenge of writing about the hot mess that is Facebook in 2021 — the mess is so hot, your take is soon out of date. So it was with An Ugly Truth: Inside Facebook's Battle for Domination, by New York Times reporters Sheera Frenkel and Cecilia Kang. Two days after the book published, we learned Facebook seems to be burying data that highlights its right-wing echo chamber. It just came under fire from President Biden for "killing people" with vaccine lies, and from English footballers for allowing racist abuse on their pages. Oh, and the FTC's antitrust case against the social media giant, which fills the first chapter, was just denied by a federal judge. Oops.But with the exception of the FTC case, which may yet be refiled, none of this news will come as a surprise to the book's readers. Frenkel and Kang's careful reporting shows a company whose leadership is institutionally ill-equipped to handle the Frankenstein's monster they built. Ignoring hate speech and lies on the platform, even when experts are sounding alarms about harmful effects up to and including genocide, is par for the course. As is trying to control the media narrative or silencing employees rather than, y'know, doing something about the underlying problem.It's not that CEO Mark Zuckerberg and COO Sheryl Sandberg come across as inherently evil; they seem genuinely shaken by each new crisis, evolve their thinking a little, and belatedly order fixes. It's more that their airy idealism, total faith in algorithms, plus endless profits, put them in bubbles where they are blind to bad actors and bad ideas. "People were not paying attention," Facebook's former security chief Alex Stamos tells the authors. He's talking about engineers who were able to stalk their dates with god-like access to their profiles (and were belatedly fired for doing so). But it could apply to everything from election interference to stolen user data to the company's ongoing lack of diversity (only 3.8 percent of employees are Black, barely up from 2 percent in 2014). So if you need more reasons to delete Facebook apps from your life, here are the main points we gleaned from An Ugly Truth — a book that mostly focuses on the years 2016 to 2020, but doesn't ignore the original sins baked into Facebook from the very beginning. 1. Zuckerberg, an 'intellectual lightweight,' was easy to manipulate. To be fair, plenty of us thought dumb stuff when we were 20. But few of us were laying the foundations for a history-changing global addiction at the time. From the start, Zuckerberg designed the platform for mindless scrolling: "I kind of want to be the new MTV," he told friends. His mantra for employees was "company over country": Do what is good for Facebook, not for America. Forget The Social Network; a more up-to-date cautionary tale would cut straight from this declaration to the Jan. 6 insurrection being plotted in Facebook groups. It was 20-year-old Zuckerberg who fell under the sway of Peter Thiel and Marc Andreesen, Silicon Valley libertarians and free-speech fundamentalists. He didn't read books at the time (veteran Valley journo Kara Swisher found him an "intellectual lightweight") and inherited a view of the First Amendment that one NAACP voting rights expert calls a "dangerous misunderstanding." Instead of protecting people from government censorship, Zuck's platform would protect and amplify the speech of authoritarian leaders. Conservatives would learn they could work the referee simply by claiming they were being censored. Once set in place, Zuckerberg's views were hard to shift — and when they did, they often managed to morph into more disastrous forms. His response to an epidemic of fake news on the site in 2016 was to downgrade all news in the algorithm's eyes. He'd long allowed for maximum data collection on users; one longtime employee says that any other path presented to him was "antithetical to Mark's DNA." Then, when the Cambridge Analytica scandal laid bare the need for privacy, he pushed for private Facebook groups — which provided safe harbor for murderous militias, QAnon believers and insurrectionists. But Zuck's unexamined privilege hurts Facebook users in everyday ways too. "He couldn't identify the systemic biases of the world: how, if you were Black, you might attract ads for predatory loans, or if you were lower income, you might attract ads for junk food and soda," the authors write. The full effects of Facebook addiction may take decades to unpack. 2. Sheryl Sandberg is a Pollyanna, with less power than we knew. Former Google exec Sandberg was long seen as the adult supervision at Facebook. She took on all the leadership roles that didn't interest Zuckerberg — including growing the ad business that supercharged revenue. Behind the scenes, it was assumed that the Lean In author was a moderating influence on Zuck's more clueless or dangerous designs for the service. Not so, turns out. "I've been consistently puzzled about the public portrayal of them as amazing partners," one business-side employee tells the book's authors. "Meetings I've been in with Mark and Sheryl, he was explicitly dismissive of what she said." Indeed, the book provides examples where Sandberg was afraid of getting fired, or being labeled as politically biased, and didn't even try to push back — such as the case of the doctored video of Nancy Pelosi that Zuckerberg decided to allow on the site. Pelosi still won't return Sandberg's calls. "To friends and peers, Sandberg tried to disassociate herself from Zuckerberg's positions, but within the company she executed his decisions," the authors write. They quote a Sandberg friend: "Her desire to appear perfect gets in the way of defending herself." Part of her perfectionism manifests in a desire to focus on the positive — to a Pollyanna-ish degree. Sandberg's conference room is named "Only Good News," and it's a fair summary of what she wants to hear from underlings. She quickly dismissed the notion that Russian-bought election ads spread further on the platform than was initially known, and screamed "you threw us under the bus" at then-CSO Stamos for telling the Facebook board what he knew about the situation. Like a lot of execs who didn't bring only good news, Stamos was gradually pushed out. 3. Joel Kaplan may be the most dangerous man at Facebook.A friend (and briefly former boyfriend) of Sandberg's, Joel Kaplan is a veteran of the George W. Bush administration who heads up Facebook's lobbying arm in Washington D.C. He's a very close friend of Justice Brett Kavanaugh, and showed up to support the controversial judge at his Supreme Court appointment hearings while on the clock for Facebook. He has Zuckerberg's ear on political matters, and on the evidence of An Ugly Truth he is more influential than Sandberg when it counts. Kaplan may have been a Never Trump Republican in his personal beliefs, but in practical fact he was the best friend Donald Trump had at Facebook. It was Kaplan who, in Dec. 2015, persuaded Zuckerberg not to take down Trump's first post calling for a Muslim ban. "Don't poke the bear," he advised, so instead Facebook carved out an exemption for hate speech if it was "newsworthy." This brand-new standard for what billions of people could see from a candidate arguably helped hand a close election in 2016 to Trump. That was just the beginning of the lobbyist's reign. It was Kaplan who downplayed Stamos' report on Russian interference, Kaplan who argued for keeping the Pelosi video up, Kaplan who helped kill a "Trending" news section when conservative politicians complained about it — laying the groundwork for the domination of Facebook's algorithm by right-wing radio hosts like Dan Bongino and Ben Shapiro. In 2019, Kaplan engineered Zuckerberg's two meetings with Trump. In 2020, he argued that Trump didn't actually suggest injecting bleach as a COVID-19 cure, therefore didn't fall foul of Facebook's rules on medical misinformation. He likewise defended Trump's "when the looting starts, the shooting starts" comment during the George Floyd protests, and when Facebook belatedly started cracking down on QAnon groups in August, it was Kaplan who made sure that antifa-related groups were also banned. Engineers admitted that such "both sides" behavior was entirely political. All of that appeasement, and for what? Trump was banned indefinitely anyway, but not before doing potentially irreparable harm to democracy by spreading the Big Lie. Government regulation of Facebook, the one thing Kaplan was trying to avoid, is now just about the only proposal that unites left and right in Washington. And yet Kaplan's position seems more secure than ever — cemented by an alliance with policy chief Nick Clegg, who knows President Biden from his time as UK deputy Prime Minister. 4. Employees are fighting the good fight.If anything can save Facebook from itself, it's the ground-level employees of Facebook. Time and again, in posts on the company's internal Workplace groups known as "Tribes," and in Zuckerberg's weekly Q&As, they are the ones forcing the uncomfortable questions. In 2016, News Feed team members were furious that blatantly fake news sites didn't run afoul of their rules, prompting a memo from early executive Andrew "Boz" Bosworth that gave the book its title. Boz wrote that the "ugly truth" was that connecting the world's people might lead to more deaths, but that Facebook would continue no matter what. Boz later claimed he was only trying to "inspire debate," but employees held his feet to the fire regardless. Next to face the workers' fury was Kaplan, for his overt support of Kavanaugh; employees dismissed Zuckerberg's falsehood that he had taken time off to do so. When Trump's post about shooting looters was allowed to stand, engineers on Tribe boards were openly asking if there were jobs at companies "willing to take a stance on their moral responsibility to the world—because Facebook doesn't seem to be it." An internal poll showed thousands of employees believed Zuck had made the wrong decision, and the company's first employee walkout followed. The one positive trend in An Ugly Truth is that employee statements, actions, and leaks to the media are getting bolder and clearer. "Stop letting people be racist on Facebook, maybe then we will believe that you support racial justice," a member of the Black@Facebook internal group wrote in 2020. We're a long way from Zuckerberg having to tell his employees not to deface Black Lives Matter signs on campus. These days, the education appears to be flowing the other way. 5. Facebook must listen to more experts — inside and out.Employee intentions are great, but they're nothing without management taking action and creating guidelines. In 2008, the first team responsible for moderating Facebook posts had only vague rules that there should be "no nudity, no terrorism" and a general desire that was summed up as "don't be a radio for a future Hitler." How'd that work out? The most heartbreaking section in An Ugly Truth concerns Myanmar, where fake news about Muslim minorities on Facebook snowballed into riots and a military-led genocide in 2018. An expert in the country, Matt Schissler, was sounding the alarm to Facebook representatives back in 2014. Visiting the Menlo Park HQ, Schissler showed a post that had gone viral with a picture of a man feeding Muslim refugees; the man had become a target for death threats for doing so. To take down the picture, Facebook told him, the man would have to complain himself — even though he wasn't on Facebook. Schissler was equally horrified to learn that the Facebook reps "seemed to equate the harmful content in Myanmar to cyberbullying," and that there was one Burmese speaker hired by the company to police the whole country's content. (Burmese is only one of dozens of languages spoken in the country.) "It would be like them saying, well, we have one German speaker, so we can monitor all of Europe," Schissler tells the authors. In the end, his complaints got about as much traction as Stamos and other election security experts hired then eased out by Facebook. Which is to say, none at all. Zuckerberg appears belatedly aware of the value of expertise — at least, in as much as it can take tough decisions off his plate. That was the thinking behind the Facebook Oversight Board, which rightly put the question of whether Trump should be banned from the platform permanently back in the CEO's court. Depending on what action Zuckerberg takes next, and its impact on the 2022 and 2024 elections, An Ugly Truth may end up being a relatively lightweight prologue to a coming Facebook dystopia.......
Govt puts half-year fiscal deficit at 3.1pc of GDP
ISLAMABAD: Highlighting certain risks to fiscal sustainability, the federal government has reported its half-year (July-December) fiscal deficit at 3.1 per cent of GDP, or the highest-ever figure of Rs1.393 trillion in absolute terms. In its mid-year Budget Review Report submitted to the parliament as required under the Public Finance Act, 2019, the Ministry of Finance on Monday claimed credit for higher revenues, controlled expenditures amid fiscal consolidation measures but conceded that there were “certain risks to fiscal sustainability”. Going forward, the fiscal position would depend on the domestic and international evolution of the Covid-19 pandemic, the ministry said, adding that faster-than-anticipated economic revival was also likely to increase demand for inputs. During the first half of the current financial year, an amount of Rs116 billion was provided to combat the pandemic from the revalidated Economic Stimulus Package, including purchase of vaccines worth Rs25bn. Moreover, Rs64bn was utilised under the Ehsaas Programme to provide relief to vulnerable segments of society. Finance ministry claims credit for higher revenues and controlled expenditures, but sees ‘certain risks to fiscal sustainability’ The finance ministry said the government had adopted a facilitative policy of releasing funds to meet the expenditures, both recurrent and development, in accordance with its spending priorities. Nevertheless, the half-year fiscal position indicates that factors will remain on track to meet the annual fiscal targets. The country’s overall fiscal deficit was, however, brought down by about Rs255bn cash surplus provided by the four provinces. The overall primary balance (excluding debt servicing and interest payments) was also reported at 0.7pc (surplus) against a full year deficit limit of 0.5pc. The report said the continuity in fiscal consolidation, stable exchange rate, improved current account and better financial management presented a promising economic outlook. The current account balance continued to improve, posting a surplus of $1.1bn (0.8pc of the GDP) during the first half of the year, it said. Tax collection grew by 5.6pc in first half on a Year-on-Year (YOY) basis despite an upsurge of Covid-19 and the Federal Board of Revenue achieved about 99pc of its half year target. Non-tax revenue remained at par with the previous year collection in spite of reduction in State Bank of Pakistan profits and non-realisation of fees from cellular license renewals. Current expenditure was controlled through austerity measures and strict financial discipline. The government said its borrowing operations remained in line with the Medium-Term Debt Management Strategy (MTDS FY20-23) and just like last year, domestic borrowing came entirely from the financial markets and about Rs285bn borrowing from the SBP was retired. Also, the borrowing to fund the fiscal deficit was made through longer-term debt, while short-term debt (T-bills) reduced by around Rs579bn. The major sources of non-tax revenue for the federal government during this period are surplus profit of the SBP and Petroleum Levy which showed more than 110pc growth. As such, the government achieved 54pc of budgeted targets despite the adverse impact of the Covid-19 pandemic on economic activity. The ministry said the expenditure on running of civil government was reduced to 40pc of the allocation by restricting supplementary grants and implementing austerity measures. Additional funds have only been approved as a supplementary grant which remained unutilised under the Economic Stimulus Package of last financial year to extend the relief measures. All additional needs of the ministries and divisions had been met through Technical Supplementary Grants from within the allocated budget, with primary reliance on re-appropriation of funds. The Rs1.475tr worth major chunk of current expenditure was spent on debt servicing, out of which, domestic interest payments amounted to Rs1.357tr and external interest payments amounted to Rs118bn. The ministry also reported that Rs232bn had been utilised against the Public Sector Development Programme’s allocation of Rs650 billion up to December last year. Published in Dawn, March 2nd, 2021...
FDA’s Data Modernization Action Plan: Putting Data to Work for Public Health
FDA’s Data Modernization Action Plan: Putting Data to Work for Public Health Anonymous (not verified) Wed, 03/03/2021 - 09:18 Detailed Description Data modernization is the next step in the agency’s overhaul of its approach to technology and data, and we are pleased to announce the Data Modernization Action Plan (DMAP). Center Office of the Commissioner By: Janet Woodcock, M.D., Acting Commissioner of Food and Drugs, and Amy Abernethy, M.D., Ph.D., Principal Deputy Commissioner & Acting Chief Information Officer With the near ubiquity of interconnected smartphones and computers in modern life, it can be difficult to remember just how quickly our familiarity with data has evolved. Data, in digital form, seem to be everywhere we look. Yet, we are still at an early stage in our ability to apply data to understand and treat disease and address other public health challenges. At this early stage, even small advances in our ability to gain useful insights from data can represent significant opportunities. Janet Woodcock, M.D. In September 2019, when we announced the U.S. Food and Drug Administration’s Technology Modernization Action Plan (TMAP) we spoke about the ways that the FDA is modernizing our approach to the use of technology for the agency’s regulatory mission, such as in the review of medical product applications and food safety, and other critical functions. Data modernization is the next step in the agency’s overhaul of its approach to technology and data, and we are pleased today to announce the Data Modernization Action Plan (DMAP). Data have always formed the basis of the FDA’s science-based regulatory decision-making. These data may come from relatively traditional sources—for example, measurements submitted to the FDA from clinical trials or observations from FDA field inspections. As technology has become more sophisticated and our world has become more connected, data from many new sources are helping us understand how medical products are performing, how we can pinpoint the source of a foodborne illness, for example, or understand an emerging public health threat. Data Sources Are Increasingly Diverse Digitization of processes, pervasive use of mobile technologies, and easier access to computing resources have created new types of data. These data types and related technologies can be used to create innovative solutions to improve public health, such as: Capabilities to track and trace medical and food products can transform national response to emergencies by identifying product and logistic information across the entire supply chain. Integration of real-world and clinical trial evidence can increase representation of diverse populations. It can transform overall efficiency of product reviews and post market surveillance. Privacy sparing innovations can advance the evaluation of potential treatments for rare diseases while protecting patient specific information. The new data strategy also calls for ways to scale agency work, such as the Advanced Semantic Search and Indexing of Text for Tobacco pilot to explore and build a model to scale search capabilities for complex tobacco submissions. Improved search capabilities based on context and intent will scale review, monitoring, and regulatory operations by reducing the burden of manual, inefficient processes on FDA staff. Delivering Value Through Driver Projects Will Inform Modernization Amy Abernethy, M.D., Ph.D.The DMAP is anchored on driver projects that contribute to the FDA’s public health responsibilities in the near term while also building critical capabilities for the future. Driver projects for DMAP are defined as initiatives with measurable value that help multiple stakeholders envision what is possible, allow technical and data experts to identify needed solutions, and develop foundational capabilities. This strategy avoids the pitfalls of focusing on data collection first and only then looking for questions the data can answer. The selected driver projects will not only address traditional performance indicators but will also support transformation across the agency by using predictive models, and appropriate application of trends, such as Artificial Intelligence (AI). Promoting Consistent and Repeatable Data Practices Across FDA A modern data strategy also requires proactive investments in foundational capabilities. Key components of Data Practices to achieve these goals are Identification; Data Curation; Governance; and, Automation. For example, the Global Substance Registration System, designed by the FDA, in collaboration with the National Institute of Health's National Center for Advancing Translational Sciences and the European Medicines Agency, provides consistent, auditable, quality and uniquely identified substance information for reviewers and scientists. Attention to data management practices such as use of international data standards, expert curation, stewardship and program area governance ensures that this data foundation is useful for all stakeholders. Fostering a Strong Talent Network at FDA It’s critical that the agency have a strong focus on talent and elastic talent networks, to ensure that the modernization plan will be swift, consistent and economical. For example, last year the FDA developed an agency-wide advisory matrix for surveillance inspections in less than six weeks by using data available from the U.S. Department of Health and Human Services and combining it with internal data. A small team of specialists and volunteers used agile methods to complete the project and launched it for use across the FDA and by state, tribal, territorial and local government liaison officers. For each DMAP key element, the plan lays out the specific actions the agency will take to implement the strategy. We will assemble a cross-agency Steering Committee to support the planning and execution of the DMAP implementation. Our data plan will leverage the foundations laid by TMAP: a modern, cloud-forward technical infrastructure; a product-oriented approach, and enhanced collaboration. The two action plans will go hand in hand to realize the full potential and value of data and technology for the FDA and its stakeholders. Image Communication Type Blog Post Short Title FDA’s Data Modernization Action Plan: Putting Data to Work Source Organization FDA Short Description Data modernization is the next step in the agency’s overhaul of its approach to technology and data. Content Owner Office of Editorial & Creative Services Publish Date Wed, 03/03/2021 - 13:51 Review Date Thu, 03/03/2022 - 00:00 Last Reviewed Date Wed, 03/03/2021 - 00:00 Site Structure FDA Voices Next Review Date 1 Year Navigational Page Off Bulk Approved Off Add Subscription Box Off Display Short Description Off First Publish Date Wed, 03/03/2021 - 13:51 Generic Boolean Off Regulated Product* Animal & Veterinary Biologics Cosmetics Dietary Supplements Drugs Food & Beverages Medical Devices Radiation-Emitting Products Tobacco Language English Show Related Information Hide Number of Related Information to Display 3 Add Subscription Box On Description Subscribe to receive FDA Voices on Policy email notifications. Email Subscription List FDA Voices on Policy Header FDA Voices on Policy Email...
The Hill's Morning Report - Presented by the National Shooting Sports Foundation - Relief bill to become law; Cuomo in trouble
Presented by the National Shooting Sports Foundation Welcome to The Hill’s Morning Report. Today is Monday and it is International Women’s Day! We get you up to speed on the most important developments in politics and policy, plus trends to watch. Alexis Simendinger and Al Weaver are the co-creators. Readers can find us on Twitter @asimendinger and @alweaver22. Please recommend the Morning Report to friends and let us know what you think. CLICK HERE to subscribe!Total U.S. coronavirus deaths reported as of this morning: 525,035. As of this morning, 17.2 percent of the U.S. population has received at least one dose of a COVID-19 vaccine and 9.2 percent are fully vaccinated, according to the Bloomberg News global vaccine tracker.The Biden administration’s $1.9 trillion COVID-19 relief package is on the verge of becoming law, pending action by the House and White House in the coming days, as health experts warn that the current level of cases in the U.S. is untenable and the nation races to vaccinate the masses. The House is set to follow the Senate’s lead and OK the nearly $2 trillion stimulus blueprint on Tuesday, with President BidenJoe BidenLawmakers, activists remember civil rights icons to mark 'Bloody Sunday' Fauci predicts high schoolers will receive coronavirus vaccinations this fall Biden nominates female generals whose promotions were reportedly delayed under Trump MORE’s signature coming shortly after to check a big-ticket item off the administration’s 100-day to-do list. Among other things, the package provides another round of stimulus checks, aid for state and local governments — an issue that had stalled out in previous rounds of relief negotiations — and more help for small businesses and schools. Despite cries from across the aisle that the bill is laden with Democratic pet projects, wasteful spending and provisions unrelated to the ongoing pandemic, the GOP has to contend with a major political issue: the bill’s popularity. Multiple polls conducted prior to the legislation’s passage in the upper chamber on Saturday show that the $1.9 trillion plan enjoys broad support. According to a recent Monmouth University survey, 62 percent of respondents were in favor of the bill, while 71 percent gave a thumb up to the plan when polled by Morning Consult. Adding to the good news for Democrats, Biden continues to receive high marks for his handling of the pandemic. According to a new ABC News/Ipsos poll, 68 percent approve of Biden's handling of the pandemic (ABC News). As The Hill’s Morgan Chalfant and Alexander Bolton write, Democrats remain ever-aware of what happened last decade after the 2009 stimulus package became law, with Republicans bludgeoning that and the Affordable Care Act en route to massive wins in the 2010 midterms. Biden last week encouraged House Democrats to “speak up and speak out” about the rescue plan, suggesting that the Obama administration “paid a price” for not taking enough of a victory lap after the 2009 recovery package. “Any of my colleagues at the time would say that we didn’t do enough to explain to the American people what the benefits were of the rescue plan and we didn’t do enough to do it in terms that people would be talking about at their dinner tables,” White House press secretary Jen PsakiJen PsakiMississippi governor defends ending mask mandate Border crisis creates new risks for Biden Cruz puts hold on Biden's CIA nominee MORE said Friday. “That’s one of the reasons we, of course, have been trying to break down the impact of the American rescue plan into the key components that will impact people directly.” The New York Times: What’s in the stimulus bill? A guide to where the $1.9 trillion is going. The Washington Post: “An essential service”: Inside Biden’s struggle to meet his school reopening promises. CNBC: Futures slip after Senate passes $1.9 trillion COVID-19 relief bill. Meanwhile, the efforts to corral the pandemic are reaching a critical phase as public health officials fret that the current level of daily infections remains too high and that another wave of the virus could be in the offing due to the rise of variants, even with the national effort to vaccinate Americans. As of Sunday, the U.S. is averaging 60,000 new COVID-19 cases per day, a steep decline from the dead of winter when the U.S. averaged more than 200,000 new infections daily. However, As The Hill’s Reid Wilson reports, case totals have plateaued over the past week, raising fears that a new wave is just around the corner. “We could not have made a more wonderful environment for this virus to take off than we have right now,” said Michael Osterholm, director of the Center for Infectious Disease Research and Prevention at the University of Minnesota. “We are not driving this tiger. We’re riding it. And the first time we may be able to drive it is with widespread use of the vaccine, and we’re not there yet.” The Hill: Former Biden COVID-19 adviser: “We are in the eye of the hurricane right now.” The Wall Street Journal: Republican governors diverge over COVID-19 restrictions as experts urge caution. The Hill: Scott Gottlieb: “Probable” that high schoolers will get coronavirus vaccines this year. Anthony FauciAnthony FauciFauci predicts high schoolers will receive coronavirus vaccinations this fall Texas patrons threaten to call ICE on Mexican restaurant for keeping mask mandate Gottlieb: 'Probable' that high schoolers will get coronavirus vaccines this year MORE, head of the National Institute of Allergy and Infectious Diseases, called the current state of daily infections “unacceptable” and warned governors against the expeditious rollback of mask mandates and reopenings. “Historically, if you look back at the different surges we’ve had, when they come down and then start to plateau at a very high level, plateauing at a level of [60,000] to 70,000 new cases per day is not an acceptable level. That is really very high,” Fauci told “Face the Nation” (The Hill). According to Bloomberg News’s daily tracker, the U.S. is averaging 2.2 million vaccinations per day over the past week, with Saturday’s total of 2.9 million doses administered setting a new single-day total. The Wall Street Journal: Russian disinformation campaign aims to undermine confidence in Pfizer, other COVID-19 vaccines, U.S. officials say. The Hill: White House COVID-19 coordinator: Administration focused on expanding vaccine access. Josh Rogin for Politico Magazine: Diplomats warned of a coronavirus danger in Wuhan — two years before the outbreak. LEADING THE DAYCONGRESS: The coronavirus relief efforts will soon be in the rearview mirror, prompting attention to be refocused toward the rest of the Biden administration’s legislative priorities and raising questions about the viability of passing many of the items on the Democratic wish list in the coming months. As The Hill’s Jordain Carney points out, the upper chamber is threatening to derail the bold agenda laid out by Democrats, with progressives training their fire at the unlikely elimination of the filibuster. The passage of a number of major items in the House is putting a spotlight on the fact that without structural changes, many of the party’s campaign promises are heading for the Senate graveyard. Senate Majority Charles SchumerChuck SchumerManchin firm on support for filibuster, mulls making it 'a little bit more painful' to use Biden takes victory lap after Senate passes coronavirus relief package Lawmakers demand changes after National Guard troops at Capitol sickened from tainted food MORE (D-N.Y.) is pledging to put the bills on the floor, daring Republicans to vote against them, effectively turning them into messaging votes. Progressives are hoping that the expected GOP blockade of Democratic bills could help sway the few Democratic senators who have voiced opposition to getting rid of the 60-vote threshold. The Associated Press: With virus aid in sight, Democrats debate filibuster changes. Chief among that group is Sen. Joe ManchinJoseph (Joe) ManchinDemocrats near pressure point on nixing filibuster All eyes on Manchin after COVID-19 aid passes Senate Justice: 'I'm not going to get in a food fight with Joe Manchin' on use of CARES Act funds MORE (D-W.Va.) (seen below). The West Virginia moderate was the center of attention on the Sunday show circuit, reiterating his belief that the filibuster is a tool for good overall and a necessity to a functioning Senate. “I'm not going to change my mind on the filibuster. I will change my mind if we need to go to a reconciliation to where we have to get something done once I know they have process into it,” Manchin told “Meet the Press.” “But I'm not going to go there until my Republican friends have the ability to have their say also,” Manchin said. “And I'm hoping they'll get involved to the point to where we have 10 of them that'll work with 50 of us or 15 of them that'll work with 45 of us” (The Hill). Manchin also received some backup from the White House. Communications director Kate BedingfieldKate BedingfieldAll eyes on Manchin after COVID-19 aid passes Senate Sunday shows: Manchin in the spotlight after pivotal role in coronavirus aid debate White House says Biden would prefer to not end filibuster MORE told CNN's “State of the Union” that Biden remains committed to winning Republican support even after GOP lawmakers voted unanimously against the massive relief bill (The Hill). The Sunday Shows: Manchin in the spotlight after pivotal role in coronavirus aid debate. The Washington Post: Narrow relief bill victory provides warning signs for broader Democratic agenda. The Hill: Rep. Jefferson Van Drew (R-N.J.), after flipping parties, bashes bills he once backed. The Hill: Lawmakers gird for spending battle over nuclear weapons. ***** POLITICS: New York Gov. Andrew CuomoAndrew CuomoTop New York Democrats call on Cuomo to resign Whitmer encourages investigation into Cuomo's conduct Sunday shows: Manchin in the spotlight after pivotal role in coronavirus aid debate MORE (D) maintained on Sunday that he will not resign after two additional women accused him of sexual harassment or unwanted advances and more high profile New York Democrats called for his ouster, saying that he will allow state Attorney General Letitia James (D) to complete her investigation into the claims. “I was elected by the people of the state. ... I'm not going to resign because of allegations,” Cuomo told reporters on a conference call. “There is no way I resign” (Fox News). On Saturday, Karen Hinton, a former aide to Cuomo, accused him of making an unwanted advance in a hotel room in 2000. The governor forcefully disputed the allegations, calling Hinton a “longtime political adversary.” “Every woman has a right to come forward. That’s true. But the truth also matters. What she said is not true,” Cuomo said of the allegation. The Wall Street Journal also reported on Saturday of another allegation from Ana Liss, a former staffer who said that Cuomo made her uncomfortable by inquiring about her dating life and kissing her on the hand in 2013. The new revelations also brought forth a potential political deathblow as Andrea Stewart-Cousins, the head of the New York state Senate, called for his resignation, saying it would be “for the good of the state.” “Every day, there is another account that is drawing away from the business of government,” Stewart-Cousins said in a statement. “We need to govern without daily distraction. Governor Cuomo must resign.” As The New York Times notes, Cuomo is following the same track as former New York Gov. Eliot Spitzer (D), who saw his standing evaporate when political leaders in Albany called for him to step down amid his 2008 prostitution scandal. Politico: Cuomo leans on crisis management playbook as walls close in. > Lone Star uprising?: It’s been a rocky start to 2021 for Texas Republicans, boosting hopes among Lone Star State Democrats that they can bounce back from a poor 2020 cycle that saw them lose a number of key congressional races, fall in their bid to flip the Texas state House and fail to unseat Sen. John CornynJohn CornynSenate holds longest vote in history as Democrats scramble to save relief bill Biden gets involved to help break Senate logjam Overnight Defense: Capitol Police may ask National Guard to stay | Biden's Pentagon policy nominee faces criticism | Naval Academy midshipmen moved to hotels MORE (R-Texas). As The Hill’s Jonathan Easley writes, the once-in-a-blue-moon power grid failure that led to a humanitarian crisis last month drew attention to the GOP’s leadership in a state that has not elected a Democrat to statewide office in nearly a quarter-century. Sen. Ted CruzRafael (Ted) Edward Cruz Cruz puts hold on Biden's CIA nominee It will be Vice (or) President Harris against Gov. DeSantis in 2024 — bet on it Senate rejects Cruz effort to block stimulus checks for undocumented immigrants MORE’s (R-Texas) brief jaunt to Cancun, Mexico, during the crisis also ignited anger and severe blowback. Last week, Gov. Greg Abbott (R) announced he was lifting the statewide mask mandate and fully reopening Texas for business despite lagging vaccination rates and a higher coronavirus infection rate than the national average, eliciting a response from the president in the process. Texas Democrats are still picking themselves up off the mat after a dismal 2020 showing. But the rough two months in the national spotlight for Texas Republicans has Democrats optimistic about 2022 as they seek to win in swing districts where they were defeated last cycle. The Hill: Nevada looks to shake up presidential primary calendar. The New York Times: Sen. Josh HawleyJoshua (Josh) David HawleyDeSantis, Pence tied in 2024 Republican poll Chamber of Commerce clarifies stance on lawmakers who voted against election certification Crenshaw pours cold water on 2024 White House bid: 'Something will emerge' MORE (R-Mo.) Is “not going anywhere.” How did he get here? Reid Wilson, The Hill: GOP's tyranny of the minority. Washingtonian: 250 of the most influential experts and advocates who shape policy debates (working outside of government), organized by subject categories. Bloomberg News: Researchers have found a stronger correlation between political instability and rising homicide rates than prevailing hypotheses, such as economic distress and prevalence of guns.IN FOCUS/SHARP TAKESADMINISTRATION: The president signed a new executive order on Sunday to leverage federal resources in an effort to protect and strengthen access to the ballot, a response to ongoing effort by GOP state legislatures to restrict voting rights after the 2020 election (The Hill). Biden's order, which directs agencies to increase access to voter registration materials and reduce barriers to voting for certain groups, comes amid a push by congressional Democrats to pass H.R. 1, a sprawling bill the House passed last week to reform voting processes and elections. House Majority Whip James Clyburn (D-S.C.) acknowledged on Sunday the reality that the bill will likely die in the Senate because of the filibuster, adding that the result will be “catastrophic.” “There’s no way under the sun that in 2021 that we are going to allow the filibuster to be used to deny voting rights. That just ain’t gonna happen. That would be catastrophic,” Clyburn told The Guardian. The president’s executive action coincided with the 56th anniversary of the “Bloody Sunday” march in Selma, Ala., which served as a catalyst for the passage of the Voting Rights Act. As The Hill’s Marty Johnson notes, Sunday’s commemoration of the 1965 violent clash between 600 civil rights marchers and white police officers on the Edmund Pettus Bridge was the first to take place without the late Rep. John LewisJohn LewisLawmakers, activists remember civil rights icons to mark 'Bloody Sunday' Clyburn: Allowing filibuster to be used to deny voting rights would be 'catastrophic' Walking in the footsteps of giants — honoring the legacy of Rep. John R. Lewis MORE (D-Ga.), who died in July. The Associated Press: Biden marks “Bloody Sunday” by signing voting rights order. The Hill: Biden to formally establish new Gender Policy Council. > Border: The Biden administration is faced with a burgeoning crisis along the U.S.-Mexico border as officials have seen the number of unaccompanied minors crossing into the U.S. rise steadily over the first six weeks of the nascent presidency. Thousands of migrants have crossed into the U.S. since Biden took office, many of them unaccompanied minors, testing the administration’s resources and ability to quickly implement its own strategy at the border in the face of criticism from across the aisle. According to The Hill’s Brett Samuels and Jonathan Easley, the Biden administration is rapidly adapting its approach to meet the growing need for space and manpower in a reflection of the seriousness of the situation. The Hill: Political landmines await Merrick GarlandMerrick GarlandSenate to vote next week on Garland's AG nomination Biden's justice reform should influence prosecutor appointments Politics in the Department of Justice can be a good thing MORE at Department of Justice. Axios: Japan Prime Minister Yoshihide Suga set to become first foreign leader to visit Biden in White House.The Morning Report is created by journalists Alexis Simendinger and Al Weaver. We want to hear from you! Email: firstname.lastname@example.org and email@example.com. We invite you to share The Hill’s reporting and newsletters, and encourage others to SUBSCRIBE! OPINIONThe road to reopening won’t be a straight line, by Scott Gottlieb and Mark McClellan, opinion contributors, The Wall Street Journal. https://on.wsj.com/3tbfDqz Do liberals care if books disappear? By Ross Douthat, columnist, The New York Times. https://nyti.ms/3v3gpaI WHERE AND WHENThe House meets at noon. The Senate convenes on Tuesday at 3 p.m. and will resume consideration of the nominations of Rep. Marcia FudgeMarcia FudgeWe need to lay the foundation for meaningful housing policy change Black Caucus members lobby Biden to tap Shalanda Young for OMB head Sanders votes against Biden USDA nominee Vilsack MORE (D-Ohio) to become secretary of Housing and Urban Development and Garland to lead the Department of Justice. The president and Vice President Harris will receive the Presidential Daily Brief at 9:50 a.m. Biden and Veterans Affairs Secretary Denis McDonoughDenis Richard McDonoughThe Hill's 12:30 Report - Presented by ExxonMobil - Increased security on Capitol Hill amid QAnon's March 4 date The Hill's Morning Report - Presented by Facebook - Lawmakers face Capitol threat as senators line up votes for relief bill Pentagon launches civilian-led commission to address military sexual assault MORE at 1 p.m. will visit the D.C. Veterans Affairs Medical Center, where the administration of COVID-19 vaccines to veterans is taking place. Biden, Harris and Defense Secretary Lloyd AustinLloyd AustinBiden nominates female generals whose promotions were reportedly delayed under Trump Sunday shows: Manchin in the spotlight after pivotal role in coronavirus aid debate Pentagon chief: Response to rocket attack in Iraq will be 'thoughtful' and 'appropriate' MORE will also deliver remarks on International Women’s Day from the White House at 4:20 p.m. First lady Jill BidenJill BidenJill Biden redefines role of first lady Biden faces criticism over push to vaccinate teachers Overnight Health Care: Biden slams Texas, Mississippi for lifting coronavirus restrictions: 'Neanderthal thinking' | Senate panel splits along party lines on Becerra |Over 200K sign up for ACA plans during Biden special enrollment period MORE at 10 a.m. will address the 2021 International Women of Courage Award ceremony hosted by the State Department and Secretary of State Antony BlinkenAntony BlinkenChina: Electoral reform would bring 'brighter future' for Hong Kong State sanctions Ukrainian billionaire over alleged corruption Australian PM Morrison says Biden will join first-ever 'Quad' meeting MORE. The event is live streamed HERE. The White House press briefing is scheduled at 11:30 a.m.. The White House COVID-19 response team will brief the news media at 11 a.m. The National League of Cities Congressional City Conference begins today through Wednesday, including virtual remarks from House Speaker Nancy PelosiNancy PelosiTrump White House associate tied to Proud Boys before riot via cell phone data Greene sounds off on GOP after Hill story 'Bloody Sunday' to be commemorated for first time without John Lewis MORE (D-Calif.), Schumer, Treasury Secretary Janet YellenJanet Louise YellenThe Hill's 12:30 Report - Presented by ExxonMobil - Senate begins marathon vote-a-rama before .9T COVID-19 relief passage The Hill's Morning Report - Presented by Facebook - Virus relief bill headed for weekend vote Debt to break WWII record by 2031 MORE, Transportation Secretary Pete ButtigiegPete ButtigiegThe Hill's 12:30 Report - Presented by ExxonMobil - Senate begins marathon vote-a-rama before .9T COVID-19 relief passage The Hill's Morning Report - Presented by Facebook - Virus relief bill headed for weekend vote Biden turns focus to next priority with infrastructure talks MORE, Centers for Disease Control and Prevention Director Rochelle WalenskyRochelle WalenskyOvernight Health Care: CDC study links masks to fewer COVID-19 deaths | Relief debate stalls in Senate | Biden faces criticism over push to vaccinate teachers CDC study links masks to fewer COVID deaths The Hill's 12:30 Report - Presented by ExxonMobil - Increased security on Capitol Hill amid QAnon's March 4 date MORE, Fauci and more. Information is HERE. Hill.TV’s “Rising” program features news and interviews at http://thehill.com/hilltv or on YouTube at 10:30 a.m. EST at Rising on YouTube. ELSEWHERE➔ INTERNATIONAL: Two protesters were shot dead in Myanmar on Monday by security forces as demonstrations continue to take place across the country in response to the military coup that took place last month. Two people were shot and killed in Myitkyina in Kachin State, with the military using tear gas and some automatic gunfire in an attempt to disperse the crowd. Protests also took place in Naypyitaw, the capital city, and Mandalay (The Associated Press). ➔ TECH: Silicon Valley giants are drawing battle lines over personal data collection practices and targeted ads. Google was the latest to take a step toward data privacy by announcing plans this week to phase out its own tracking features used for personalizing ads. That move came after Apple drew Facebook's ire with its anticipated anti-tracking feature. Experts and pro-privacy advocates say that while Google’s update may be a small step toward giving users more control over their data protection, the change may hurt Google’s rivals more than the company itself (The Hill). ➔ UN-ROYALED: Prince Harry and wife, Meghan, the Duchess of Sussex, told Oprah WinfreyOprah Gail WinfreyPrince Harry says he felt 'trapped' in royal familiy Meghan Markle says she wasn't able to get help for suicidal thoughts as a member of the royal family Meghan Markle says royal family discussed her unborn son's skin color MORE during a much-ballyhooed interview on CBS on Sunday that they struggled mightily with life in the Royal Family, including Harry saying that he felt “trapped” and Meghan making a number of allegations that could rock the British royals. The Duchess made two stunning revelations: that she had suicidal thoughts after marrying Harry and that the royals did not give her access to mental health resources, and that there were conversations within the family about "how dark” their son Archie’s “skin might be" (The Associated Press). THE CLOSERAnd finally … Dogs are good. Sports are good. Dogs and sports together though? Of course they’re good! The fabulous combo came together on Sunday as the 49th edition of the Iditarod kicked off (or mushed off?) in Deshka Landing in Alaska. Like other sporting events these days, it has been modified to comply with COVID-19 restrictions. The mushers — of which there are a limited number this year — are wearing masks from start to finish, and social distancing is being adhered to at all times. The course has also been shortened to 852 miles, and organizers have asked fans to stay home. The event will also end in Deshka Landing, marking the first time in the Iditarod’s history that it will end somewhere other than Nome, Alaska. Last year, the Iditarod was the final sporting event to be completed before the worldwide shutdown took place (Anchorage Daily News). Share on Twitter JW Video Type: CutdownPerson: Rochelle WalenskyKate BedingfieldDenis McDonoughMerrick GarlandCharles SchumerAntony BlinkenPete ButtigiegAnthony FauciMeghan MarkleOprah WinfreyLloyd AustinNancy PelosiMarcia FudgeJanet YellenAndrew CuomoJohn CornynJoe ManchinJosh HawleyJill BidenJohn LewisJoe BidenJen PsakiTed CruzExcluded from Just In: 0Video comments: Video comments...
Digital Preservation at the University of Minnesota Libraries
Overview The need to develop strategy and take action in the area of digital preservation and data archiving has grown significantly in the Libraries and at the University in recent years. In response, the Libraries aspire to take a campus leadership role in digital preservation and data archiving. The Digital Preservation and Repository Technologies (DPRT) department is in the Data & Technology Division of the University of Minnesota Libraries. The department’s mission is to provide contemporary and enduring access to digital objects under the collection stewardship of the University Libraries, through the application of professional digital preservation methodologies, standards, and technologies. This work is accomplished through the delivery and support of robust high quality discovery, access, management, and preservation systems, achieved in coordination with relevant Libraries’ departments, the Office of Information Technology (OIT), and other University and external partners and collaborators. What is Digital Preservation? Digital preservation encompasses a broad range of activities designed to extend the usable life of machine-readable computer files and protect them from media failure, physical loss, and obsolescence. Activities Planning and Development The department researches and develops a digital preservation and data archiving plan for University Libraries, designed with potential extensibility for broader campus participation. In this process, they review all relevant existing Libraries practices, assess needs, researches best practices, standards, reference models, and state-of-the-art technologies, and closely collaborate with Libraries staff and other stakeholders to establish policies and best practices for long-term digital preservation and archiving. Data types of concern include, but are not limited to, images, audio, video, text, and scientific datasets. They assist in ongoing policy development concerning format specifications for preservation purposes. Over time, the department will play a role in the selection and/or development of a certified trustworthy repository of digital resources for the Libraries and, potentially, the campus and beyond. Collaboration and Education The department assists in the ongoing development of requirements and specifications, including formats, for digital material accepted through the University Digital Conservancy (UDC). It informs members of the UDC Management and Working groups on strategies and best practices to preserve digital objects deposited in the institutional repository. Similar interactions with the Data Management and Curation Initiative and efforts to establish a media repository are expected. Through these exchanges and others, the department works to increase staff awareness about and knowledge on the issues related to digital archiving and preservation. In addition, general digital preservation trainings (often based on the Library of Congress' Digital Preservation Outreach & Education program) are given online or in person in partnership with Minitex. Campus Partnerships & Services The department fosters collaboration and exchange with departments, labs, and centers across the University in identifying, storing, and preserving digital resources; identifies assets in need of Libraries services; develops relationships with those responsible for managing such assets; shares expertise in digital preservation management with the campus; and builds strong working relationships towards these ends. External Connections The department identifies external funding opportunities for digital preservation projects. They prepare specifications for vended services that support the digital preservation program, evaluate responses to proposals for such services, and makes recommendations for selecting vendors. The department acts on behalf of the Libraries as a technical liaison on digital preservation issues to vendors providing services. They also represent the University Libraries in external projects or programs in the area of digital preservation. Contact Us: For questions, please contact us at: firstname.lastname@example.org Upcoming Events...
How (Not) to Write a Privacy Law
After many years of failed starts, proposed information privacy legislation has begun moving forward in both houses of Congress. The docket in the recently ended 116th Congress was crowded, with a number of different proposals jostling for attention and no agreement on which deserved to be the front runner. Even so, as the 117th Congress begins, there is growing inside-the-Beltway consensus on the list of features that a successful bill will need to include. This paper critically assesses those zones of emerging consensus. Many of the shared features of proposed privacy legislation embody fundamentally backward-looking approaches that cannot hope to constrain the activities they attempt to address. To varying extents but across the board, the current crop of proposals reflects what behavioral psychologist Abraham Maslow identified as the tendency to conform tasks to preexisting tools rather than vice versa.1 1. Abraham H. Maslow, The Psychology of Science: A Reconnaissance 15–16 (1966). Encapsulated in the saying that “if all you have is a hammer, everything looks like a nail,” Maslow’s insight is not really about hammers or nails but rather about the way human beings conceptualize approaches to new problems, and it contains an important lesson for would-be privacy reformers. The rote, brute-force application of laws designed around the governance challenges of a prior era will not resolve the governance dilemmas created by today’s surveillance-based business models. But all is not lost; the key lies in recognizing that governance too can be a site of innovation. Part I discusses the zone of emerging consensus surrounding the definition and assertion of individual information privacy rights. Current approaches to crafting privacy legislation are heavily influenced by the antiquated private law ideal of bottom-up governance via assertion of individual rights, and that approach, in turn, systematically undermines prospects for effective governance of networked processes that operate at scale. Part II turns to the question of public governance. Many of the proposed bills designate public governance structures, but they also import antiquated public law mechanisms that operate without due regard for the topography of networked, data-driven processes. Effective privacy governance requires a model organized around problems of design, networked flow, and scale. I identify some essential components of such a model and a few more specific strategies for lawmakers and regulators to consider. Part III evaluates proposals for enforcement of newly recognized rights and obligations. Here there is less unanimity—some propose to create private rights of action while others would authorize only public enforcement litigation—but all parties seem to agree on the menu of choices, and none of those choices promises efficacy. The debate about private versus public enforcement litigation has been an unproductive distraction from the task of crafting more effective public enforcement mechanisms, of which I identify three. Rights-Based Governance: Legislating in the shadow of the cathedral Both existing information privacy laws and the recent crop of legislative proposals are pervasively informed by a governance paradigm that is deeply embedded in the U.S. legal tradition and that relies on individual assertion of rights to achieve social goals. To be clear, none of the bills recently before Congress purports, in so many words, to recognize property rights in personal data. Even so, almost all adopt a basic structure that is indebted to property thinking. Within that structure, individual control rights function as the primary mechanism for governing the collection and processing of personal data, with no or only residual provision for ongoing governance at the collective level. Atomistic, post hoc assertions of individual control rights, however, cannot meaningfully discipline networked processes that operate at scale. Nor can they reshape earlier decisions about the design of algorithms and user interfaces. Most of the bills introduced in the 116th Congress begin by assigning sets of control rights to consumers. Consumers may then consent to collection and processing, effectively waiving their purported control rights. Some proposals would require consumers to opt in to data collection and processing, as in the case of the Online Privacy Act of 2019, sponsored by Rep. Anna Eshoo (D-CA), and the Privacy Bill of Rights Act, sponsored by Sen. Edward Markey (D-MA).2 2. Online Privacy Act of 2019, H.R. 4978, 116th Cong. (2019); Privacy Bill of Rights Act, S. 1214, 116th Cong. (2019). Others that require opt-in consent are: Social Media Privacy Protection and Consumer Rights Act of 2019, S. 189, 116th Cong. (2019) (sponsored by Sen. Amy Klobuchar [D-MN]); Consumer Online Privacy Rights Act, S. 2968, 116th Cong. (2019) (sponsored by Sen. Maria Cantwell [D-WA]); Consumer Data Privacy and Security Act of 2020, S. 3456, 116th Cong. (2020) (sponsored by Sen. Jerry Moran [R-KS]); Information Transparency & Personal Data Control Act, H.R. 2013, 116th Cong. (2019) (sponsored by Rep. Suzan K. DelBene [D-WA]). One such bill, Sen. Maria Cantwell’s (D-WA) Consumer Online Privacy Rights Act, additionally defines more general duties that covered information processing entities owe to consumers and permits consumers to sue when they believe those duties have been violated.3 3. Consumer Online Privacy Rights Act, S. 2968, 116th Cong. (2019) (sponsored by Sen. Maria Cantwell [D-WA]). I discuss the ways that regulatory oversight might lend substance to general duties in Part II. Others would require consumers to opt out of data collection and processing, as in the case of the Mind Your Own Business Act of 2019, sponsored by Sen. Ron Wyden (D-OR), and the Balancing the Rights of Web Surfers Equally and Responsibly Act of 2019, sponsored by Sen. Marsha Blackburn (R-TN).4 4. Mind Your Own Business Act of 2019, S. 2637, 116th Cong. (2019) (requiring creation of a “Do Not Track” data sharing opt-out website); Balancing the Rights of Web Surfers Equally and Responsibly Act of 2019, S. 116, 116th Cong. (2019) (proposing opt-in approval for sensitive user information and opt-out approval for non-sensitive user information). The precise mechanisms for opting in or out vary—some bills specify detailed mechanisms while others would require agency rulemaking—but everyone seems to agree that such mechanisms are important. The two laws repeatedly held up as models for a new wave of stricter privacy protection at the state level, the Illinois Biometric Information Privacy Act (BIPA) and the California Consumer Protection Act (CCPA), also adopt a control-rights-plus-opt-in-or-out approach, with BIPA adopting opt-in requirements for biometric data and CCPA establishing an opt-out mechanism for personal data more generally.5 5. See California Consumer Privacy Act of 2018 (CCPA), Cal. Civ. Code § 1798.120(a) (2018) (outlining a consumer right to opt-out). See also Illinois Biometric Information Privacy Act (BIPA), 740 Ill. Comp. Stat. 14/15 (2008). The continuing optimism about consent-based approaches to privacy governance is mystifying, because the deficiencies of such approaches are well known and relatively intractable. Many of the bills do attempt to impose new procedural requirements, and some, such as Sen. Wyden’s Mind Your Own Business Act and the Do Not Track Act of 2019, sponsored by Sen. Josh Hawley (R-MO), would empower regulators to create a system that records consumers’ expressed preferences.6 6. See, e.g., Do Not Track Act, S. 1578, 116th Cong. (2019) (empowering the FTC to establish and enforce a national Do Not Track system); Mind Your Own Business Act of 2019, S. 2637, 116th Cong. (2019) (same). For reasons ably explained by many talented privacy scholars and advocates, however, such provisions are unlikely to result in real control in any meaningful sense. In brief: The issues that users must navigate to understand the significance of consent are too complex and the conditions surrounding consent too easy to manipulate.7 7. Alessandro Acquisti, Curtis Taylor & Liad Wagman, The Economics of Privacy, 54 J. Econ. Literature 442 (2016); Woodrow Hartzog, Privacy’s Blueprint: The Battle to Control the Design of New Technologies (2018); Nora A. Draper & Joseph Turow, The Corporate Cultivation of Digital Resignation, 21 New Media & Soc’y (2019), https://doi.org/10.1177/1461444819833331 [https://perma.cc/J8B4-K3ES]. Most formulations of user control rights don’t clearly include information derived from user behavior, thereby opening the way for gamesmanship by tech firms around the synthetic data that lie at the core of advertising-based business models.8 8. See, e.g., Lillian Edwards & Michael Veale, Enslaving the algorithm: From a ‘right to an explanation’ to a ‘right to better decisions’? 16 IEEE Sec. and Priv. 46–54 (2018); Michael Veale, Reuben Binns, & Max Van Kleek, Some HCI Priorities for GDPR-Compliant Machine Learning, CHI-GDPR (2018), https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3143705 [https://perma.cc/R36K-D7Q7];Lillian Edwards & Michael Veale, Slave to the Algorithm? Why a ‘Right to Explanation’ is Probably Not the Remedy You Are Looking For, 16 Duke L. & Tech. Rev. 18, 68–72 (2017). See also Joe McNamee, Is Privacy Still Relevant in a World of Bastard Data?, EDRi Ed., (March 9, 2016), https://edri.org/enditorial-is-privacy-still-relevant-in-a-world-of-bastard-data [https://perma.cc/UE6Q-YCH7]. Additionally, it’s not clear what the right to revoke consent means in the context of machine-learning-based models trained on a large corpus that includes the to-be-withdrawn data.9 9. See, e.g., Edwards & Veale, supra note 8,at 33-35. The problem I want to highlight here concerns the aggregate efficacy of such consent mechanisms: Organizing a regulatory regime around individual control rights imports a governance structure that is atomistic and post hoc. Individual users asserting preferences over predefined options on modular dashboards have neither the authority nor the ability to alter the invisible, predesigned webs of technical and economic arrangements under which their data travels among multiple parties. Nor can they prevent participants in those webs from drawing inferences about them—even when the inferences substitute for the very same data that they have opted out of having collected. The assumption that bottom-up governance driven by self-interested rights assertion will actually work derives from long-held, nearly automatic ways of thinking about property rights as mechanisms for collective ordering. The property tradition holds that property rights internalize governance incentives and minimize governance costs by situating authority over resource access and use where it can be exercised most wisely and effectively.10 10. See, e.g., Henry E. Smith, Exclusion Versus Governance: Two Strategies for Delineating Property Rights, 31 J. Legal Stud. S453 (2002); Henry E. Smith & Thomas W. Merrill, Optimal Standardization in the Law of Property: The Numerus Clausus Principle, 110 Yale L. J. 1 (2000). Contemporary property thinkers do recognize that such an approach can undervalue certain types of collective harms. Even so, they argue that because collective governance is costly, property rights should be the default arrangement in most situations, and if rights to access and use a resource need to be transferred or aggregated, property owners can negotiate the arrangement that they all prefer. Within this way of thinking about the relationship between individual control and governance, externally imposed regulatory requirements are the exception—and this is a feature, not a bug.11 11. For a good explanation, see Henry E. Smith, Mind the Gap: The Indirect Relation between Ends and Means in American Property Law, 94 Cornell L. Rev. 959, 964-69 (2009) (characterizing the default to private ordering as a “rebuttable presumption” and acknowledging its outer limits). By analogy, one might think that individual control rights (and presumed self-regulatory incentives flowing from assertion of those rights) offer the most effective and appropriate way of channeling data collection and processing activities to achieve other regulatory goals. And, to be fair, reliance on disaggregated, bottom-up governance flowing from assertion of individual control rights makes some sense in one-on-one negotiations over such matters as the conditions of access to real property or consent to medical treatment. But such reliance has also failed repeatedly and spectacularly as a mechanism for ensuring effective governance of collective interests in land use and in medical research ethics.12 12. See generally Lee Anne Fennell, The Problem of Resource Access, 126 Harv. L. Rev. 1471 (2013); Carl H. Coleman, Rationalizing Risk Assessment in Human Subjects Research, 46 Ariz. L. Rev. 1 (2004). Many of the most spectacular failures have involved the interests of marginalized and/or low-income communities. See generally Thomas W. Mitchell, From Reconstruction to Deconstruction: Undermining Black Landownership, Political Independence, and Community through Partition Sales of Tenancies in Common, 95 Nw. U. L. Rev. 505 (2001); Vernellia R. Randall, Slavery, Segregation and Racism: Trusting the Health Care System Ain't Always Easy—An African American Perspective on Bioethics, 15 St. Louis U. Pub. L. Rev. 191 (1995). Privacy failures also follow that pattern. See generally Ruha Benjamin, Race After Technology: Abolitionist Tools for the New Jim Code (2019); Nancy S. Kim, Consentability: Consent and Its Limits (2019); Safiya Umoja Noble, Algorithms of Oppression: How Search Engines Reinforce Racism (2018). It makes no sense whatsoever where networked, large-scale processes are involved. It’s worth noting, moreover, that property thinking extends beyond the rights that the proposed privacy bills purport to define, shaping important underlying exceptions in ways that frustrate the accountability the bills profess to guarantee. In many legal systems—most notably, those in European Union member states—disclosure of personal information for law enforcement or national security purposes doesn’t eliminate the need to comply with data protection obligations, although it does change their form and content. Under the U.S. approach, law enforcement and national security exceptions tend to move the activity beyond the reach of data protection obligations altogether. So, for example, in Fourth Amendment jurisprudence, courts generally understand themselves to be navigating a universe of control rights that is divided into two parts. Data acquisition may be subject to heightened procedural requirements, but once data has been lawfully acquired, it passes into government control.13 13. On heightened process requirements for data acquisition, see Carpenter v. United States, 138 S. Ct. 2206 (2018); Riley v. California, 573 U.S. 373 (2014); United States v. Warshak, 631 F.3d 266 (6th Cir. 2010). Statutes governing information collection beyond the Fourth Amendment’s reach tend to follow the same pattern, saying little about what happens to data once it has been properly acquired.14 14. See 18 U.S.C. § 2518 (Wiretap Act); 18 U.S.C. § 2703 (Stored Communications Act). National security legislation has become a partial exception to this rule, incorporating concepts like minimization that are familiar to data protection lawyers.15 15. See 50 U.S.C. § 1806(a) (“Information acquired from an electronic surveillance ... may be used and disclosed ... only in accordance with minimization procedures.”). See also 50 U.S.C. § 1801(h) (defining “minimization procedures”); see generally Daphna Renan, The FISC’s Stealth Administrative Law, in Global Intelligence Oversight: Governing Security in the Twenty-First Century 121 (Zachary K. Goldman & Samuel J. Rascoff eds., 2016). Yet the veil of secrecy surrounding national security data practices also frustrates the capacity for collective governance and precludes the post hoc forms of accountability to individual citizens that, for example, European human rights courts have required.16 16. Case C-311/18, Schrems and Facebook Ireland v. Data Protection Commissioner (2020), http://curia.europa.eu/juris/document/document.jsf?text=&docid=228677&pageIndex=0&doclang=en#Footnote* ¶45 (“While individuals, including EU data subjects, therefore have a number of avenues of redress when they have been the subject of unlawful (electronic) surveillance for national security purposes, it is equally clear that at least some legal bases that U.S. intelligence authorities may use (e.g., E.O. 12333) are not covered. Moreover, even where judicial redress possibilities in principle do exist for non-U.S. persons ... claims brought by individuals ... will be declared inadmissible where they cannot show ‘standing.’”); Case C-362/14, Schrems v. Data Protection Commissioner (2015) ¶90 (concluding that the U.S. government’s use of personal information went “beyond what was strictly necessary and proportionate to the protection of national security” and “that the data subjects had no administrative or judicial means of redress.”). None of the privacy bills proposed in the 116th Congress addresses accountability for government data practices. (None, therefore, would cure the defects identified by the European Court of Justice in invalidating the EU-U.S. Privacy Shield agreement, which was intended to enable transfer of European citizens’ data to the U.S. for processing in the context of commercial activities.17 17. Case C-311/18, Schrems and Facebook Ireland v. Data Protection Commissioner (2020).) Some argue that user-governed data cooperatives might enable scaling consent for the era of data-driven, networked processes in a way that enables users to retake control of privacy dashboards and command adherence to their preferred sets of terms.18 18. See generally Eric A. Posner & E. Glen Weyl, Radical Markets: Uprooting Capitalism and Democracy for a Just Society (2018). But see Elettra Bietti, Locked-in Data Production: User Dignity and Capture in the Platform Economy (Oct. 14, 2019), https://ssrn.com/abstract=3469819 [https://perma.cc/AB4V-J9LF] (arguing a market-based approach to give individuals rights for their data contributions perpetuates harm). There are no extant examples of such arrangements, however, and confident predictions of their eventual emergence seem ill-informed for two reasons. First, to the extent that such arguments rely on theoretical work by economists on collective mechanisms for governing common resources, they tend to ignore important qualifications that affect the ability of common-governance arrangements to scale. Such arrangements are most effective in smaller, more homogeneous communities attempting to govern resources whose boundaries can be demarcated relatively clearly; they become less effective as the communities grow in size and heterogeneity and as the boundaries of the resource pool become less easy to control.19 19. See generally Elinor Ostrom, Governing the Commons: The Evolution of Institutions for Collective Action (1990); Brett Frischmann, Michael J. Madison, & Katherine J. Strandburg, eds., Governing Knowledge Commons (2014). Second, power differentials within communities also shape the scaling-up process. So, for example, open-source licensing scaled relatively well during the two or so decades when the community of internet developers consisted primarily of hobbyists and research scientists, and much less well after conflicts with the interests of large, for-profit technology firms began to mature, and the Creative Commons licensing system never scaled to encompass the activities of large, for-profit content interests at all.20 20. See Julie E. Cohen, Property and the Construction of the Information Economy: A Neo-Polanyian Ontology, in Handbook of Digital Media and Communication 333-49 (Leah Lievrouw & Brian Loader, eds., 2020). To be clear, consent provisions do invite some kinds of collective entrepreneurship around governance of personal information flows; it just isn’t the sort of entrepreneurship likely to produce greater control or greater privacy for individuals. Instead, reliance on consent as the principal governance mechanism for personal data has invited forms of entrepreneurship that follow the “contracting into liability rules” model used by collective rights organizations for managing intellectual property rights.21 21. Robert P. Merges, Contracting into Liability Rules: Intellectual Property Rights and Collective Rights Organizations, 84 Calif. L. Rev. 1293 (1996). That model is well-suited for scaling up licensing processes over large numbers of low-value transactions. It produces a type of governance arrangement designed to operate at scale, but it does so—necessarily—by standardizing the options presented to users in ways that interpose additional barriers to meaningful consent on a more granular level. So, for example, because European data protection laws require affirmative consent from individuals for many kinds of processing, smaller entities unable to internalize the resulting compliance costs have begun affiliating to offer “automated consent management panels” through which users can signal their choices to all of the member entities.22 22. Midas Nouwens, Dark Patterns after the GDPR: Scraping Consent Pop-Ups and Demonstrating their Influence, CHI’20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems 1 (April 2020), https://dl.acm.org/doi/pdf/10.1145/3313831.3376321. Within such arrangements, consent becomes a fig leaf deployed to achieve compliance with a regime that requires symbols of atomistic accountability. Users remain unable to demand or specify changes in the basic conditions of information processing or the design of networked services. Should the U.S. adopt consent requirements similar to those that now obtain in Europe, one imagines that industry adoption of automated consent management panels would quickly follow. In short, both faith in the efficacy of disaggregated governance and hopes for the possibility of collective governance are best understood as reflecting backward-looking, conceptually entrenched commitments to private ordering rather than anything resembling evidence-based reasoning. Both arguments for bottom-up governance flowing from assertion of individual rights and arguments for commons-based cooperative governance of personal data collection and processing overlook the structural and temporal effects of design operating at scale. Effective governance of such activities requires public oversight—and, as we are about to see, it also requires new thinking about how to structure and conduct such oversight. Coverage and Oversight: Maslow’s hammer strikes again … and again and again Assuming legislation drafted to prioritize effective governance rather than atomized, post hoc assertions of control rights, how should privacy governance work? To varying extents but across the board, the bills introduced in the 116th Congress exemplify Maslow’s hammer in action. All lean heavily on the set of existing public governance tools in ways that drastically reduce the likelihood of effective intervention. An initial choice concerns where to situate oversight authority. Most recent proposals would vest authority in the Federal Trade Commission (FTC) or, for proposals aimed at voter privacy, the Federal Election Commission (FEC). The choice of regulator is enormously consequential because it tends to bake in preexisting jurisdictional limitations that weren’t designed with the networked information economy in mind. So, for example, the FTC lacks jurisdiction over common carrier functions of information businesses, which in turn means that any grant to the FTC effectively gives the Federal Communications Commission (FCC), which does have that jurisdiction, some power to limit the extent of the FTC’s authority. Only some of the proposed bills would expressly transfer authority to oversee the personal data practices of common carriers to the FTC.23 23. See, e.g., Information Transparency and Personal Data Control Act, H.R. 2013, 116th Cong., 1st Sess. (2019) (sponsored by Rep. Suzan Delbene, D-WA); Social Media Privacy Protection and Consumer Rights Act, S. 189, 116th Cong., 1st Sess. (2019) (sponsored by Sen. Amy Klobuchar, D-MN); Digital Accountability and Transparency to Advance Privacy Act, S. 583, 116th Cong., 1st Sess. (2019) (sponsored by Sen. Catherine Cortez Masto, D-NV). The FTC also has more limited rulemaking and enforcement powers than other independent agencies, for reasons that have always been political and that reflect ingrained reluctance to give consumer protection regulators authority to meddle with market activity, and it has very limited resources. Proposed bills that designate the FTC as privacy regulator without removing the preexisting limits should be understood as subscribing to those choices, whether or not their sponsors acknowledge it.24 24. Chris Jay Hoofnagle, Federal Trade Commission Privacy Law and Policy 55-56, 333-35 (2016); see, e.g., Social Media Privacy Protection and Consumer Rights Act of 2019, S. 189, 116th Cong. (2019); Balancing the Rights of Web Surfers Equally and Responsibly Act of 2019, S. 116, 116th Cong. (2019); Application Privacy, Protection, and Security Act of 2020, H.R. 6677, 116th Cong. (2020); Do Not Track Act, S. 1578, 116th Cong. (2019). For its part, the FEC has authority only over “electioneering communications” and only in certain ways. Even if the relevant definitions were amended to treat digital advertising the same way as broadcast advertising, the most potent and toxic flows of data-driven, networked misinformation and disinformation would not qualify as electioneering communications.25 25. See Richard L. Hasen, Cheap Speech: Saving American Elections in the Disinformation Era (forthcoming, Yale Univ. Press 2021) (manuscript on file with author); see also Julie E. Cohen, Tailoring Election Regulation: The Platform Is the Frame, 4 Geo. L. Tech. Rev. 641, 649-53 (2020). Sponsors of bills confidently promising to counter election manipulation by bringing greater transparency to digital electioneering typically do not seem to understand this. Even proposals to create wholly new privacy agencies, however, aren’t immune to the temptations posed by Maslow’s hammer. Consider two different, increasingly popular ways of framing delegations of regulatory authority: The first approach appropriates and repurposes trust-based concepts first developed in fiduciary and corporate law. It exists in two principal strands, one arguing for articulation of duties of confidentiality, care, and loyalty that mirror fiduciary duties, and the other arguing more generally for obligations of trustworthiness.26 26. Jack M. Balkin, The Fiduciary Model of Privacy, 134 Harv. L. Rev. Forum 11 (2020); Jack M. Balkin, Information Fiduciaries and the First Amendment, 49 U.C. Davis L. Rev. 1183 (2016); Neil M. Richards & Woodrow Hartzog, A Duty of Loyalty for Privacy Law (July 3, 2020), https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3642217 [https://perma.cc/QS5R-JY2R]. Both arguments rest to varying extents on analogies to other sorts of arrangements that have been thought to trigger fiduciary obligations. Because the practices that generate personal data for processing and targeted advertising are relational—users enter into long-term arrangements with many different providers of platform-based services—the idea of relationships extending throughout time and across different contexts might (in theory) support imposing obligations based on the idea that such relationships create heightened duties. One important critique of this approach points out that the concept of heightened duties has tended to scale poorly in the corporate context, where managerial self-interest and perceived short-term obligations to shareholders tend to prevent internalization of more public-regarding sensibilities.27 27. David E. Pozen & Lina M. Khan, A Skeptical View of Information Fiduciaries, 133 Harv. L. Rev. 497 (2019). One might, of course, seek to constrain self-interest and short-termism by specifying duties of care and loyalty clearly and precisely.28 28. See Richards & Hartzog, supra note 26; Claudia E. Haupt, Platforms as Trustees: Information Fiduciaries and the Value of Analogy, 134 Harv. L. Rev. 34 (2020); Andrew F. Tuch, A General Defense of Information Fiduciaries, 98 Wash. U. L. Rev. (forthcoming in 2021). But the problem of how to foster trust in networked digital economies is even more complicated than debates about the relevance of fiduciary analogies within corporate settings tend to assume. The mere fact of an ongoing service relationship signifies relatively little in an era when relationships have been redefined as mass-market products and are mediated by standardized interfaces designed for large-scale, networked interconnection.29 29. See generally Shmuel I. Becker & Sara Dadush, Relationship as Product: Transacting in the Age of Loneliness, 2021 U. Ill. L. Rev. (forthcoming in 2021). In the era of relationship-as-mass-market-product, it is worth remembering that our legal and regulatory traditions surrounding product safety have evolved in ways that are different and more exacting. One might of course describe the web of obligations and standards that exists in the realm of product design using the language of trust; so, for example, when I sit in the desk chair in my office, I trust that it won’t collapse. Yet I suspect most modern lawyers would understand that formulation as too general to be helpful. When accidents inevitably occur, it is far more useful to be able to speak concretely about such matters as material tolerances and manufacturing specifications—and to be able to invoke corresponding tort and regulatory frameworks—than it is to talk in airy generalities about the nature of my relationship to the chair manufacturer. This point about the necessary relationship between design and regulatory oversight extends beyond the manufacture of consumer goods to a vast range of complex, information-intensive products and services that are, in fact, comprehensively regulated. We might posit that pharmaceutical companies should have duties of care, loyalty, and scientific integrity toward those who ingest their drugs; that insurance companies should have duties of care, loyalty, and actuarial integrity toward their policyholders; and that banks should have duties of care, loyalty, and solvency toward their depositors. But traditions of public governance of all three groups of actors are sufficiently robust that most would understand such formulations as adding very little to debates about how those relationships-as-products ought to be governed. A second approach to framing delegations of regulatory authority involves using ideas of market domination derived from antitrust law to structure oversight regimes. So, for example, some of the proposed bills include special disclosure and reporting mandates that would apply only to covered information businesses of a certain size.30 30. See, e.g., Consumer Data Privacy and Security Act of 2020, S. 3456, 116th Cong. (2020) (enhanced management requirements for larger businesses and reduced consumer access rights vis-à-vis smaller businesses); Mind Your Own Business Act of 2019, S. 2637, 116th Cong. (2019) (enhanced reporting and user-facing accountability requirements for larger businesses). Conceptually, such proposals map to—but also represent a partial retreat from—arguments for mandating breakups and reorganizations of the very largest tech giants. (Breakups and other antitrust interventions may also be on the table in the 117th Congress.31 31. See Staff of Subcomm. on Antitrust, Com. and Admin. L. Comm. on Judiciary, 116th Cong., Investigation of Competition in Digital Markets (2020).) But antitrust-based approaches do not align well with surveillance abuses. Antitrust law has long grappled with the question of how to reconcile intangible intellectual property rights with competition mandates; addressing market domination within networked information ecosystems requires confronting similar questions about the appropriate extent of control over networked data flows structured by technical and legal protocols.32 32. See Nikolas Guggenberger, Essential Platforms (Sept. 30, 2020), Stan. Tech. L. Rev. (forthcoming in 2021), https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3703361 [https://perma.cc/5LJC-LZ5S]; Philip J. Weiser, Law and Information Platforms, 1 J. Telecomm. & High Tech. L.1 (2002). In order for proposals targeting dominant actors to produce meaningful restructuring of surveillance-based business models, they would need to disrupt not only corporate ownership and control structures but also licensed flows of data. In particular, the software developer kits supplied to app developers by dominant platform providers embed data collection and transmission protocols in ways that developers themselves often do not understand.33 33. See Aaron Sankin & Surya Mattu, The High Privacy Cost of a “Free” Website, The Markup (Sept. 22, 2020), https://themarkup.org/blacklight/2020/09/22/blacklight-tracking-advertisers-digital-privacy-sensitive-websites [https://perma.cc/6S2P-9LEG]. Privacy legislation designed to disrupt dominance must speak clearly to the ways that data flows are designed, embedded, layered, concealed, and propagated via networks of relationships that include multiple actors. And antitrust interventions designed to extend data flows outside the licensing ecosystems of dominant entities will only make privacy problems worse if they are not paired with other, privacy-focused interventions. More generally, the dysfunctions of the networked information economy reflect underlying problems of networked flow and scale that are distinct from existing patterns of market domination. Regulation of information-economy phenomena confronts what Paul Ohm has characterized as an “order of magnitude problem”: networked flows of information produce effects that manifest at scale.34 34. Paul Ohm, Regulating at Scale, 2 Geo. L. Tech. Rev. 546 (2018). Market domination plays an important role in that process, and the self-serving actions of dominant legal actors can exacerbate order of magnitude harms, but networked phenomena generate scalar effects even when a dominant legal entity is not present. Marginal actors in networked information ecosystems have become adept at leveraging both the services of dominant platforms and the underlying attributes of information networks that connect human populations. New and established media companies, disinformation farms, and extremist brokers of hate and ethnic supremacy all employ strategies for collecting, processing, and exploiting personal data, and those strategies rely on accumulated learning about how to optimize content for networked, social circulation across multiple platforms and applications.35 35. See Anthony Nadler, Matthew Crain & Joan Donovan, Weaponizing the Digital Influence Machine: The Political Perils of Online Adtech, Data & Soc’y (2018), https://datasociety.net/wp-content/uploads/2018/10/DS_Digital_Influence_Machine.pdf [https://perma.cc/5RHD-2E5J]; Caitlin Petre, The Traffic Factories: Metrics at Chartbeat, Gawker Media, and The New York Times, TOW CTR. FOR DIG. JOURNALISM (2015), https://doi.org/10.7916/D80293W1 [https://perma.cc/4L4V-5GLU].To be effective at all, regimes for privacy governance need to target order of magnitude problems in ways that enable oversight and enforcement to scale up and out commensurately. For some European observers, the cardinal sin of U.S. legislative proposals for privacy governance is their failure to implement a third approach: a suite of European-style data protection obligations that would apply to all entities regardless of their size or market position and that would map to data stewardship issues more straightforwardly and comprehensively than fiduciary principles do. In particular, data protection incorporates duties of minimization and purpose limitation that, some argue, would make a difference in the ways information-centric business models operate.36 36. Regulation 2016/679 of the European Parliament and of the Council of 27 April 2016 on the Protection of Natural Persons with Regard to the Processing of Personal Data and on the Free Movement of Such Data, and Repealing Directive 95/46/EC, May 4, 2016, 2016 O.J. (L 119), art. 5(1)(b)-(c). Policymakers in the U.S. have largely acquiesced in the narrative advanced by tech firms and their advocates that innovation requires broad leeway to repurpose collected data and so, unsurprisingly, most of the bills recently before Congress don’t take the idea of purpose limitation seriously.37 37. On the innovation narrative, see Julie E. Cohen, Between Truth and Power: The Legal Constructions of Informational Capitalism 71-72, 89-106 (2019). A more fundamental problem with the data protection approach is that, to the extent that it relies on prudential obligations rather than on more concrete specifications for structural limits on data collection and use, it invites death by a thousand cuts. Data protection law was originally conceived as a suite of requirements for enabling information collection and transfer with appropriate safeguards for data subjects. And data protection in practice can reduce to an exercise in managerial box-checking.38 38. See Przemyslaw Palka, Data Management Law for the 2020s: The Lost Origins and New Needs, 68 Buff. L. Rev. 559 (2020). The European General Data Protection Regulation (GDPR) imposes a substantive duty of data protection by design and default, but it does not specify the sorts of design practices that such a duty might require. There is a hole at the center where substantive standards ought to be—and precisely for that reason, data protection regulators often rely on alleged disclosure violations as vehicles for their enforcement actions, reflexively reaching back for atomistic governance via user control rights as the path of least resistance.39 39. See Margot Kaminski & Meg Leta Jones, An American’s Guide to the GDPR, 98 Denver L. Rev. (forthcoming in 2021); Palka, supra note 38. In short, while problems of trust and market domination each undeniably contribute to the dysfunctions that surveillance-based business models create, responding adequately to those dysfunctions requires moving beyond reactive conceptions of data protection toward a governance model organized around problems of design, networked flow, and scale,and framed in terms of concrete requirements that must be satisfied by firms collecting, processing, and exchanging personal information. How, though, are regulators to develop and exercise the sort of authority that I have just described? Here Maslow’s hammer makes a third appearance. Legislators framing delegations of regulatory authority tend to assume—in the face of mounting evidence to the contrary—that regulators will be able to pursue the goals they have defined using a preexisting tool set consisting largely of century-old techniques for economic regulation. In legislative drafting, questions about regulatory tool sets tend to be afterthoughts. Legislators and their staff seem to understand the universe of available tools as a fixed and relatively static category. So, if one wants to gain regulatory traction on a new problem—or, conversely, if one wants to appear reformist while moving the needle only very slightly—one confers authority to make rules and bring enforcement actions, and then one waits to reap the predicted beneficial results. Many of the proposed bills follow this well-worn formula, ignoring that it has already broken down. Even informal rulemaking has proved wholly unsuited to the task of constraining networked, highly informationalized processes because both the processes and available learning about how to govern them evolve so quickly.40 40. See, e.g., Chris Brummer, Disruptive Technology and Securities Regulation, 84 Fordham L. Rev. 977 (2015): 977-1052; Henry T.C. Hu, Disclosure Universes and Modes of Regulation: Banks, Innovation, and Divergent Regulatory Quests, 31 Yale J. Reg. 565 (2014); Paul Ohm, Broken Promises of Privacy: Responding to the Surprising Failure of Anonymization, 57 UCLA L. Rev. 1701 (2010); Philip J. Weiser, The Future of Internet Regulation, 43 U.C. Davis. L. Rev. 529 (2009). For its part, enforcement practice has largely devolved into standard-form consent decree practice, creating processes of legal endogeneity that simultaneously internalize and dilute substantive mandates.41 41. See Cohen, supra note 37, at 160-63, 188-89; Ari Ezra Waldman, Privacy Law’s False Promise, 97 Wash. U. L. Rev. 773 (2020). On legal endogeneity, see generally Lauren Edelman, Working Law: Courts, Corporations, and Symbolic Civil Rights (2016). Other parts of the regulatory landscape do offer more diverse suites of oversight tools—in particular, contemporary approaches to financial regulation, which have themselves undergone rapid change over the last quarter century, suggest a variety of strategies that privacy regulators might appropriate and repurpose—but the need to operate within preexisting regulatory silos prevents beneficial experimentation with such tools. Empowering a regulator to conduct effective privacy governance requires three kinds of disruption to business as usual so that regulatory innovation can proceed. First and most basically, effective privacy governance requires a suite of modern oversight authorities and staff with the ability to develop and execute them. Tools for privacy regulators might include design requirements borrowed in concept from consumer finance regulation; operating requirements for auditing, benchmarking, and stress testing borrowed in concept from bank regulation; monitoring requirements borrowed in concept from a range of regulatory fields; and more.42 42. See, e.g., Mehrsa Baradaran, Regulation by Hypothetical, 67 Vand. L. Rev. 1247 (2014); Rory Van Loo, The Missing Regulatory State: Monitoring Businesses in an Age of Surveillance, 72 Vand. L. Rev. 1563 (2019); Rory Van Loo, Regulatory Monitors: Policing Firms in the Compliance Era, 119 Colum. L. Rev. 369 (2019); Rory Van Loo, Rise of the Digital Regulator, 66 Duke. L.J. 1267 (2017); Lauren E. Willis, Performance-Based Consumer Regulation, 82 U. Chi. L. Rev. 1309 (2015). Equally important, it might include other types of tools developed in consultation with experts on matters ranging from dark patterns to algorithmic bias to network threat modeling.43 43. See, e.g., Hartzog, supra note 7; David Freeman Engstrom, et al., Government by Algorithm: Artificial Intelligence in Federal Administrative Agencies, Report submitted to the Administrative Conference of the United States, Feb. 2020, https://www-cdn.law.stanford.edu/wp-content/uploads/2020/02/ACUS-AI-Report.pdf [https://perma.cc/LMM2-LRBK]. Second, implementing the new tools requires disruptions to entrenched patterns of privatized oversight that too often stand in for true public-private cooperation in regulatory matters. Across a wide and growing number of economic sectors, regulatory authorities have become more heavily reliant on third-party auditors, technology vendors, and other professional intermediaries to evaluate regulatory compliance.44 44. Kenneth A. Bamberger, Technologies of Compliance: Risk and Regulation in a Digital Age, 88 Tex. L. Rev. 669 (2010); Kenneth A. Bamberger, Regulation as Delegation: Private Firms, Decisionmaking, and Accountability in the Administrative State, 56 Duke L.J. 377 (2006); Waldman, Privacy Law’s False Promise, supra note 41. As Margot Kaminski explains, the emerging system of “binary governance” of privacy reflects the same trends; as Ari Waldman shows, the result is a system of increasingly widespread but increasingly performative compliance.45 45. Margot Kaminski, Binary Governance: Lessons from the GDPR’s Approach to Algorithmic Accountability, 92 S. Cal. L. Rev. 1529 (2019); Ari Ezra Waldman, Industry Unbound: Privacy, Practice, and Corporate Power (forthcoming in 2021) (manuscript on file with author). Although third-party intermediaries may have valuable roles to play in scaling up governance mechanisms, regulatory design for the networked information era must also include mechanisms for rendering such intermediaries accountable to public regulatory authorities. Put another way, there is a difference between delegating authority to entities that are also self-interested actors and deputizing those actors to conduct oversight activities on the public’s behalf. Finally, a regime of privacy governance needs to impose public transparency obligations on both the actors in networked information ecosystems and the regulators who oversee their operations. To do so effectively, legislation and implementing regulation need to specify those obligations in ways that anticipate the continuing gravitational pull of property thinking. A principal transparency tool used today, the Federal Freedom of Information Act, is riddled with exemptions that reinforce the de facto property logics that Part I described. Some shield national security and law enforcement operations from scrutiny, reinforcing the proposition that transfers of information into those domains effect a surrender of control. Others protect claimed trade secrets and more generally “confidential information” embodied in information-processing tools supplied by private contractors, reinforcing de facto rights to exclude that are understood to operate even against government users.46 46. Sonia Katyal & Charles Tait Graves, From Trade Secrecy to Seclusion, 109 Geo. L.J. (forthcoming in 2021). Emerging conventions for “binary governance” of personal data processing often repeat these errors, stopping short of requiring public-facing transparency about information-handling practices.47 47. See Kaminski, supra note 45. Honoring the public’s right to know requires a less deferential approach to the secrecy claims that have become endemic in the networked information era. Only two of the bills introduced in the 116th Congress—Rep. Eshoo’s (D-CA) Online Privacy Act and the Data Protection Act sponsored by Sen. Kirsten Gillibrand (D-NY)—come anywhere near this approach to privacy governance, and both also have fatal flaws. The Eshoo bill would create an independent digital privacy agency and give it robust enforcement powers, but would orient the agency’s rulemaking powers largely toward violations of a dramatically expanded array of individual control rights. The Gillibrand bill would create an independent data protection agency to oversee a range of “high-risk data practices,” including both profiling and processing of biometric data and other sensitive data, but would limit the agency’s rulemaking and enforcement powers by imposing conditions similar to those that currently constrain the FTC.48 48. Online Privacy Act of 2019, H.R. 4978, 116th Cong. (2019); Data Protection Act of 2020, S. 3300, 116th Cong. (2020), §§7(b), 8(b). One bill that remained on the drawing board during the 116th Congress—Sen. Sherrod Brown’s (D-OH) Data Accountability and Transparency Act—begins to reenvision public governance of personal data processing more thoroughly. It forbids certain operations using personal data, prohibits various forms of data-driven discrimination, and proposes an independent data protection agency with authority to enforce the prohibitions and to supervise the testing of automated decision systems.49 49. Data Accountability and Transparency Act of 2020 Discussion Draft, 116th Cong., 2020 Session (2020), https://www.banking.senate.gov/imo/media/doc/Brown%20-%20DATA%202020%20Discussion%20Draft.pdf [https://perma.cc/7JP7-LDG6]. Brown’s DATA Act also retains a suite of control rights and includes language curtailing federal enforcement authority that is modeled on the FTC Act. Its more innovative provisions, however, could form the kernel of a viable privacy governance regime for the networked information era. They merit genuine and sustained consideration. Remedies and Enforcement: The (false) choice between bad and worse A third important area of emerging consensus about the structure of privacy legislation involves enforcement mechanisms. Notwithstanding deep disagreement about whether to authorize private rights of action for violations, all parties seem to be working from the same unquestioned assumptions about what the universe of enforcement mechanisms includes. According to the conventional wisdom, there are two principal strategies available for pursuing information privacy violations: private remedial litigation initiated by affected individuals and public enforcement action initiated by agencies.50 50. For perhaps the clearest expression of this conventional wisdom, see Cameron F. Kerry, et al., Bridging the Gaps: A Path Forward to Federal Privacy Legislation, Governance Studies atBrookings (June 2020). Proposals to double down on one or both of those strategies tend to overlook the inconvenient truth that ex post, litigation-centered approaches have not proved especially effective at constraining Big Tech’s excesses. They also tend to overlook or downplay other possibilities that might prove more effective because they are better tailored to the scaled-up risks and harms that networked information flows create. To be clear, both private remedial litigation and agency enforcement proceedings can serve important expressive and normative functions. Public enforcement proceedings against those who flagrantly violate public mandates reassert the importance of the values those mandates express. In theory at least, affording adequate scope for such proceedings can demonstrate a public commitment to holding powerful, for-profit entities accountable to the citizens whose interests the laws were enacted to protect.51 51. See, e.g., Richard H. McAdams, The Expressive Powers of Law: Theories and Limits, 193–94 (2015) (discussing the signaling function of civil enforcement); Cass R. Sunstein, On the Expressive Function of Law 144 U. Pa. L. Rev. 2021, 2032 (1996) (considering the expressive function of law with or without accompanying enforcement activity). Affording individuals the ability to vindicate statutorily conferred rights in private litigation can also demonstrate such a commitment—at minimum, by compelling the defendant to provide an account of its actions. Additionally, depending on how they are structured, enforcement proceedings have the potential to validate important dignitary interests.52 52. Cf. Rachel Bayefsky, Remedies and Respect, 109 Geo. L.J. (forthcoming in 2021). At the same time, though, discussions about both the relative and absolute efficacy of litigation-centered enforcement mechanisms reflect magical thinking about litigation’s upsides. In recent decades, sustained assaults on standing to sue and on class-action eligibility and scope have drastically narrowed the feasibility horizon for litigation asserting harm to individual interests.53 53. See generally Daniel J. Solove & Danielle Keats Citron, Risk and Anxiety: A Theory of Data-Breach Harms, 96 Texas L. Rev. 738 (2018); Cindy A. Schipani & Terry Morehead Dworkin, Class Litigation After Dukes: In Search of a Remedy for Gender Discrimination in Employment, 46 U. Mich. J.L. Reform 1249 (2013). Cf. Deborah R. Hensler, Has the Fat Lady Sung? The Future of Mass Toxic Torts, 26 Rev. Litig. 883, 892 (2007) (“[W]hat distinguishes mass toxic tort litigation procedurally from virtually all other forms of mass tort litigation ... is that plaintiff and defense attorneys in mass toxic torts have not relied exclusively on class actions to aggregate cases.”) Efforts to recognize new information privacy rights eligible for vindication by private litigants largely pretend this is not the case. Public enforcement litigation, meanwhile, is chronically underfunded, and in recent decades, public agencies have largely acquiesced in the emergence of conventions for structuring consent decrees that delegate most oversight to private auditors and in-house compliance officers.54 54. See Cohen, supra note 37, at 186-93; Waldman, Privacy Law’s False Promise, supra note 41. Returning to the themes developed in Part I, the surprisingly unanimous faith in enforcement litigation—private or public—as a remedial lever reflects the cathedral’s lingering shadow, in two complementary ways. First, because enforcement litigation is predominantly atomistic in its identification and valuation of harms, it cannot effectively discipline networked phenomena that produce widely distributed, collective harms manifesting at scale. The mismatch is most obvious for private remedial litigation, which takes individual injury as the proper frame of reference even when claims are aggregated using class-action devices or processed together using newer multidistrict litigation mechanisms.55 55. See generally Julie E. Cohen, Information Privacy Litigation as Bellwether for Institutional Change, 66 DePaul L. Rev. 535 (2017). (This, it should be said, is not the privacy class action bar’s fault; they are simply following the path of least resistance.). Public enforcement, however, is scarcely better. Agency enforcement staff operating under significant resource constraints consider their targets carefully and select them for maximum impact, but an approach that singles out selected bad actors tends to validate the mainstream of current conduct rather than meaningfully shifting its center of gravity. To the extent that the prevailing approach identifies smaller actors outside the mainstream as enforcement targets, moreover, those actors simply have insufficient authority to catalyze upstream redesign of the networked processes, protocols, and interfaces within which they operate. Second, enforcement litigation tends to express the same bottom-up approach to governance of resources that characterizes property thinking generally, and so it has little to say about how violations ought to be remedied. Recall that according to property thinking, forcing cost internalization incentivizes property owners to do the right thing while leaving them appropriate discretion as to which inward-facing governance mechanisms to implement. As a practical matter, though, such efforts do not reliably produce lasting behavioral change unless they are paired with more specific mandates. To the contrary, and especially when the challenged behavior is both highly profitable and relatively opaque to outside observers, it empowers violators to treat the costs of occasional enforcement actions as operating expenses. Taken cumulatively, the results of the FTC’s privacy enforcement proceedings—including even the widely publicized contempt order against Facebook following the Cambridge Analytica disclosures—are consistent with that pattern. Over the past two decades, tech industry reliance on surveillance-based business models has only grown more entrenched.56 56. See generally Cohen, supra note 37; Shoshana Zuboff, The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power (2019); Shoshana Zuboff, The Coup We Are Not Talking About, N.Y. Times (Jan. 31, 2021), https://www.nytimes.com/2021/01/29/opinion/sunday/facebook-surveillance-society-technology.html [https://perma.cc/Z4BH-LS9Y]. The balance of this section briefly describes three mechanisms that might enable enforcement interventions to attain more lasting and far-reaching impact. There may be others; the point is not to develop an exclusive list but rather to challenge entrenched habits of thinking and encourage experimentation with strategies designed to target the design of networked processes, protocols, and interfaces and to scale up commensurately with the conduct at issue. Each of the suggestions described below has appeared in at least one proposed privacy bill, where it prompted inside-the-Beltway reactions ranging from derision to embarrassed silence. Not coincidentally, such proposals—paired with a more active approach to privacy governance along the lines described in Part II—likely represent the only realistic prospects for moving the needle on enforcement. An essential strategy for scaling enforcement authority involves leveraging gatekeeper power to demand and guarantee adherence to the design, operational, and monitoring requirements that public oversight processes have defined. For information businesses that qualify as online service providers, that would entail near-complete reversal of prevailing thinking about the appropriate extent of responsibility for the acts of third parties. As Rory Van Loo demonstrates, however, from a regulatory perspective, that approach is an anomaly; in many other industries, deputizing intermediaries to enforce appropriate standards of corporate behavior is vital and accepted.57 57. See generally Rory Van Loo, The New Gatekeepers: Private Firms as Public Enforcers, 106 Va. L. Rev. 467 (2020); cf. Lauren E. Willis, Performance-Based Remedies: Ordering Firms to Eradicate Their Own Fraud, 80 L. & Contemp. Probs. 7 (2017). By analogy, it is entirely rational to suggest deputizing online intermediaries to discipline smaller actors operating within information ecosystems that they have created and that generate enormous profits. Moreover, the presence of intermediary-based regimes in other fields gives the lie to the well-rehearsed contention that such regimes necessarily collapse into “censorship.” We have already seen some examples of how such a system might work. For example, it might prescribe standards for the data collection and exchange functions built into software developer kits, and it might set outer limits on optimization for networked, social spread of content via recommendation feeds. To be fully effective, scaled-up enforcement authority must be paired with strategies for scaling up sanctions against violators. The difficulties associated with crafting such strategies are well known and extend far beyond privacy. The very largest technology companies, however, have shown by their repeated, passive-aggressive flouting of enforcement decrees both in the U.S. and elsewhere that they may be uniquely immune to the incentives thought to be afforded by monetary fines on a conventional scale. The contempt sanctions levied against Facebook following the Cambridge Analytica disclosures are a case in point—the $5 billion fine, although by far the largest the FTC had ever imposed, represented only a month’s worth of earnings for the tech giant.58 58. Nilay Patel, Facebook’s $5 Billion FTC Fine Is an Embarrassing Joke, The Verge (July 12, 2019), https://www.theverge.com/2019/7/12/20692524/facebook-five-billion-ftc-fine-embarrassing-joke. As Paul Ohm has proposed, violations that produce order of magnitude effects require a commensurate response.59 59. Ohm, supra note 34, at 554-55. One often overlooked element of the public enforcement tool kit, with the potential to scale in a way rarely matched by ordinary civil fines, is disgorgement of the profits accruing from unlawful activity. The Supreme Court recently confirmed that federal courts have inherent equitable authority to order disgorgement even absent more specific authority.60 60. Liu v. SEC, 140 S. Ct. 1936 (2020). And several existing agencies, including both the FTC and the Securities and Exchange Commission, have disgorgement authority in certain kinds of cases.61 61. For now. See AMG Capital Mgmt. v. FTC, 910 F.3d 417 (9th Cir. 2018), cert. granted, 141 S. Ct. 194 (2020). Only two of the privacy bills proposed in the 116th Congress prescribe a disgorgement remedy.62 62. Data Protection Act of 2020, S. 3300, 116th Cong. (2020) (Relief available to be granted by a court or agency includes “rescission or reformation of contracts,” “restitution,” and “disgorgement or compensation for unjust enrichment.”); Online Privacy Act of 2019, H.R. 4978, 116th Cong. (2019) (Relief available also includes “rescission or reformation of contracts,” “restitution,” and “disgorgement or compensation for unjust enrichment.”). Current approaches to conceptualizing disgorgement, however, too often return reflexively to the atomistic and transactional approach to governance described in Part I. Activating either the inherent judicial disgorgement authority or existing agency disgorgement authority tends to require a standard of traceable economic injury that many complaints alleging information privacy harms cannot meet. To be most effective in the privacy context, disgorgement authority should be tied to violation of publicly defined design, operational, and monitoring requirements. Privacy legislation should clearly prescribe disgorgement as a remedy for such violations, and it should empower regulators to define—and justify to the public—mechanisms for attributing profits to lawbreaking and for calibrating recovery based on order of magnitude effects.63 63. See Ohm, supra note 34, at 554-55; cf. Samuel N. Liebmann, Note, Dazed and Confused: Revamping the SEC's Unpredictable Calculation of Civil Penalties in the Technological Era, 69 Duke L. J. 429 (2019). Last and importantly, rather than dissipating the benefits of disgorgement awards by distributing a few dollars to each affected consumer, it should specify that at least part of the amount recovered will be used to fund public oversight operations. Another underused element of the public enforcement tool kit is personal liability for senior executives and board members who thwart or undermine effective public privacy governance. As one option, privacy legislation might empower public authorities to pursue criminal sanctions against individual executives who deliberately violate applicable rules designed to preserve the integrity of important collective processes.64 64. See, e.g., Mind Your Own Business Act of 2019, S. 2637, 116th Cong. §1352 (2019) (requiring that CEO and chief privacy officer certify annual data protection reports, and imposing criminal sanctions for certain violations). As a practical matter, though, establishing the intent required for criminal culpability can be difficult within corporate contexts—in no small part because the same processes of legal endogeneity that undermine real reform also work to negate criminal intent when violations arise.65 65. See, e.g., James M. Anderson & Ivan Waggoner, The Changing Role of Criminal Law in Controlling Corporate Behavior (2014); H. Nejat Seyhun, The Effectiveness of the Insider-Trading Sanctions, 35 J. L. & Econ. 149, 153, 157–58 (1992); Developments in the Law: Corporate Crime: Regulating Corporate Behavior Through Criminal Sanctions, 92 Harv. L. Rev. 1227, 1367–68 (1979). A more promising approach to personal liability would borrow and adapt veil-piercing mechanisms from the corporate enforcement tool kit. Particularly where corporate privacy violators have adopted dual-tier ownership structures to preserve disproportionate voting power for founding “innovators” and venture capitalists—as is the case with most of the tech companies that are now household names66 66. Rani Molla, More Tech Companies Are Selling Stocks that Keep Their Founders in Power, Vox Recode (April 11, 2019), https://www.vox.com/2019/4/11/18302102/ipo-voting-multi-dual-stock-lyft-pinterest.—it makes doubly good sense to adopt penalties that target personal wealth accruing from participation in surveillance abuses, even when knowledge or intent regarding specific violations cannot be proved. Conclusion: Lessons for legislators Policymakers prize consensus, but at times of great economic and technological transformation, consensus can be a double-edged sword. The emerging inside-the-Beltway consensus on the shape of information privacy legislation is a case in point; it purports to map the road forward but promises only to excuse business as usual and further entrench systemic surveillance abuses. Drafting effective privacy legislation requires a starkly different approach. At this critical juncture in governance of networked information processes, it is urgently important to avoid the temptation to take the easy road. Governance institutions and techniques also can—and should—be sites of innovation. I have sketched an approach to designing public governance institutions capable of constraining networked processes that operate at scale, and have identified additional resources that interested legislators can consult. The 117th Congress has an opportunity to begin that process in earnest. Thanks to Lindsey Barrett, Kiel Brennan-Marquez, Erin Carroll, Jeff Gary, Woodrow Hartzog, Paul Ohm, Rory Van Loo, Ari Waldman, and participants in the Knight First Amendment Institute-L...
Privacy self-management and the issue of privacy externalities: of thwarted expectations, and harmful exploitation
1. Introduction This article examines the interdependent dimension of privacy and criticises the individualistic framework of notice and consent (hereafter ‘N&C’) through which one’s personal data is in practice protected. This framework is presented as problematic due to the way it obscures the role of data subjects other than the ‘main’ data subject (the user of the product or service, henceforth ‘the user’ of the ‘service’), and thereby prevents privacy externalities from being confronted and adequately addressed. ‘Externality’ is a term from the field of economics which designates the by-product of a business activity; it occurs when the production or consumption of a good or service by an agent imposes a cost or benefit on an unrelated third party (Tietenberg and Lewis, 2018, p. 26). A textbook example of the concept would be the activity of an industry which pollutes a water stream, generating profits for those actively engaged in the activity but also covertly impacting the health of locals. By extension, the concept of privacy externality refers to the inclusion of others’ personal data in the processing activity agreed to between the controller and the user, whereby costs are imposed on these third-party data subjects: the undermining of their privacy and of their right to data protection, as well as potential harm. For example, we routinely upload pictures of others to proprietary platforms such as Facebook. We disclose the genetic data of our whole family, together with our own, when we get DNA testing kits from companies such as MyHeritage. The discussions we have with our friends fuel the training of Amazon’s AI when they enter our Alexa-equipped ‘smart home’. None of the aforementioned individuals in practice benefits from adequate privacy protection, because the means we too often primarily rely on to ensure the protection of data subjects’ personal data (such as contract-like Terms of Service between user and service provider), allow the exercise of data protection rights to the user only (Solove, 2013). This article thus shows that, independently and in spite of one’s effort to manage it, one’s privacy and right to data protection can be fundamentally undermined by the behaviour of others; further, that this disclosure can be (and often is) exploited by data-hungry organisations whose business model is the insatiable extraction, accumulation and monetisation of personal data (Shoshana Zuboff’s surveillance capitalism (2015, 2019); see also European Data Protection Supervisor (EDPS) (2020, p. 5). The economics’ aspect of privacy externalities has hitherto often remained absent from the debate about the phenomenon. Indeed, the interdependent dimension of privacy, as well as the issue of privacy externalities, are being directly addressed in legal, policy and philosophical scholarship at least since the 2010s. Part of the contribution made by this article is the collection of relevant literature, which otherwise stands in isolated clusters and refers to a similar phenomenon using different concepts, such as: joint controllership, and privacy infringements (Helberger and van Hoboken, 2010; van Alsenoy, 2015; Edwards et al., 2019) or infringements of data protection law and networked services (Mahieu et al., 2019); collective privacy (Squicciarini et al., 2009) and collective action problems in privacy law (Strahilevitz, 2010); multi-party privacy (Thomas et al., 2010); collateral damage and spillover (Hull et al., 2011; Symeonidis et al., 2016); interpersonal management of disclosure (Lampinen et al., 2011); networked privacy (boyd, 2011; Lampinen, 2015; Marwick and boyd, 2014); interdependent privacy (Biczók and Chia, 2013; Symeonidis et al., 2016; Pu and Grossklags, 2017; Kamleitner and Mitchell, 2019); peer privacy (Chen et al., 2015; Ozdemir et al., 2017); multiple subjects personal data (Gnesi et al., 2014); privacy leak factor, shadow profiles and online privacy as a collective phenomenon (Sarigol et al., 2014); privacy externalities (Laudon, 1996, pp. 14-6; MacCarthy, 2011; Humbert et al., 2015, 2020; Symeonidis et al., 2016; Choi et al., 2019), especially as compared to externalities in the context of environmental pollution (Hirsch, 2006, 2014; Hirsch and King, 2016; Froomkin, 2015; Nehf, 2003; Ben-Shahar, 2019); 1 genetic groups (Hallinan and De Hert, 2017); or sociogenetic risks (May, 2018). 2 While the phenomenon has thus been addressed in scholarly and policy settings already (although often with a different goal or scope), the present article frames it in a way which puts into light an important aspect hitherto mostly unaddressed. This aspect is the financial incentives and the exploitative dynamics behind these disclosures of others’ data; it is not only a major factor in making the phenomenon ethically problematic, it is also the very reason the phenomenon is perpetuated. These incentives and dynamics give competition and consumer-protection ramifications to this data protection issue, and failing to pick up on them has hindered scholars and authorities from adequately grasping and addressing the problematic phenomenon. This concern about externalities is moreover different from more traditional data protection issues of inappropriate disclosure such as leaks and hacks: privacy externalities are not only about bad personal data management, but also about impossible personal data management. Privacy cannot adequately be managed alone, as it is in some aspects necessarily an interdependent matter. Whereas this is a neutral fact about the world, the way we (do not) deal with it is problematic, because individual users and controllers take advantage of it and allow costs to be imposed onto others, undermining their privacy. This is even more deeply problematic as the current data ecosystem (which generally harvests every bit of data for monetisation or exploitation) has been designed in a way that often amplifies the negative nature of privacy externalities. Framing the issue as one of privacy externalities and exploitation, instead of as the mere downside of certain technologies, is moreover important if we want to have an adequate philosophical, societal and juridical debate on the issue of privacy externalities, because it allows us to recognise the responsibilities upon which the relevant parties fail to act. In this article, I begin by introducing the ideal of privacy self-management which, in an ecosystem that heavily relies on consent as the legal basis for data processing, is de facto commonly imposed onto data subjects through the ‘Notice and Choice’ (N&C) framework; this self-management ideal is contrasted with the reality of the interdependent dimension of privacy (section 2). I argue that improperly taking this dimension into account allows for the creation of privacy externalities, whereby others inconspicuously and unfairly pay (part of) the price for others’ benefit; moreover, I argue that this is the term most appropriate to conceptualise and analyse the phenomenon (section 3). Building upon the concepts and concrete examples discussed in the existing body of literature collected, I then attempt to draw a systematic and comprehensive picture of the phenomenon, analysing the various forms it takes (section 3.1). Finally, I briefly explore two possible ways of addressing the issue of privacy externalities (section 3.2). In terms of methodology, this article does a conceptual analysis of a concrete issue (privacy externalities), combining theoretical insights from the field of economics with knowledge of data protection legislation and real-life examples. This analysis responds to, and is informed by relevant works in the existing literature. 2. Privacy self-management and interdependent privacy The 2016/679 General Data Protection Regulation (GDPR) states (art. 5) that personal data shall be (a) processed lawfully, fairly and in a transparent manner in relation to the data subject, (b) collected for specified, explicit and legitimate purposes and not further processed in a manner that is incompatible with those purposes, and (c) adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed. While additional principles are important in the European data protection regulation, these ‘lawfulness,’ ‘fairness,’ ‘transparency,’ ‘purpose-limitation’ and ‘data minimisation’ principles are its pillars. To ensure the lawfulness of their processing, however, the majority of processors in practice rely on only one of the multiple grounds available: consent. Consent, as an expression of individual autonomy, is accorded great value in Europe and particularly in the field of data protection, with the consequence that some controllers over-rely on it or use it to (erroneously attempt to) legitimise routine or even disproportionate data processing (for an in-depth analysis of this topic, see van Alsenoy et al., 2013, pp. 4-6). In consequence, the framework of N&C (especially through online privacy notices) has sprung forward as the de facto preferred means by controllers to ensure the transparency of their practices and to collect the consent of data subjects (see also Barocas and Nissenbaum, 2014; Hull, 2015; Mantelero, 2017, p. 72). In practice, this, together with a widespread business model relying on the collection and monetisation or exploitation of personal data (EDPS, 2020, p. 5; Holloway, 2019), has led to an individualistic system of personal data protection where the consent of individuals is repeatedly queried for a multitude of purposes, whereas in theory, a data subject would not necessarily have to micro-manage their privacy as much as they currently do. This means that privacy management often takes the contractual form of two parties agreeing about the processing (the collection, use, disclosure, etc) of the data subject’s personal information (personal data), in exchange for a service offered by the controller. It is furthermore reflected in one of the currently dominant legal and philosophical definitions of privacy, which is: the relative control over the ways and the extent to which one selectively discloses (information about) oneself to others. 3 This (over-)reliance on consent has the impractical effect that the privacy of individuals is only protected per individual, i.e., it is achieved in an individualistic fashion, where data subjects have to (and are expected to) micro-manage their privacy (Whitley, 2009; Solove, 2013; van Alsenoy et al., 2013; Mantelero, 2014; Taylor et al., 2017, p. 6). In addition to the burden of self-management it creates for individuals, it will become clear that this individualism is also problematic because it obscures the fact that, in many instances, the data subject’s choice to consent in fact impacts other data subjects, and thereby pre-empts these third parties’ own consent. Indeed, privacy has both a collective and an interdependent dimension to it. To see this, one has to understand the scope of the GDPR’s definition of personal data, which is any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person. (GDPR, art. 4.1) This definition is extensive, and protects data subjects whenever information about them is processed. Crucially for this article, the scope of this definition also entails that one’s personal data may also be another’s personal data. When I upload material on a website, it is related to me (and, therefore, is my personal data) in that it is uploaded by me, and about me—two relations of ‘identifiable relatedness’ arguably relevant for constituting personal data. Accordingly, when I upload content clearly about someone else (henceforth a ‘third-party subject’), it is both my personal data and theirs—as long as they are identifiable—because, although it is uploaded by me, it is about them. These relations can be referred to as ‘causal agency’ and ‘personal relevance.’ 4 If they do not also have a ‘causal agency’ relation to it, controllers rarely (if ever) provide N&C or other rights to data subjects who have a ‘personal relevance’ relation with the material (data) processed. For instance, Facebook has a portal dedicated to the provision of their personal data to Facebook users; yet, this access is restricted to “information you’ve entered, uploaded or shared [yourself].” 5 This is incoherent, when one realises that the range of our personal data processed (often knowingly) by Facebook exceeds the data we have provided ourselves. This also means that a narrow understanding of personal data is often applied, and therefore that many data subjects’ right to effective data protection is unfairly restrained. The distinction made between the two kinds of ‘identifiability’ is important, because it allows me to identify and frame a major obstacle to privacy self-management: the interdependent dimension of privacy, i.e., the idea that in a networked world, one’s privacy depends at least partly on others (and on the choices they have themselves made regarding their own privacy). While I may decide what information about myself I give to the world (and to individual controllers), others may decide it for me as well; I am thus at least partly dependent on others for retaining my informational privacy. This interdependent dimension is an obstacle insofar as privacy is framed as an individualistic matter (through the N&C mechanism that is the favoured tool of many controllers to achieve appropriate data protection), an aspect of one’s life which is self-(sufficiently-)manageable. 3. Privacy externalities As mentioned earlier, an externality is a cost or benefit imposed on a third party who did not choose to incur that cost or benefit, and which is the by-product of an activity (such as the production or consumption of a service). Externalities often occur when the equilibrium price (i.e., the price when supply and demand are balanced) for the production or consumption of a service does not reflect the true costs or benefits of that service for society as a whole (see Heath, 2007). In the context of informational privacy, this article argues that people’s decisions to use certain services, or to share their personal information, may allow the data controller to know more about them, but also about others. To the (limited) extent that people can be said to ‘pay’ for a service with their data, 6 part of the price is actually also other people’s data. That is, the full costs of the production or consumption of the service include the impact on others’ privacy and the (dis)utility resulting therefrom—a form of latent harm peculiar to the 21st century (Calo, 2011; Laudon, 1996, pp. 16-17; see also Article 29 Data Protection Working Party (WP29), 2014, p. 37; see also van Dijk et al., 2016, on “increased risks to rights”; see also Ben-Shahar, 2019, on how these externalities “undermine and degrade public goods and interests”); 7 the problem is that this is neither transparent nor accounted for in the transaction between user and service-provider. Hence the term privacy externalities. Referring to the phenomenon as privacy externalities allows me to capture a crucial aspect of the issue: the cost of services in the digital, hyperconnected era and, furthermore, the externalisation of these costs. While other terms used to refer to the phenomenon (see section 1) conceptualise it as a mere side-effect of certain digital practices, using the concept of externality brings to light the fact that this side-effect is not neutral, i.e., that users and/or controllers are not indifferent to it (whether they are conscious of it or not). On the one hand, by not investing as much as they should in the design of their service, and by not addressing all their obligations toward (third-party) data subjects, controllers can de facto dump costs and responsibilities onto the user (such as the duty to notify the user’s peers of the data processing, in the case of smart homes), 8 thereby saving resources. On the other hand, by not carefully choosing privacy-respecting services (when that is possible), and/or by not taking adequate precautions for others’ privacy when using these services, users may often themselves be dumping costs onto third-party subjects: the infringement of their privacy, increased risks to their rights, potential harm, as well as the time and energy required for taking the appropriate measures (when possible). 9 This means that privacy externalities can cause distortions in the production and consumption of social goods, by making the perceived price of a service lower than the actual total cost, and therefore more attractive than it should be. Moreover, because some service providers’ business model relies on the accumulation and monetisation of as much data as possible (Zuboff, 2015; EDPS, 2020, p. 5; Holloway, 2019), privacy externalities are costs for third-party subjects not only in the sense that they expose the latter and undermine their rights to privacy and data protection, but also in the sense that they make way for profit-driven controllers to (illegally) exploit this data for their own benefits at the expense of the data subjects (see esp. Court of Justice of the European Union (CJEU), 2019, para 80, and Ben-Shahar, 2019, p. 115; see also Humbert et al., 2020, on service providers as “adversaries”). For instance, exploiting the externalities generated through the sharing of users’ contact data is part of Facebook’s massive targeted prediction and advertising endeavour, which is how the company makes most of its profit (Venkatadri et al., 2019). Similarly, direct-to-consumers genetic testing services which offer to predict medical risk factors and to reveal ancestry or genealogy actually make profit through reusing the data for medical research, profiling, and offering their services to law enforcement (EDPS, 2020, pp. 5, 25). Thus not only does the flawed price of the relevant services allow controllers to save resources, it also leads users to consume more of these services, feeding the controllers even more data to extract value from. Realising the potential residing in these troves of data, some rogue controllers may even intentionally design the structure of their services so as to encourage and capture such privacy externalities, leading powerless, careless or unaware users to provide the system with not only data about themselves, but also about others. These passively and actively beneficial aspects of privacy externalities are thus, effectively, incentives for the perpetuation of the phenomenon. They are crucial to understanding and tackling it, and their absence from the existing literature on the topic is therefore regrettable. Moreover, in addition to not picking up on the economic aspect of the phenomenon, the authors of the works cited in section 1 often only addressed it in relation to a unique context—such as social networks or databases. Similarly, when the issue was addressed at court- or policy-level, the kind of privacy externalities taken into account did not necessarily reflect the whole range of the phenomenon (see the categories discussed below). This substantially limited the scope of both their analysis and the solutions they sought, with for instance the CJEU and the WP29 focusing on the ‘disclosure to [a certain number] and [a certain kind] of peers’ as a criterion determining the wrongness and illegitimacy of the processing (CJEU, 2003; WP29, 2013, p. 2). Further, and especially as these works are scattered in ‘clusters’ that do not necessarily refer to each other, 10 the absence of the broader perspective in these studies would lead one to believe, at first sight at least, that these works (or clusters) address a different issue from one to the other. By abstracting the contingencies from each case and putting them all under the umbrella of privacy externalities, however, one can identify the different clusters, and it becomes apparent that there actually is a whole body of literature on the phenomenon, instead of scattered studies of different phenomena. Still in contrast to the works referenced above, the present article brings about the clear distinction between two separate things that their authors often discuss inextricably interwoven, and unites their work as revolving around this distinction. This distinction is between the interdependent (or networked, interpersonal, collective, social) aspect of privacy—which is a necessary fact about the world—and the phenomenon of privacy externalities (or spillovers, collateral damages, disclosures, leaks)—which is partly contingent on controllers’ and users’ decisions. This distinction, and especially this contingency (i.e., the fact that it depends on other factors, such as default privacy settings or on the way a service is used), is important when (if) the responsibility of the various actors is addressed (section 3.2). We may thus start to see the broader picture, and to focus on the cause of the problem instead of its symptoms. This article argues that privacy externalities are mostly the result of the necessarily interdependent dimension of individual privacy being coupled with economic incentives to externalise certain costs (and to exploit and monetise them further in complex and obscure ways when possible). The phenomenon is widespread and may produce or amplify future harm (including intrusive predictions and advertising), at least when the issue is systemic and the externalities accumulate. Besides, the phenomenon violates individuals’ right to privacy and threatens the ideal of privacy self-management itself, independently from whether it produces concrete (latent, tangible or intangible) harm or not, and independently from whether it is exploited by the controller or not; it is yet another risk to the rights of data subjects. Like Martin Tisne (2019, n.p.) succinctly puts it, “we are not only bound by other people’s data; we are bound by other people’s consent [… and] this effectively renders my denial of consent meaningless. […] The issue is systemic, it is not one where a lone individual can make a choice and opt out of the system” (see also Barocas and Nissenbaum, 2014). This practice, whereby one’s consent is overridden, should receive the attention it deserves, especially in light of the expectations of privacy self-management. Disclosure of others’ personal data through one’s activities can be repetitive, commonplace, extensive and substantial, and is thus a serious issue. Building upon the concepts and examples discussed in the existing literature, I will now have a closer and systematic look at the various forms privacy externalities can take. 3.1. Four different kinds of disclosure Privacy externalities can take multiple forms, each problematic in their own way. Once abstracted from their individual contingencies, they can be separated into the following four (possibly overlapping) categories: Direct disclosure: data is revealed about subject A when subject B discloses data about subject A. Indiscriminate sensing: data is revealed about subject A when subject B reveals data about subject B that was formed through an indiscriminate process of capture, and which therefore included data about subject A alongside the data of subject B. Fundamentally interpersonal data: data is revealed about subject A when subject B reveals data about subject B, which necessarily is also data about subject A. Predictive analytics: subject B discloses data about subject B, from which the data controller is able to infer or predict more data about subject B as well as about subject A. The difference between categories (2) and (3) is that in the former, the interpersonal data (the term used here for data which is about more than one subject) is only contingently interpersonal, whereas in the latter it is necessarily so. In the former, the data could have been only about the user, if she had been cautious for instance; that is not an option in the latter category. The distinction becomes clearer with examples from each category: Direct disclosure: as long as it is digitally-recorded, any activity that consists in explicitly discussing about someone counts as revealing that person’s personal data, and thus as an activity relevant to interpersonal privacy. This includes blogging about people (Solove, 2007, p. 24); talking about them and posting pictures of them on social networks (Wong, 2009, p. 143 et seq.; van Alsenoy et al., 2009, p. 70; Helberger and van Hoboken, 2010; College Bescherming Persoonsgegevens (‘CBP’), 2007, pp. 12-13; Belgisch Commissie voor de Bescherming van de Persoonlijke Levenssfeer, 2007, pp. 21-22); outing a sexual preference online, broadcasting a traumatic experience, public shaming or posting ‘revenge porn’ (van Alsenoy, 2015); or “tagging” others (see Privacy International, 2019 about the app ‘TrueCaller’). Beside this, category 1 also involves directly handing over other people’s data to the data controller, like when Facebook apps ask the user to access her friends’ list and their data (Besmer and Lipford, 2010; Hull et al., 2011; Bizcók and Chia, 2013; Symeonidis et al., 2016, Facebook Inc., 2018). Moreover, embedding a Facebook “Like” button into one’s personal website (CJEU, 2019, para 76-7) de facto means handing over the personal data of visitors to Facebook, and similarly for other buttons and third-party services allowing behavioural targeting and user analytics (Mahieu et al., 2019). Indiscriminate sensing: recording one’s voice or environment often also implies indiscriminately recording others. Sensors capture all the available data of a given category (e.g., sound or image) within a perimeter, and do not discriminate between consenting and non-consenting data subjects. Therefore, the following activities will also capture the personal data of other people who may neither be aware nor capable of resisting the invasion of their privacy: uploading pictures of crowded places on social media; using a drone or Google Glass (van Alsenoy, 2015; EDPS, 2014 a); driving someone in one’s connected car (EDPS, 2019b, p. 3) or just driving a self-driving car around; ‘Netflix & Chilling’ in front of a smart TV; relying on a Ring doorbell (Herrman, 2020); using ‘voice assistants’ 11 or ‘smart’ speakers in one’s home (EDPS, 2019a, p. 3). Recording events in sound or image can be a sensitive practice, because many personal aspects of one’s and others’ life can be thus made available to data controllers, including sensitive data like political opinions, religious beliefs, or health data (Vallet, 2019). This data can moreover be automatically ‘mined’ by image-processing, voice-processing, and facial-recognition software. This category is quite broad, and includes CCTV (ICO, 2017; CJEU, 2014b); Internet of Things objects; or smart homes (see Kitchin, 2014b). Fundamentally interpersonal data: there are some kinds of data which necessarily constitute or reveal personal data of multiple persons. A striking example is genetic data: giving rights to a data controller to process your genetic data not only affects you and your privacy, but also potentially countless individuals to whom you are related—knowingly or unknowingly (Chadwick et al., 2014; Olejnik et al., 2014; Hallinan and De Hert, 2017; Taylor et al., 2017, p. 9; Erlich et al., 2018; May, 2018; Molteni, 2019). Because certain genetic traits are necessarily shared with family members, it suffices that a single person undertakes such an analysis for a kind of ‘family-wide sharing of personal data’ (i.e., a generational data breach). Other practices involving such interpersonal data include telecommunications (where the metadata reveals at least the identity of correspondents and the frequency of calls); the use of certain email providers (Dodds and Murphy, 2018; Ben-Shahar, 2019, p. 115); or the use of a shared system (such as smart grids, see McDaniel and McLaughlin, 2009). Finally, the category of fundamentally interpersonal data also includes relational data (Jernigan and Mistree, 2009; boyd, 2011; Backstrom and Kleinberg, 2014; see also the activity of address book sharing described in section 3.2.1), but also data about groups (such as households or neighbourhoods) (Taylor et al., 2017). Predictive analytics: when enough people disclose ample information about themselves, data controllers (especially data brokers) are able to understand the relation between having a given trait and a specific characteristic. For example, there is a correlation between, on the one side, buying felt-pads to prevent one’s furniture from scratching the floor, and on the other side paying one’s bills on time (Duhigg, 2009). When correlations like these have been found (through mining massive troves of data), the small, seemingly-insignificant pieces of information that even prudent people disclose (willingly or not) will reveal more data about them, whether they like it or not (Barocas and Nissenbaum, 2014, the “tyranny of the minority”; Choi et al., 2019, p. 8, Wachter and Mittelstadt, 2019). This is the case of the ‘dynamic’ groups from profiling categories, ‘Big Data’ analytics, predictive analytics and recommendation systems (Vedder, 1997; boyd et al., 2014; Mantelero, 2014, 2017). 12 These four categories and the examples provided show how important and diverse the cases are where one’s behaviour can negatively impact (the privacy of) others, and thus that the issue at stake here is not a rare or minor one. Each of the non-sensitive pieces of data that are thereby processed may seem innocuous on their own; however, not only does their processing remain an encroachment on and increased risk to third-party subjects’ fundamental rights, but when the phenomenon is widespread, the aggregation of all its instances will worsen its potential to do harm. Furthermore, even the smallest disclosures are significant, due to the possibility of the data being exchanged with others (such as data brokers, see Symeonidis et al., 2016; Choi et al., 2019, p. 8). Finally, in some cases (such as with biometric or genetic data) the data can be very sensitive, and the harm brought by the disclosure can be lifelong. Figure 1: Notification from the Facebook Messenger app requesting access to the user’s contacts 3.2. Whose responsibility? Different categories of privacy externalities will plausibly require different coping strategies; for instance, categories 1 and 4 seem to be unavoidable, to a certain extent, and would motivate a mitigating strategy rather than a prevention strategy. It is out of the scope of this article to solve the issue of privacy externalities; however, what the article can still do before closing, is briefly exploring two promising paths. The common denominator to the most problematic kinds of privacy externalities is the perpetuating force behind them, i.e., the passive and active benefits of externalities—respectively: dumping costs, and (the potential for) exploiting the third-party subjects’ data. Tackling these incentives should be at the heart of any response to the phenomenon. However, it should be noted that while the active benefits are enjoyed by data controllers alone, the passive ones (cheaper prices, less effort required, etc) are enjoyed by both the controllers and the users. While focusing on data controllers is therefore the logical place to start (and thus the first path examined), the roles of users should not be overlooked. 3.2.1. Enforcing data protection by design and by default The controller often plays an important role in the generation of externalities. For instance, some controllers offer services through which the acquisition of the personal data of the subject’s peers is requested, even though such services could do without it. The comparison between messaging apps Facebook Messenger and Signal illustrates this well. Messenger asks the user to (consent to) upload her contacts to Facebook’s servers, and to do so continuously (see figure 1). Facebook thus stores internally the contacts’ data, with the ensuing function creep Facebook is notorious for (Gebhart, 2018; Venkatadri et al., 2019). Signal, on the other hand, periodically sends the user’s contacts’ phone numbers to its servers in truncated, cryptographically-hashed form; it then identifies the overlap (i.e., the user’s contacts who also use Signal) and indicates this overlap on the user’s device exclusively, after which the server discards the information it received about the user’s contacts. 13 In general, even if only limited data, such as the nickname and a phone number, were disclosed for each contact in the user’s list, it would remain a potentially fruitful acquisition for the controller, as the widespread disclosure by users of their contact list would allow the controller, if it were as privacy-invasive as Facebook is, to identify the overlapping contacts in users’ phones, create network maps and start building ‘shadow profiles’ about non-users (WP29, 2009, p. 8; Sarigol et al., 2014; boyd et al., 2014; Levy, 2020, p. 222). Even solely knowing about this network of relations is valuable to the data controller, based on homophily—the tendency people have to interact with others who are similar to them. Homophily can be relied on to infer the “ethnicity, gender, income, political views and more” of people based on their communication networks (Caughlin et al., 2013, p. 1; see also Sarigol et al., 2014; Garcia, 2017; Jernigan and Mistree, 2009 (the “Gaydar”)). Thus, my ability to remain under Facebook’s (or others’) radar is heavily undermined by other individuals’ seemingly innocuous actions, which not only disclose information about them, but also (foreseeably) about me—even if I am not a Facebook user myself. This is not the case for data subjects using Signal. While Facebook in this case is invasive by design, Signal follows the approach of Data Protection by Design and by Default (DPbDD) which requires (GDPR art. 25) taking technical and organisational measures to (a) implement data-protection principles in an effective manner and to (b) ensure that only personal data which are necessary for each specific purpose of the processing are processed. DPbDD forces complying controllers to take the necessary steps to prevent, contain and mitigate the privacy externalities that might result from (the way they offer) their services. As such, the strength of DPbDD is that it is a solution generic enough to be applied to privacy externalities beyond messaging services (i.e., to “handle various data types and adversaries” (Humbert et al., 2020, p. 33)). For instance, Facebook incorporated some mechanisms to reduce privacy externalities on its platform, such as requiring the peers’ assent before a user can tag them in a picture or make them appear on her Facebook wall. Another example of useful DPbDD is found in clinics, where there are legal and other mechanisms governing conduct when genetic information about an inherited disease is relevant to the tested person’s relatives, or for cases where a diagnostic incidentally indicates misattributed paternity. While DPbDD requirements are specified in the GDPR and are thus the remit of data protection authorities, privacy externalities hitherto persist nearly unchallenged (perhaps due to these authorities’ lack of adequate funding, see Ryan and Toner, 2020; Satariano, 2020). In light of the harm certain practices can cause to third-party subjects, it could be argued that other authorities, especially consumer-protection authorities, should take data protection issues more seriously into consideration (without prejudice to the data protection authorities' powers) (on this, see Rhoen, 2016; see also EDPS, 2014b). Further research is needed to explore the extent to which this is possible; either way, the idea is that the stricter enforcement of DPbDD requirements—especially for services that seem to be invasive by design (rather than merely not designed with privacy in mind (see Helberger and van Hoboken, 2010, p. 106; see also CJEU, 2019, para 80))—could efficiently address part of the privacy externalities (van Alsenoy, 2015, p. 32, Edwards et al., 2019), i.e., the part where controllers are otherwise incentivised to dump certain costs and obligations onto the user and third-party subjects. One should be held accountable when one facilitates risks and harms for the peers of the users of one’s services; inaction is unacceptable, even more when one is profiting from this inaction, and taking advantage of the issue should be a no-go. Yet, the whole issue cannot be averted through the enforcement of these DPbDD requirements alone, because the user often plays an important part in the creation of externalities (through the way they use certain technologies, or the invasive practices they opt-in for), and because the issue can sometimes be most effectively and cost-efficiently addressed by users themselves. 14 The question is, in the current state of affairs, how much can we rely on individuals to adequately internalise the costs of their behaviour? This question leads us to a second, arguably more intricate, way out: the framework of joint controllership. 3.2.2. Joint controllership To illustrate the need for this complementary strategy, let us take the case of the smart home. A smart home is a data-driven premise which necessarily monitors all its occupants to provide its services, since its sensors most of the time cannot distinguish between the user and her relatives or visitors. In such a scenario, it is inevitable that the service will generate privacy externalities, and it is unclear whether thorough DPbDD would adequately prevent or mitigate them all. In essence, when multiple natural or legal persons determine the purpose and means of processing of the personal data, under the GDPR they are joint controllers and each is responsible for the part of the processing that it controls, to the degree it has control over it (see GDPR art. 26). Following European jurisprudence (CJEU, 2018) and guidance from the WP29 (2010, p. 18), it appears that “[i]nfluencing the processing (or agreeing to the processing and making it possible) [is] enough to qualify as determining both the purposes and the means of that processing operation” (Mahieu et al., 2019, p. 95). If the user may indeed be considered a joint controller in such cases (but we will see shortly that this claim may be contested), privacy externalities would be internalised (or their negative impact reduced) insofar as the user would be legally responsible for any inadequate processing of her peers’ personal data, and would hence be incentivised to take measures to avoid such unlawful processing—such as giving appropriate notice to visitors, or turning the smart devices off before they enter the premises. 15 However, important uncertainties remain regarding how this framework is to be applied, which raise substantial doubts as to the extent to which joint controllership could form (part of) the solution to the issue of privacy externalities. They are briefly listed below: 1. A first issue is that “the framework for assigning responsibilities to different stages of processing and different degrees of responsibilities is underdeveloped; there are no guidelines for assigning specific responsibilities to specific ‘stages’, no clear principles to determine different ‘degrees of responsibility’, nor criteria to connect particular consequences (enforcement actions) to particular levels of responsibility” (Mahieu et al., 2019, p. 99). That is, joint controllership as an effective framework of governance might not be mature enough yet, for this specific context at least. 2. A second issue comes from the GDPR’s ‘household exemption’, which states (Recital 18) that the GDPR “does not apply to the processing of personal data by a natural person in the course of a purely personal or household activity and thus with no connection to a professional or commercial activity,” though it “applies to controllers or processors which provide the means for processing personal data for such personal or household activities”. 16 When it comes to its application to privacy externalities, recent judgements from the CJEU (2003, 2014a, 2014b, 2018, 2019) advance criteria for determining whether data subjects using certain services should (a) be considered joint controllers and (b) benefit from the household exemption. The criteria put forward in these judgements would exclude many of the privacy externalities discussed above, 17 but not all externalities would be dismissed: depending on the weight accorded to a criterion endorsed in the Fashion ID CJEU ruling (2019, para 80: that the “processing operations are performed in the economic interests of both” parties), all the externalities passively beneficial to both the user and the controller would be admissible. Furthermore, the active exploitation (in various forms) of third-party subjects’ data by controllers, which is a crucial component of the privacy externalities that are most problematic, may also mean that the household/personal activity actually often has an important connection to a commercial activity (or at least that the distinction between personal and commercial is blurred), and may thus not benefit from the exemption (see also WP29, 2017, p. 8). 18 For a more in-depth discussion of privacy externalities, joint controllership and the household exemption, see also De Conca, 202 0. 3. A third issue to be considered comes from the burden of data protection and privacy (self-)management, and from the complexity of being a controller. The GDPR’s framework of joint controllership was primarily intended to adequately divide tasks and responsibilities between controllers of organisations—it was not intended to make private individuals take on the burden of being a data controller (see OECD, 2011, pp. 27-28). Being a controller entails legal duties and requires thorough understanding of both the legal landscape and the technicalities of data processing; the framework of joint controllership could hence plausibly be too burdensome in practice to be realistically applicable to private individuals (on this issue, see Helberger and van Hoboken, 2010, p. 104; van Alsenoy, 2015, pp. 6, 24, 28; Edwards et al., 2019). And let’s not even discuss the increased strain on data protection authorities’ limited resources that this solution would entail (see Ryan and Toner, 2020). 4. Finally, if, as the term ‘privacy externalities’ suggests (as well as the analysis of the incentives behind the phenomenon), this data protection issue can be linked to the context of externalities in environmental pollution, then there may be valuable policy lessons to learn from the latter field. This is what Omri Ben-Shahar (201 9) does, as he frames privacy externalities as “data pollution” (see also Hirsch, 2006, 2014; Froomkin, 2015; Hirsch and King, 2016). However, a central element of his argument is that, just like in environmental protection, “[t]he optimism that contracts and behaviorally-informed choice architecture would help people make wise data sharing decisions and reduce data pollution is fundamentally misplaced, because private bilateral contracts are the wrong mechanisms to address the harm to third parties” (2019, p. 108). He adds that “[i]t is not people who need to be protected from a mesh of data-predatory contracts; but rather, it is the ecosystem that needs to be protected from the data sharing contracts that people endlessly enter into” (ibid). If this is right and the analogy with environmental protection holds, then joint controllership will be inadequate to solve privacy externalities, and DPbDD (as part of a wider data protection ex ante package, which Ben-Shahar includes within the promising solutions he analyses) is the way to go. 19 4. Conclusion Many people remain to this day oblivious to the fact that ‘free’ services online only mean ‘without a monetary cost’, and that they actually ‘pay’ (to a limited and imperfect extent) with their data, i.e., by providing information (presumably about themselves) and agreeing that it be leveraged, in particular for intrusive advertising, prediction services or research. However, even those who realise this may not realise that it is not just their data they give away: it is often also the data of others. This ‘cost’ that is imposed on others, the article argued, is a form of disclosure most adequately conceptualised as privacy externalities. This article has demonstrated, in concord with existing literature, that one’s privacy is sometimes dependent on others—that is, that there is an interdependent aspect to individual informational privacy. This dimension makes it fundamentally impossible for a data subject to be fully in control of her personal data, despite such expectations. Part of the issue is that, in contempt of other important elements and legal bases in the GDPR, the protection of personal data nowadays still largely relies on consent. This happens through an individualistic mechanism of N&C, whereby only the data subject in direct relation to the controller providing the service is consulted, even if she will foreseeably also provide the personal data of other data subjects as part of the service. This individualistic framework obscures the possibility that one’s peers might need to be consulted, or that measures should be taken to mitigate the collateral processing of their personal data, for example. However, because the existing literature has often conceived of the issue precisely in this sense—that is, as a collateral damage, a neutral side-effect—the important dynamics behind the phenomenon have hitherto been poorly highlighted, if at all. The advantage of talking of interdependent privacy, and of taking the economic lens of externalities, is that it allows us to uncover the unethical incentives perpetuating the phenomenon. These are, first and foremost, the passive benefits of dumping costs on others: data controllers on users, and users on their peers. The savings realised are the time, resources and energy that would otherwise be invested in: designing a product of appropriate quality; putting in place legal and other mechanisms governing appropriate conduct in case externalities are created; due diligence; or taking steps to mitigate the externality. As a result, the services offered by data controllers can be offered for cheaper than if the appropriate efforts had been taken to ensure their quality—something which may distort the market by increasing the production and the consumption of these lower-grade services, at the expense of services of better quality (the price of which reflect better their true costs). The negative externalities resulting from the use of these cheaper services are the invisible price for these users’ peers. Concretely, these externalities are the unlawful processing of the peers’ personal data, the increased risks to their rights that result from it, as well as possible latent, tangible or intangible harm. Notwithstanding the risks and harms that result from the ‘passive’ benefits from externalities, additional risks and harms arise when some data controllers also actively create and/or harvest privacy externalities. In a hyperconnected world marked by surveillance capitalism, to rogue data controllers the privacy externalities are only a bonus—a bonus that further subsidises their cheap (or ‘free’) services. However, when the externalities become a feature rather than just a bug, their inexcusable exploitation undermines even further the data protection rights of countless unaware data subjects. This is highly problematic, both ethically and legally, and should be addressed by data protection, but also perhaps by competition and consumer-protection authorities. I briefly pointed toward two possible solutions, marking a preference for the path of better enforcement of data protection by design and by default. This article has furthermore served the goal of drawing together and listing the abundant and diverse scholarly (and policy) works on the topic. Pertaining to different fields and jurisdictions, using different terms to conceptualise a similar phenomenon, or simply not referring to related publications, the existing literature can be found in clusters that do not make reference to each other. The two lists found in Sections 1 and 3.1 can therefore be used to connect, learn from, and avoid repeating what has already been expressed. This article, however, does no literature analysis, comparison or evaluation of this existing body of works and of the solutions (if any) each put forward. What it does, besides framing the phenomenon in a particular way and scrutinising the elements revealed under this particular light, is using the different examples and conceptions of privacy externalities discussed in this body of works to draw a holistic picture of the phenomenon and of the four different forms it can take—something which had not been done before and which is indispensable to fully understand privacy externalities, and hence to appropriately address them. References Article 29 Data Protection Working Party (‘WP29’). (2009). Opinion 5/2009 on Online Social Networking (WP163). https://ec.europa.eu/justice/article-29/documentation/opinion-recommendation/files/2009/wp163_en.pdf Article 29 Data Protection Working Party (‘WP29’). (2010). Opinion 1/2010 on the Concepts of ‘Controller’ and ‘Processor’ (WP169). https://ec.europa.eu/justice/article-29/documentation/opinion-recommendation/files/2010/wp169_en.pdf Article 29 Data Protection Working Party (‘WP29’). (2013). Statement of the Working Party on Current Discussions Regarding the Data Protection Reform Package—Annex 2: Proposals for Amendments Regarding Exemption for Personal or Household Activities. https://ec.europa.eu/justice/article-29/documentation/other-document/files/2013/20130227_statement_dp_annex2_en.pdf Article 29 Data Protection Working Party (‘WP29’). (2014). Opinion 06/2014 on the Notion of Legitimate Interests of the Data Controller under Article 7 of Directive 95/46/EC (WP217). https://ec.europa.eu/justice/article-29/documentation/opinion-recommendation/files/2014/wp217_en.pdf Article 29 Data Protection Working Party (‘WP29’). (2017). Guidelines on Data Protection Impact Assessment (DPIA) and Determining Whether Processing Is ‘Likely to Result in a High Risk. http://ec.europa.eu/newsroom/document.cfm?doc_id=47711 Barocas, S., & H, N. (2014). Big Data’s End Run around Procedural Privacy Protections. Communications of the ACM, 57(11), 31–33. https://doi.org/10.1145/2668897 Belgisch Commissie voor de Bescherming van de Persoonlijke Levenssfeer. (2007). Aanbeveling Uit Eigen Beweging Inzake de Verspreiding van Beeldmateriaal. Belgisch Commissie voor de Bescherming van de Persoonlijke Levenssfeer (Belgian data protection authority). Bennett, C. J., & Raab, C. D. (2006). The Governance of Privacy: Policy Instruments in Global Perspective (2nd and updated ed.). MIT Press. https://doi.org/10.1080/19331680801979039 Ben-Shahar, O. (2019). Data Pollution. Journal of Legal Analysis, 11, 104–159. https://doi.org/10.1093/jla/laz005 Besmer, A., & Lipford, H. R. (2010). Users’ (Mis)Conceptions of Social Applications. Proceedings of Graphics Interface. https://doi.org/10.1007/978-3-319-07509-9_2 Biczók, G., & P.H, C. (2013). Interdependent Privacy: Let Me Share Your Data. Financial Cryptography and Data Security. Springer. https://doi.org/10.1007/978-3-642-39884-1_29 Bloustein, E. J. (1978). Individual and Group Privacy. Transaction Publishers. Boyd, d. (2011). Networked Privacy. Personal Democracy Forum. https://www.danah.org/papers/talks/2011/PDF2011.html. Boyd, d, K., L., & Marwick, A. E. (2014). The Networked Nature of Algorithmic Discrimination. In S. P. Gangadaranwith, V. Eubanks, & S. Barocas (Eds.), Data and Discrimination: Collective Essays. Open Technology Institute and New America. http://www.newamerica.org/downloads/OTI-Data-an-Discrimination-FINAL-small.pdf Bygrave, L. A. (2004). Privacy Protection in a Global Context. A Comparative Overview. Scandinavian Studies in Law, 47. https://www.uio.no/studier/emner/jus/jus/JUS5630/v13/undervisningsmateriale/privacy-and-data-protection-in-international-perspective.pdf Calo, R. (2011). The Boundaries of Privacy Harm. Indiana Law Journal, 86(3). http://ilj.law.indiana.edu/articles/86/86_3_Calo.pdf Caughlin, T. T., Ruktanonchai, N., Acevedo, M. A., Lopiano, K. K., Prosper, O., Eagle, N., & Tatem, A. J. (2013). Place-Based Attributes Predict Community Membership in a Mobile Phone Communication Network. Angel Sánchez. PLoS ONE 8, 2. https://doi.org/10.1371/journal.pone.0056057 Chadwick, R. F., Levitt, M., & Shickle, D. (Eds.). (2014). The Right to Know and the Right Not to Know: Genetic Privacy and Responsibility (Second). Cambridge Bioethics and Law. Cambridge. https://doi.org/10.1017/CBO9781139875981 Chen, J., Ping, J. W., Xu, Y., & Tan, B. C. Y. (2015). Information privacy concern about peer disclosure in online social networks. IEEE Transactions on Engineering Management, 62(3), 311–324. https://doi.org/10.1109/TEM.2015.2432117 Choi, J. P., Jeon, D., & Kim, B. (2019). Privacy and Personal Data Collection with Information Externalities. Journal of Public Economics, 173, 113–124. https://doi.org/10.1016/j.jpubeco.2019.02.001 Cohen, J. (2000). Examined Lives: Informational Privacy and the Subject as Object. Stanford Law Review, 52, 1373–1438. https://doi.org/10.2307/1229517 College Bescherming Persoonsgegevens. (2007). Publicatie van Persoonsgegevens Op Internet [Guidelines]. College Bescherming Persoonsgegevens (Dutch Data Protetection Authority). https://autoriteitpersoonsgegevens.nl/sites/default/files/downloads/rs/rs_20071211_persoonsgegevens_op_internet_definitief.pdf College Bescherming Persoonsgegevens. (2013). Investigation into the processing of personal data for the ‘whatsapp’ mobile application by WhatsApp Inc (Report No. Z2011-00987; Issue Z2011). College Bescherming Persoonsgegevens (Dutch Data Protetection Authority). https://autoriteitpersoonsgegevens.nl/sites/default/files/downloads/mijn_privacy/rap_2013-whatsapp-dutchdpa-final-findings-en.pdf Culnan, M. J. (2000). Protecting Privacy Online: Is Self-Regulation Working? Journal of Public Policy & Marketing, 19(1), 20–26. https://doi.org/10.1509/jppm.126.96.36.19944 Culnan, M. J., & Armstrong, P. K. (1999). Information Privacy Concerns, Procedural Fairness, and Impersonal Trust: An Empirical Investigation. Organization Science, 10(1), 104–115. https://doi.org/10.1287/orsc.10.1.104 De Conca, S. (2020). Between a rock and a hard place: Owners of smart speakers and joint control. SCRIPT-Ed, 17(2), 238–268. https://doi.org/10.2966/scrip.170220.238 De Hert, P. (2008). Identity Management of E-ID, Privacy and Security in Europe. A Human Rights View. Information Security Technical Report, 13(2), 71–75. https://doi.org/10.1016/j.istr.2008.07.001 Edwards, L., Finck, M., Veale, M., & Zingales, N. (2019). Data Subjects as Data Controllers: A Fashion(Able) Concept? Internet Policy Review. https://policyreview.info/articles/news/data-subjects-data-controllers-fashionable-concept/1400 Erlich, Y., Shor, T., Pe’er, I., & Carmi, S. (2018). Identity Inference of Genomic Data Using Long-Range Familial Searches. Science, 362(6415), 690–94. https://doi.org/10.1126/science.aau4832 European Commission. (2012). Proposal for a Regulation of the European Parliament and of the Council on the Protection of Individuals with Regard to the Processing of Personal Data and on the Free Movement of Such Data. https://www.europarl.europa.eu/registre/docs_autres_institutions/commission_europeenne/com/2012/0011/COM_COM(2012)0011_EN.pdf European Data Protection Supervisor. (2014a). Opinion of the European Data Protection Supervisor on the Communication from the Commission to the European Parliament and the Council on ‘A New Era for Aviation Opening the Aviation Market to the Civil Use of Remotely Piloted Aircraft Systems in a Safe and Sustainable Manner, COM(2014) 207 Final. https://edps.europa.eu/sites/edp/files/publication/14-11-26_opinion_rpas_en.pdf European Data Protection Supervisor. (2014b). Preliminary Opinion of the European Data Protection Supervisor on Privacy and Competitiveness in the Age of Big Data. https://edps.europa.eu/sites/edp/files/publication/14-03-26_competitition_law_big_data_en.pdf European Data Protection Supervisor. (2019a). Connected Cars (TechDispatch). Publications Office of the European Union. https://doi.org/10.2804/70098 European Data Protection Supervisor. (2019b). Smart Speakers and Virtual Assistants (TechDispatch). Publications Office of the European Union. https://doi.org/10.2804/755512 European Data Protection Supervisor. (2020). A Preliminary Opinion on Data Protection and Scientific Research. https://edps.europa.eu/sites/edp/files/publication/20-01-06_opinion_research_en.pdf Fashion ID, C-40/17, EU:C:2019:629 (Court of Justice of the European Union 29 July 2019). https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:62017CA0040&qid=1590355470801&from=EN Froomkin, M. (2015). Regulating Mass Surveillance as Privacy Pollution: Learning from Environmental Impact Statements. University of Illinois Law Review, 2015(5), 1713–1790. https://doi.org/10.2139/ssrn.2400736 Garcia, D. (2017). Leaking Privacy and Shadow Profiles in Online Social Networks. Science Advances, 3(8). https://doi.org/10.1126/sciadv.1701172 Garcia-Murillo, M., & MacInnes, I. (2018). Così Fan Tutte: A Better Approach than the Right to Be Forgotten. Telecommunications Policy, 42(3), 227–40. https://doi.org/10.1016/j.telpol.2017.12.003 Gnesi, S., Matteucci, I., Moiso, C., Mori, P., Petrocchi, M., & Vescovi, M. (2014). My Data, Your Data, Our Data: Managing Privacy Preferences in Multiple Subjects Personal Data. In B. Preneel & D. Ikonomou (Eds.), Privacy Technologies and Policy (Vol. 8450, pp. 154–171). Springer International Publishing. https://doi.org/10.1007/978-3-319-06749-0_11 Google Spain and Google, EU:C:2014:317 (Court of Justice of the European Union 13 May 2014). https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:62012CJ0131&qid=1590355288547&from=EN Hallinan, D., & Hert, P. (2017). Genetic Classes and Genetic Categories: Protecting Genetic Groups Through Data Protection Law. In Linnet Taylor, L. Floridi, & B. Sloot (Eds.), Group Privacy (pp. 175–196). Springer International Publishing. https://doi.org/10.1007/978-3-319-46608-8_10 Hann, I.-H., Hui, K.-L., Lee, T. S., & Png, I. (2002). Online Information Privacy: Measuring the Cost-Benefit Trade-Off. Proceedings of the International Conference on Information Systems (ICIS. https://aisel.aisnet.org/icis2002/1 Heath, J. (2007). An Adversarial Ethic for Business: Or When Sun-Tzu Met the Stakeholder. Journal of Business Ethics, 72(4), 359–374. https://doi.org/10.1007/s10551-006-9175-5 Helberger, N., & Hoboken, J. (2010). Little Brother Is Tagging You—Legal and Policy Implications of Amateur Data Controllers. Computer Law International, 11(4), 101–109. https://hdl.handle.net/11245/1.337383 Hirsch, D. (2006). Protecting the Inner Environment: What Privacy Regulation Can Learn from Environmental Law. Georgia Law Review, 41(1), 1–63. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1021623 Hirsch, D. (2014). The Glass House Effect: Big Data, the New Oil, and the Power of Analogy. Maine Law Review, 66(2), 373–395. https://digitalcommons.mainelaw.maine.edu/mlr/vol66/iss2/3 Hirsch, D., & King, J. H. (2016). Big Data Sustainability: An Environmental Management Systems Analogy. Washington and Lee Law Review Online, 72(3), 406–419. https://scholarlycommons.law.wlu.edu/wlulr-online/vol72/iss3/4 Holloway, D. (2019). Surveillance capitalism and children’s data: The Internet of toys and things for children. Media International Australia, 170(1), 27–36. https://doi.org/10.1177/1329878X19828205 Hull, G. (2015). Successful Failure: What Foucault Can Teach Us about Privacy Self-Management in a World of Facebook and Big Data. Ethics and Information Technology, 17(2), 89–101. https://doi.org/10.1007/s10676-015-9363-z Hull, G., Lipford, H. R., & Latulipe, C. (2011). Contextual Gaps: Privacy Issues on Facebook. Ethics and Information Technology, 13(4), 289–302. https://doi.org/10.1007/s10676-010-9224-8 Humbert, M., Ayday, E., Hubaux, J.-P., & Telenti, A. (2015). On Non-Cooperative Genomic Privacy. In R. Böhme & T. Okamoto (Eds.), Financial Cryptography and Data Security (pp. 407–426). Springer. https://doi.org/10.1007/978-3-662-47854-7_24 Humbert, M., Trubert, B., & Huguenin, K. (2020). A Survey on Interdependent Privacy. ACM Computing Surveys, 52(6). https://doi.org/10.1145/3360498 Inc, F. (2018). Facebook Post-Hearing Responses to Commerce Committee: “Facebook, Social Media Privacy, and the Use and Abuse of Data. https://www.judiciary.senate.gov/imo/media/doc/Zuckerberg%20Responses%20to%20Commerce%20Committee%20QFRs1.pdf Information Commissioner’s Office. (2017). In the Picture: A Data Protection Code of Practice for Surveillance Cameras and Personal Information [Report]. Information Commissioner’s Office. https://ico.org.uk/media/1542/cctv-code-of-practice.pdf Introna, L. D. (1997). Privacy and the Computer: Why We Need Privacy in the Information Society. Metaphilosophy, 28(3), 259–75. https://doi.org/10.1111/1467-9973.00055 Jernigan, C., & Mistree, B. F. T. (2009). Gaydar: Facebook Friendships Expose Sexual Orientation. First Monday, 14(10). https://firstmonday.org/article/view/2611/2302 Jia, H., & Xu, H. (2016). Measuring Individuals’ Concerns over Collective Privacy on Social Networking Sites. Cyberpsychology: Journal of Psychosocial Research on Cyberspace, 10(1). https://doi.org/10.5817/CP2016-1-4 Kamleitner, B., & Mitchell, V. (2019). Your Data Is My Data: A Framework for Addressing Interdependent Privacy Infringements. Journal of Public Policy & Marketing, 38(4), 433–450. https://doi.org/10.1177/0743915619858924 Kitchin, R. (2014a). The Data Revolution: Big Data, Open Data, Data Infrastructures & Their Consequences. SAGE Publications. Kitchin, R. (2014b). The Real-Time City? Big Data and Smart Urbanism. GeoJournal, 79(1), 1–14. https://doi.org/10.1007/s10708-013-9516-8 Kupfer, J. (1987). Privacy, Autonomy, and Self-Concept. American Philosophical Quarterly, 24(1), 81–89. Lampinen, A. (2015). Networked Privacy Beyond the Individual: Four Perspectives to “Sharing”. Aarhus Series on Human Centered Computing, 1(1). https://doi.org/10.7146/aahcc.v1i1.21300 Lampinen, A., Lehtinen, V., Lehmuskallio, A., & Tamminen, S. (2011). We’re in It Together: Interpersonal Management of Disclosure in Social Network Services. Proceedings of the 29th International Conference on Human Factors in Computing Systems. https://doi.org/10.1145/1978942.1979420 Laudon, K. C. (n.d.). Markets and Privacy (1996. Communications of the ACM, 39(9), 92–104. https://doi.org/10.1145/234215.234476 Le Borgne-Bachschmidt, F., Girieud, S., Leiba, M., Munck, S., Limonard, S., Poel, M., Kool, L., Helberger, N., Guibault, L., Janssen, E., Eijk, N., Angelopoulos, C., Hoboken, J., & Swart, E. (2008). User-Created-Content: Supporting a participative Information Society. https://www.ivir.nl/publicaties/download/User_created_content.pdf Levy, S. (2020). Facebook: The inside Story. Dutton. Lindqvist, C-101/01, EU:C:2003:596 (Court of Justice of the European Union 6 November 2003). https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:62001CJ0101&from=EN MacCarthy, M. (2011). New Directions in Privacy: Disclosure, Unfairness and Externalies. I/S: A Journal of Law and Policy for the Information Society, 6(3), 425–512. https://kb.osu.edu/handle/1811/72971 Mahieu, R., Hoboken, J., & Asghari, H. (2019). Responsibility for Data Protection in a Networked World: On the Question of the Controller, ‘Effective and Complete Protection’ and Its Application to Data Access Rights in Europe. JIPITEC, 10(1). https://nbn-resolving.org/urn:nbn:de:0009-29-48796 Mantelero, A. (2014). The Future of Consumer Data Protection in the EU: Rethinking the ‘Notice and Consent’ Paradigm in the New Era of Predictive Analytics. Computer Law & Security Review, 30(6), 643–660. https://doi.org/10.1016/j.clsr.2014.09.004 Mantelero, A. (2016). Personal Data for Decisional Purposes in the Age of Analytics: From an Individual to a Collective Dimen...
Personal information management systems: a user-centric privacy utopia?
1. Introduction Online systems and services are driven by data. There are growing concerns regarding the scale of collection, computation and sharing of personal data, the lack of user control, individuals’ rights, and generally, who reaps the benefits of data processing (German Data Ethics Commission, 2019). Currently, data processing largely entails the capture of individuals’ data by organisations, who use this data for various purposes, in a manner that is often opaque to those to whom the data relates. This general lack of transparency has meant that consent and other legal arrangements for the safe and responsible processing of personal data are considered rather ineffective (Blume, 2012; Cate & Mayer-Schönberger, 2013; Tolmie et al., 2016; German Data Ethics Commission, 2020). Privacy Enhancing Technologies (PETs) are technologies that aim to help in addressing privacy concerns (The Royal Society, 2019). Personal data stores (PDSs), otherwise known as personal information management systems (PIMS), represent one class of such technology, focused on data management. In essence, a PDS equips an individual (user) with a technical system for managing their data (a ‘device’). Generally, a PDS device provides the user with technical means for mediating, monitoring and controlling: (i) the data captured, stored, passing through, or otherwise managed by their device; (ii) the computation that occurs over that data; and (iii) how and when the data, including the results of computation, is transferred externally (e.g., off-device, to third-parties). Proponents of PDSs argue that it empowers users, by “put[ting] individuals in control of their data” (Crabtree et al., 2018). This is because PDSs provide means for ‘users to decide’ what happens to their data; in principle, third-parties cannot access, receive or analyse the data from a PDS without some user agreement or action. In this way, PDSs purport a range of user benefits, from increased privacy and the ability to ‘transact’ (or otherwise monetise) their data, to better positioning users to gain insights from their own data (see subsection 2.3). More broadly, PDSs seek to provide an alternative to today’s predominant form of data processing, where organisations collect, store and/or use the data of many individuals. As this often occurs within a single organisation’s technical infrastructure, there may be limited scope for individuals to uncover – let alone control – what happens with their data. The vision for PDSs is to decentralise data and compute, away from organisations, such that it happens with more user control. PDS technology is nascent, but growing in prominence. Exemplar PDS platforms currently at various stages of development and availability include Hub of All Things & Dataswift (Dataswift) 1; Mydex, CitizenMe, Databox and Inrupt/Solid (Inrupt) 2 (which is led by Sir Tim Berners-Lee). As nascent technology, PDSs raise several areas for investigation by academia, policymakers, and industry alike. There is already work, for instance, on how PDSs might facilitate better accountability (Crabtree, 2018; Urquhart, 2019), and on the legal uncertainties surrounding the technology, particularly concerning data protection (Janssen et al., 2020; Chen et al., 2020). This paper takes a broader view, questioning the extent to which PDS technology can actually empower individuals and address the concerns inherent in data processing ecosystems. After giving an overview of the technology, and its purported benefits in section 2, we examine, in section 3, some data protection implications of PDSs focusing on the user’s perspective: whether they support particular legal bases for processing personal data; the social nature of personal data captured by PDSs; and the relation of PDSs to data subject rights. In section 4, we argue that the broader information and power asymmetries inherent in current online ecosystems remain largely unchallenged by PDSs. Section 5 synthesises the discussion, indicating that many of the concerns regarding personal data are systemic, resulting from current data surveillance practices, and concluding that PDSs – as a measure that ultimately still requires individuals to ‘self-manage’ their privacy – only go so far. 3 2. Technology overview PDSs represent a class of data management technologies that seek to localise data capture, storage and the computation over that data towards the individual. Generally, they entail equipping a user with their own device for managing their data. A device operates as a (conceptual) data ‘container’, in a non-technical sense of the word: a strictly managed technical environment in which data can be captured or stored or can pass through, and within which certain computation can occur. 4 Some devices are wholly virtual (e.g. Digi.me), hosted in the cloud, while others encompass particular physical equipment such as a box or hub (see e.g. Databox). PDSs generally purport to empower users through their devices. Though offerings vary, generally PDSs provide technical functionality for: Local (within-device) capture and storage of user data. Mechanisms for users to populate their PDS with data from a range of sources, which may include from their phones, wearables, online services, manual data entry, sensors, etc. Local (on-device) computation. Enabling computation to occur (software to execute) on the device, which generally entails some processing of data residing with the device. Mediated data transfers. Allowing control over the data transferred externally (off-device); including ‘raw’ user data, the results of computation, and other external interactions (e.g. calls to remote services). Transparency and control measures. Tooling for monitoring, configuring and managing the above. This includes governance measures for users to set preferences and constraints over data capture, transfer and processing; visualising and alerting of specific happenings within the device; etc. The device’s technical environment (infrastructure) manages security aspects. This can include data encryption, managing and controlling user access to the device and its data, and providing means for isolating data and compute. Further, it also works to ensure adherence with any policies, preferences and constraints that are set (see #4 above). For instance, if a user specifies that particular data cannot be transferred to some party (or component), or should not be included in some computation, the device’s technical environment will ensure these constraints are respected. Core to many PDSs is allowing for computation (potentially any form of code execution, including analytics) to be ‘brought’ to the data. This occurs through an app: software that executes on a user’s device for processing that device’s data. 5 Some apps may provide the user with functionality without any external transfer of data. Though often apps will transfer some data off-device (such as the results of computation). PDS proponents describe such functionality as of key industry interest, arguing that receiving only the results of computation (e.g. aggregated findings) avoids the sensitivities, overheads and resistance associated with receiving and managing granular and specific user data (see subsection 2.4). Apps operate subject to constraints: they must define what data sources they seek, the data they transfer, and other details; and users may put constraints on how apps behave, e.g. regarding the data that apps may access, process, and transfer. The device’s technical environment ensures adherence to these constraints. Legal mechanisms also operate to govern the behaviour and operation of PDS ecosystems (see subsection 2.2). 6 2.1 A multi-actor ecosystem It is worth recognising that there are several actors within a PDS ecosystem. We now introduce those most pertinent for this discussion. The focus is on users, but this article is about empowerment and power, so other actors need to be introduced. Users are those individuals who hold a device, leveraging the PDS functionality to manage their data. Organisations are those interested in processing user data. Here, we describe organisations as app developers, as they build apps that process user data for installation on user devices. Again, apps will often transfer some data to the organisation, such as the results of computation. PDSs may also support the transfer of data to an organisation without a specific app. This process is managed through the direct data transfer mechanisms provided by the device (which may itself be a form of app, packaged with the device). Platforms are the organisations that provide the PDS and/or manage the PDS ecosystem. There will be a range of platforms that differ in their offerings. Often a platform’s core offering is equipping a user with a device; though this could vary from merely providing the codebase for users to compile and self-manage the operation of their devices, to providing the entire operational infrastructure—perhaps including hardware, managed cloud services for backup, and so forth (Janssen et al., 2020). Moreover, some platforms envisage hosting ‘app stores’ or ‘data marketplaces’ that broker between users and the organisations seeking to process their data, while many platforms require adherence with ‘best practices’, have defined terms of service, and may even have contractual agreements with users and organisations. In this way, platforms vary in their level of involvement in the operation of the PDS ecosystem. 2.2 Governance regimes In addition to technical aspects, PDS platforms often entail legal governance mechanisms. These operate to help ensure that app behaviour, and data usage more generally, is compliant with user preferences, and platform requirements. Some of these are encapsulated in a platform’s Terms of Service (ToS), which commonly define how the platform can be used, and the platform’s position on the allocation of responsibilities and liabilities. Platform ToS often require app developers to have appropriate measures in place to safeguard users against unlawful processing (e.g. Dataswift’s acceptable use policy), and to safeguard users against accidental data loss or destruction (idem) while requiring them to, for instance, safely keep their passwords or to regularly update their PDSs for security purposes (e.g. Dataswift’s terms for users). Platforms may also have contracts with app developers, which contain business specific terms and conditions, governing their interactions with user data, the functionality of their apps etc. ToS and contracts might stipulate, for example, that app developers must fully comply with platform policies and principles regarding user data processing, where failure to do so may result in the platform terminating their data processing activities (example from Mydex ToS). 2.3 Purported user benefits PDSs generally purport to provide functionality to empower users. Some claimed benefits for users include: Users having granular control over the data captured about them, and how that data is shared and used (Article 29 Data Protection Working Party 2014; Crabtree et al., 2018; Urquhart et al., 2019); Better protecting personal data (including ‘sensitive’ personal data) from access by third parties, by way of the technical functionality provided (Crabtree et al., 2018; Lodge et al., 2018); Better informed user consent, by giving more information about data processing. This may be through various means, including the device’s monitoring functionality; the app’s data usage specifications; platform features, such as app stores ranking and describing app data usage, requiring transparency best practices, etc. (Mydata); Compartmentalised data storage and computation to prevent apps from interacting with data (and other apps) inappropriately, inadvertently and without user agreement/intervention (e.g. Crabtree et al., 2018); Providing opportunities for users to gain more insights from their data (e.g., Mydex; Mydata); Allowing users to transact with or monetise their personal data (Ng & Haddadi, 2018); Generally incentivising developers towards more privacy friendly approaches (Crabtree et al., 2018). PDSs have also caught the attention of policymakers; the European Commission recently expressed that PDSs and similar tools have significant potential as “they will create greater oversight and transparency for individuals over the processing of their data […] a supportive environment to foster [their] development is necessary to realise [their] benefits” (European Commission, 2020). This potentially indicates that the European Commission might in the future define policy encouraging the development of these tools. 2.4 Purported organisational benefits For organisations (app developers), the appeal of PDSs is the promise of access to more data—potentially in terms of volume, richness, velocity and variety—for processing. PDS enthusiasts argue that if users better understand how their data is being processed, and feel empowered by way of PDS’s control mechanisms, they may be less ‘resistant’ and harbour a greater ‘willingness’ for (managed) data sharing and processing (e.g., Control-Shift; Mydata; Digi.me; CitizenMe mention this in their descriptions). Similarly, given that PDSs will encapsulate a variety of user information, PDSs might offer app developers access to a broader range of data types than if they attempted to collect the data themselves (Mydata). Though PDSs are typically described with reference to an individual, most aim to support ‘collective computation’, whereby the processing of data across many users or (particular) populations is enabled through apps operating on their devices (e.g., Mydata; Databox; CitizenMe; Digi.me). 7 Collective computations often entail some user or population profiling to support various organisational aims—customer insight, market research, details of product usage, or indeed, and as is common in online services to support a surveillance-driven advertising business model (as discussed in section 5). In this way, PDS platforms effectively provide a personal data processing architecture that operates at scale across a population. This is attractive for organisations, as PDS platforms with large user-bases offer access to a wider population and thus more data than the organisation would otherwise themselves have access to. Importantly, this also comes without the costs, risks, and compliance overheads incurred in undertaking data collection, storage, and management ‘in-house’, using their own infrastructure (Crabtree et al., 2018). 2.5 PDS platforms: the commercial landscape Some predict that PDSs could generate substantial economic benefits for businesses and consumers alike (Control-Shift; Brochot et al., 2015; European Commission, 2020). Although the business models for organisations are likely similar to those already existing, the business models for the PDS platforms are unclear and remain under development (Bolychevsky & Worthington, 2018). A range of possible revenue streams for PDS platforms have been developed and proposed. These include: Platforms charging organisations fees for access to the PDS ecosystem (e.g., annual fee, Mydex); charges for access to the platform’s app store, per user download of their app, etc); Platforms charging organisations per ‘data transaction’ with a PDS device, where the type of transaction (access, computation, and/or transfer of data, including raw data, see e.g. Mydex) and/or the type of data requested (e.g. queries, behavioural data) often determines the price (see e.g. CitizenMe); Organisations sharing revenue with the platform through in-app purchases (e.g. Digi.me); Platforms charging organisations for support services (e.g. Mydex); Users paying a subscription fee, or to unlock additional functionality (Digi.me); Platforms selling, renting or leasing PDS devices to users, which could include service or maintenance contracts (Crabtree et al., 2018); or Platforms in the public interest (e.g. PDSs platforms for public health) might be ‘fee-free’, funded through, e.g. donations, and public funds (see e.g. BBC-Box). As PDSs are a developing area, the business models of platforms are nascent. In practice, one expects that platforms will likely employ a range of monetisation mechanisms. 3. Data protection A key aim of PDSs is to give users greater visibility and control over the processing of their personal data. PDS architectures concern issues regarding personal data, and therefore the General Data Protection Regulation (GDPR) must be considered. GDPR centres around three legal roles: controllers (acting alone or with others together as joint controllers; (Arts. 4(7), 26 GDPR), processors (including sub-processors; Arts. 4(8), 28(4) GDPR), and data subjects (Art. 4(1) GDPR). The role of a particular actor as a controller or processor is generally a question of their factual influence over data processing; how an actor describes their role (for example in contract) may be indicative, but won’t be definitive (Article 29 Working Party, 2010). GDPR tasks both controllers and processors with a range of responsibilities and obligations, the bulk of which fall on controllers, given their role in determining the nature of the data processing. Obligations for controllers include complying with data protection principles (Art. 5(1) GDPR), that this compliance is demonstrable (Art. 5(2) GDPR), that their processing of personal data is predicated on one of the GDPR’s lawful grounds (Art. 6(1) GDPR), to name a few. Typical rights afforded to data subjects (i.e. those whose personal data is being processed) which controllers are tasked with meeting, include the rights to object to data processing, to have their data erased, or to port data (subsection 3.3). While PDS technologies and their governance models are still developing, many unresolved data protection issues exist. The assignment of roles and responsibilities in PDS systems is complex, given such ecosystems are largely shaped by the collaboration of multiple parties, including the key actors mentioned here. This reality can be difficult to reconcile with GDPR’s approach with controllers who ‘orchestrate’ the data processing in an entire system. In practice, a PDS’s ecosystem can take a number of forms, and the legal position of those involved will depend on the circumstances. Issues of roles and responsibilities under the GDPR in different PDS contexts are explored in detail by Chen et al., and Janssen et al. (2020). In this paper, we consider three key ‘user-facing’ data protection considerations: (1) how PDSs, in being oriented towards consent, relates to GDPR’s lawful grounds; (2) how personal data often relates to more persons than just the PDS user; and (3) the relationship between PDSs and data subject rights. 3.1 Lawful grounds for processing GDPR requires that processing is predicated on one of its lawful bases as defined by Art. 6(1) GDPR. Controllers must determine which lawful ground is most appropriate in a given situation, depending on specific purposes and contexts for use, the nature of parties involved, and their motivations and relationships, and of course, the requirements for the lawful basis on which they rely. However, due to the ePrivacy Directive, where the PDS entails a physical (hardware) device, consent will generally be required for app developers to process any data (Art. 5(3) ePrivacy Directive; Janssen et al., 2020). In this context, for such devices the only available basis for processing on these devices will be consent (Arts. 6(1)(a) & 7 GDPR; Recitals 32, 42, 43 GDPR) and explicit consent (for special category data—particular classes of data deemed to require extra protections (Art. 9(1), Recitals 51-56 GDPR)). For ‘virtual’ PDS devices, such as those cloud hosted (as are currently by far the most common), legal bases other than consent may be available (unless that data is special category data, in which case explicit consent is often the only option). PDS devices are fundamentally oriented towards supporting the grounds of (user) consent and contract (where the processing is necessary for the performance of a contract to which the user is a party) as the legal bases for processing. Importantly, both consent and contract are grounds that require agreement by the data subject to render the processing lawful. PDS platforms are generally explicitly designed for supporting such, by requiring active user agreement regarding data processing (Crabtree et al., 2018; Urquhart 2019). PDSs generally purport functionality that aims at informing users, e.g. providing them information about an app and its related data processing, and requiring the user to take positive actions, e.g. agreeing to terms upon installing the app, configuring data usage preferences and policies, in order for that processing to occur. There are also lawful grounds for processing, such as legal obligation, public interest or legitimate interest which allow the controllers—not the data subjects (users)—to decide whether processing can occur. That is, user consent is not required for certain public tasks (e.g. perhaps in taxation), or for legitimate controller interest (e.g. perhaps for the processing of certain data to detect fraud). The requirements vary by legal basis, and can include (depending on the ground) considerations like the necessity of that processing (Arts. 6(1)(b)—(f) GDPR), that controller interests are balanced with the fundamental rights of the data subject (Art. 6(1)(f) GDPR; Kamara & De Hert, 2018), and a foundation in compatible member state law (Arts. 6(1)(c) and (e) GDPR). These grounds for processing that are not based on specific and active user involvement or agreement are rarely considered in PDS architectures, and at present it is unclear how PDS architectures would support or reconcile with these grounds where they may apply (Janssen et al., 2020). 3.2 Social nature of personal data Personal data is relational and social by nature; it often does not belong to one single individual, as much personal data is created through interactions with other people or services (Article 29 Working Party, 2017; Crabtree & Mortier, 2015). In practice, a PDS device will likely capture data relating to multiple individuals other than the user—for example, through sensing data from other dwellers or visitors in and around someone’s home. This raises interesting questions regarding the mechanisms for one to control what is captured about them in someone else’s PDS. That is, there may be conflicting visions and preferences between the user and others regarding the use and processing of ‘joint’ data, and these others may also have data subject rights (see subsection 3.3). At present, PDSs generally give a device’s user greater control over the processing related to that device; functionality enabling the preferences and rights of others to be managed and respected has yet had little consideration. This is an area warranting further attention. 3.3 Supporting data subject rights GDPR affords data subjects several rights regarding the processing of their personal data. These include the rights of access to their personal data (Art. 15), rectification of inaccurate personal data (Art. 16), erasure (Art. 17), to object (Art. 21), to restrict the processing of their data (Art. 18), to port their data to another controller in a commonly used machine-readable format (Art. 20 GDPR), and to not be subject to solely automated decision-making or profiling which produces legal or similarly significant effects (Art. 22 GDPR). Controllers are tasked with fulfilling these rights. Data subject rights are not absolute—GDPR imposes conditions on the exercise of some rights, and not all rights will apply in every situation. Data subject rights have had little consideration in a PDS context. Again, to improve the transparency of processing, PDSs usually afford users some visibility over what occurs on-device and provide information on their device’s interactions (data exchanges) with organisations (Urquhart et al., 2018). They also generally offer certain controls to manage on-device processing. As such, some have suggested that PDSs may (at least for data within the PDS device) to some extent “negate” a user’s need to exercise certain data subject rights (Urquhart et al., 2018), where such mechanisms could potentially provide means for users to themselves restrict certain processing, and erase, delete or port data, and so forth. However, current PDS tooling, at best, only gives certain users visibility and the ability to take action regarding processing happening on-device (see subsection 4.1). Data subject rights, however, are broader, and encompass more than simply giving users visibility over on-device data processing. Users will, for instance, have interests in the behaviour of organisations involved in processing. GDPR requires controllers to account for data protection considerations, including those relating to rights, in their technological and organisational processes (Data protection by design, GDPR Art 25(1)). This has implications not only for app developers, but also for PDS platforms, who could provide mechanisms that specifically and more holistically facilitate users in exercising their rights. Though there may be questions as to whether this is legally obliged—for instance in light of the complexities regarding a platform’s roles and responsibilities given that Art 25(1) applies to controllers (see Chen et al., 2020; Janssen et al., 2020). Indeed, these considerations are exacerbated as some PDSs represent ‘open source’ projects, potentially involving a wide range of entities in the development, deployment and operation of the platform and/or device functionality. However, regardless of any legal obligation, any PDS platform should aim to better support users with regards to their data rights, given that this is wholly consistent with the stated aims of PDSs as ‘empowering users’. Beyond PDS functionality that specifically aims at rights, there is potential for PDS transparency mechanisms to assist users with their rights more generally. For instance, PDSs might, by providing information, help users in detailing and targeting their rights requests. User observation of, or a notification by the platform indicating particular application behaviour, might encourage users to exercise their right ‘to find out more’, or perhaps encourage them to validate that their rights requests were properly actioned. This might help users to determine whether processing should continue, or help them confirm whether the information provided by the controller corresponds to the operations observed on-device. The right to data portability grants users the right to receive copies of the data they provided to a controller in an electronic format, and to transfer that data or to have it transferred to another controller. This can only be invoked if the processing was based on the lawful grounds of consent or contract (Art. 20(1)(a) GDPR), and concerns only that data provided by data subjects themselves (Art. 20 (1) GDPR; Article 29 Working Party, 2016; Urquhart et al., 2017). Portability is considered a key means for users to ‘populate’ their PDSs by bringing their data from an organisation’s databases to the PDS (Art. 20 GDPR; Article 29 Working Party, 2019). Indeed, some PDS platforms describe the right as enabling users to ‘reclaim’ their data from organisations (e.g. CitizenMe; Dataswift; Digi.me),and envisage offering users technical mechanisms that leverage portability rights for populating their devices (idem). Subject access requests (Art. 15(3) GDPR) may also assist in populating devices, particularly given they are less constrained in terms of when it can be used, and usually result in more information than would be received from a portability request. However, subject access requests do not require that the data be returned in a machine-readable format. Without agreed-upon interoperability standards, using subject access requests (and indeed, even portability requests to some degree) to populate PDSs will often be impractical and cumbersome. PDSs’ transparency mechanisms are also relevant here, as they can work to improve the user’s position. This is because such mechanisms can expose the on-device computations, possibly including the results of those computations, and potentially in a meaningful technical format. This is useful not only for portability considerations (e.g. in a PDS context, potentially moving the results of computations across apps), but also in generally providing users with more knowledge and insight into the nature of data processing occurring. 4. Information asymmetries PDS platforms state that they empower users by providing them with means for increased transparency and control, enabling users to take better, more informed decisions about whether to engage or, indeed, disengage with particular processing. However, systemic information and power asymmetries are inherent in current digital ecosystems, whereby the highly complex and largely opaque nature of data processing amplifies the asymmetries between data subjects and the organisations processing their data (Mantelero, 2014). These asymmetries, stemming from an unequal distribution of opportunities in terms of understanding, knowledge, prediction, risk assessment, and so forth (Mantelero, 2014), make it difficult if not impossible for even knowledgeable users to properly evaluate and come to genuinely informed decisions about the processing of their data (Solove, 2013; Solove, 2020). The opaque nature of data processing is largely systemic because users of digital services often lack (or are prevented from gaining) knowledge or understanding of: (1) the practices of organisations capturing and processing their data, including the details, reasons for and implications of holding particular data or performing particular computation; (2) the data sharing practices of those organisations with third parties and beyond; (3) the technical details of the systems involved; (4) the data-driven, and indeed, often surveillance-driven business models (see section 5); and (5) the insights and power that organisations can gain through having access to data, particularly where data is aggregated or computation occurs at scale (collective computation). Legal issues may further contribute to systemic problems—including information asymmetries—within digital ecosystems (Cohen, 2019); for example, copyright, trade secrecy, or documents or databases owned by large organisations might work to restrict the information that is available to the public. However, these restrictions are not absolute and do not apply to every stakeholder. Under certain conditions, courts or regulators can be given access to data relating to trade secrets or databases not generally available to the public (Art. 58(1)(e); Recital 63 GDPR). Crucially, PDSs only partially respond to these issues and therefore only partially address the systemic nature of the information asymmetries of digital ecosystems. Providing a localised, user-centric containerisation of data and processing may assist users in gaining some knowledge of what happens with their personal information, but only to a limited extent. While users might gain some greater understanding over the data processing relating to their device, PDSs themselves are unlikely to solve these systemic information asymmetries. Fundamentally, PDSs are grounded in the mistaken idea that with enough information presented in the right way, individuals will be able to overcome barriers that are ultimately structural and systemic in nature (Nissenbaum, 2011). 4.1 Organisational data processing practices remain largely opaque An organisation’s intentions, motivations and behaviours may not always be clear to users (Burrell, 2016). Attempting to address this, PDSs require app developers to provide some information about their organisational processes and intentions. Such information (often encapsulated in ‘app manifests’) might include details of the types of data an app will process; the app developer’s purposes for that processing; the risks of the app; or with whom the app developer may share data received from the PDS (Crabtree, 2018; Janssen et al., 2020). 8 However, less discussed in PDS proposals is conveying information about why that particular data is necessary (as opposed to other, perhaps less sensitive data), why these weights are attached to particular data in the analytics process, and, more broadly, why that particular processing needs to occur, and the possible data protection implications this may have. This is an area needing attention. We now elaborate two additional aspects: (i) the lack of information available regarding data that flows beyond organisational boundaries, and (ii) how the opacity of app developers’ processes can hinder PDS platform’s governance processes. Note, however, that even if PDSs could provide additional information on developers’ processing practices, the utility of this for users is unclear. Moreover, this risks potentially creating a false sense of having adequately informed users while in actuality the problems caused by information asymmetries remain (this dimension is explored in subsection 4.2). 4.1.1 Transparency and control diminish as data moves across boundaries Once data moves beyond a system or organisation’s boundaries, the visibility over that data typically diminishes, as does the ability to control any subsequent processing (Singh et al., 2017; Crabtree et al., 2018; Singh et al., 2019). So, while PDSs might provide users with insights into device-related processing, PDSs generally will not (at least at a technical-level) provide users with information about – let alone access to – data that has moved to app developers (and, indeed, beyond). Even in a PDS context, users will (still) therefore have little meaningful information regarding the specifics of the data actually being shared between organisations and third parties. 9 The fact that some data usage is essentially out of sight raises various risks, including, for instance, around secondary uses of data that a user would not have agreed with, e.g. undisclosed monetisation (Silverman 2019), or unexpected or undesired inferences or profiling, which could be used to influence, nudge or manipulate (Wachter et al., 2019). Moreover, as many online services entail a ‘systems supply-chain’ (Cobbe et al., 2020) – whereby services from various organisations are used to deliver functionality – there may be little visibility regarding the specific organisations involved in processing once the data moves ‘off-device’. Though these issues are not typically the focus of PDSs, they relate to the technology’s broader aims. PDSs might potentially assist where technical mechanisms can improve the visibility over data processing and transfer from the device to the first recipient (one-hop), and legal means can govern such transfers (subsection 2.2). For instance, Mydex stipulates in its ToS that app developers may not transfer user data that is obtained through the platform’s service to third-parties, except to the extent that this is expressly permitted in the relevant app developer notice (see, for another example, Dataswift). Through these measures, PDSs might better inform users of – and offer greater control over – what is initially transferred ‘off-device’. However, the ability to actually monitor, track and control data as it moves across technical and administrative boundaries is an area for research (e.g. see Singh et al., 2017; Singh et al., 2019; Pearson & Casassa-Mont, 2011). 4.1.2 Issues with opacity and non-compliance for PDS platforms Many PDS platforms describe ToS and contractual arrangements with app developers, which define how app developers may process user data. However, organisational data processing opacities can also hinder platforms in uncovering and assessing the risks of non-compliant app and developer behaviour (Crabtree et al., 2018). Platforms’ monitoring and compliance measures might to some extent mitigate the implications of limited user understanding of app developers’ data processing practices, where non-compliance by a developer could result in termination of their processing, the app’s removal from the platform, payment of damages, etc (e.g. ToS of Mydex). This could entail log file analysis, app audits, and manual reviews, including ‘sandboxing’ (examining behaviour in a test environment), and reporting measures when non-compliance is detected on a device (comparable to software ‘crash reports’ in other contexts). However, there are questions around whether platforms themselves can effectively detect or otherwise uncover non-compliance by app developers. Platform operators generally position themselves to not have direct access to user devices (including data, processing and logs thereof), which limits their visibility over what is happening ‘on the ground’. Platforms becoming actively involved in device monitoring, by gaining visibility over the happenings on user devices, brings additional data protection considerations, while effectively involving a device ‘backdoor’ which has security implications and could undermine the PDS ecosystem. Questions of incentives are also raised, e.g. regarding the propensity for a provider to take action against app developers where doing so has impacts on the platform’s income or business. These issues need further attention. 4.2. Users still require knowledge and expertise PDSs are oriented towards data protection concerns, particularly regarding the difficulties in obtaining genuinely informed consent and offering users real control. But for this to be effective, users must also be able to understand the potential data protection implications of processing. This means PDS users will require some degree of data protection expertise and knowledge to enable them to comprehend the implications of certain computation and transfers. Though PDSs seek to provide users with more information about processing, and may offer some general guidance, it will not always be clear to users what the full implications of certain data processing or transfers are—not least given the risks are often contextual. A user might, for instance, allow an app developer to build a detailed profile, not realising these could subsequently be used to influence, nudge or manipulate themselves and others (Wachter & Mittelstadt, 2019). Similarly, an app’s or platform’s explanations and visualisations of data flows, technical parameters, configuration and preference management mechanisms, and so forth, can also be complex and difficult to understand for non-experts (Anciaux et al., 2019). Moreover, identifying where app behaviour does not comply with user preferences or is unexpected can be challenging even for expert users, let alone the non-tech-savvy. Users will therefore also require some technical expertise and knowledge to meaningfully interrogate, control and interact with the functionality of the platform (Crabtree et al., 2018). As a result, though PDSs seek to better inform users, simply providing them with more information may not produce substantially better informed and empowered users. That is, the information asymmetries currently inherent in digital ecosystems may remain largely unaddressed, and many users may remain largely unempowered and under-protected. There is on-going research by the PDS community on how platforms can make their transparency and control measures more effective (Crabtree et al., 2018). Default policies or usage of ‘policy templates’ might enable third parties (civil society groups, fiduciaries, etc) to set a predefined range of preferences (in line with certain interests and values) which users can easily adopt. Generally, mechanisms facilitating the meaningful communication and management of data protection risks and implications are an important area of research, not just for PDSs, but for digital ecosystems as a whole. 4.3 App developers may still collect and process at scale Many PDSs seek to support collective computations, allowing app developers to process user data at scale to generate insights from across a population (subsection 2.4). In practice, this likely contributes to further consolidating the information asymmetries between users and organisations. PDSs may help users to understand these asymmetries to some extent, as they allow users to generate insights into the personal data in their own PDSs. However, the fact that app developers can operate across user PDSs—and are encouraged by platforms to do so—means that they can process the data from many users, and thus remain better informed than individual users can ever be. Although an individual’s data may be interesting to that individual, it is analysing data at scale that can provide the insights into user behaviour and preferences that are often truly valuable to organisations. It is unlikely that PDSs will address this systemic issue by means of any of their measures; indeed, by enabling and encouraging collective computations, PDSs are likely to even further contribute to these asymmetries. As we will explore next, these asymmetries do not only exist with respect to individual users, but also society as a whole. This is because in the current digital environment, power resides with organisations who have the ability to access and process data. In facilitating collective computations, PDSs continue to support organisations to process data at scale. 5. Discussion: PDSs, privacy self-management and surveillance capitalism A range of commercial business models are surveillance oriented, where economic value is extracted by collecting and analysing extensive data about people’s behaviour, preferences, and interests (Andrejevic, 2011; Fuchs, 2011; Palmås, 2011; Zuboff, 2015). At present, this typically involves aggregating individual data, and analysing that aggregated data to identify patterns. The knowledge obtained through that analysis is used for various purposes. In the context of online services, where the issues are particularly pronounced, this includes algorithmically personalisation to keep users engaged with the service and to target advertising (Cobbe & Singh, 2019). Often this involves profiling, which poses threats to personal integrity, and online services often target user vulnerabilities for exploitation with addictive designs, dark patterns, and behavioural nudging (Yeung, 2017). Online service providers can work towards vendor lock-in and systemic consumer exploitation. Given the central commercial and economic imperatives of most online services, nearly all data-driven business models involve (to some degree) the trading of data and insights for profit (German Data Ethics Commission, 2019). Note, however, that not only online service providers are surveillance-oriented; PDSs themselves also encourage traditional off-line business models to be augmented with some form of user surveillance, for example, to observe the nature of product usage in a home. The extensive processing of personal data in surveillance-oriented or supported business models raises a range of concerns (Kerber, 2016; Christl, 2017; Myers West, 2017). As discussed in section 2, PDSs seek to address these concerns by giving users greater ‘control’ over their data and its processing through more information and options regarding processing and then enforcing their choices (by bringing the data processing closer to the user and placing legal and technical constraints on it). In this way, as discussed in section 3, PDSs adopt an approach to privacy and data protection that is still centred on consent-based grounds for processing, working to achieving more effective ‘notice and consent’. Although the approach taken by PDSs may seem to empower users by giving them more ‘control’, (i) the problems with ‘notice and consent’ as a way of protecting users in digital ecosystems are well-established (Barocas & Nissenbaum, 2009; Sloan & Warner, 2013; Barth & De Jong, 2017; Bietti, 2020), and (ii) it does not fundamentally challenge the logic of those business models and surveillance practices. PDSs therefore remain firmly grounded in the logic of ‘privacy self-management’ (Solove, 2013; Solove, 2020), whereby individuals are expected to manage their own privacy and are themselves held responsible where they fail to adequately do so. This can be understood as part of a broader trend of ‘responsibilisation’ in Western societies (Hannah-Moffat, 2001; Ericson & Doyle, 2003; Brown, 2015); putting ever more responsibility on individuals to manage risks in various aspects of their lives, despite the existence of systemic issues beyond their control that can make doing so difficult if not impossible (such as the asymmetries described in section 4 that PDSs do not sufficiently alleviate). Further, PDSs fail to deal with the realities of collective computations, whereby app developers process user data in aggregate and at scale (subsection 2.2), or with the social nature of personal data (subsection 3.3). Collective computations still exist in—indeed, largely result from—the often commercial drivers for PDS platforms and apps. Through these computations PDSs both allow and contribute to further consolidation of power and information asymmetries (subsection 4.3). However, concerns about collective computations go beyond commercial processing, such as where platforms or app developers pursue public policy or security ends (rather than or additional to commercial gains). This is of significant concern, given the rich, detailed and high-personal nature of the information that a PDS device might capture. Moreover, the social nature of personal data means that individual-level controls are sometimes inappropriate (subsection 3.2)—processing may affect a number of people, only one of whom will have had an opportunity to intervene to permit or constrain it. In all, the individualist approach taken by PDSs, rooted firmly in self-management, does not and cannot capture these more collective, social dimensions of privacy and data protection. The inability of PDSs to adequately address these concerns speaks to a more fundamental issue with PDSs as a concept: they put too much onus on the individual and not enough focus on the business models (or other incentives for data processing). The root cause of the appropriation of user’s personal data is generally not, in fact, the failure of individuals to exercise control over that data, but those surveillance-supported business models that demand the data in the first place. These business models operate at a systemic level, supported by information asymmetries, commercial considerations, legal arrangements (Cohen, 2019), network effects, and other structural factors, and beyond the control of any individual user. Indeed, the information asymmetries inherent in surveillance business models result in a significant asymmetry of power between users and app developers (Mantelero 2014). As Lyon argues, through information asymmetries, surveillance “usually involves relations of power in which watchers are privileged” (Lyon, 2017, p. 15). This power asymmetry is at the core of how surveillance capitalism attempts to extract monetary value from individuals, by modifying their behaviour in pursuit of commercial interests (Zuboff 2015). Yet, as discussed above, PDSs seek to ‘empower’ users without significantly dealing with those asymmetries. Nor do they address other systemic factors with structural causes that disempower users in favour of organisations. While PDSs seek to decentralise processing to users’ devices, then, it does not follow that power will also be decentralised to users themselves: decentralising processing does not necessarily imply decentralising power. Without a more systemic challenge to surveillance-based models for deriving value, shifting away from individualised forms of notice and consent and alleviating the effect of information asymmetries and other structural issues, the underlying power dynamic in those surveillance models—skewed heavily in favour of organisations rather than individuals—remains largely unchanged. Relevant is what Fuchs describes as a form of academic ‘victimisation discourse’, where “privacy is strictly conceived as an individual phenomenon that can be protected if users behave in the correct way and do not disclose too much information” (Fuchs, 2011, p. 146), while issues related to the political economy of surveillance capitalism—advertising, capital accumulation, the appropriation of user data for economic ends—are largely ignored or unchallenged. Responses to these business models that are grounded in placing ever-greater responsibility onto users to actively manage their own privacy, in the face of systemic challenges such as endemic surveillance and data monetisation, are destined to fail. This is the case with PDSs as currently envisaged. Indeed, as previously noted, PDSs have even been described as a way of reducing user ‘resistance’ to data sharing, bringing about a greater ‘willingness’ to allow personal data to be processed (subsection 2.4). This not only explicitly accepts the logic of these business models, but appears to make them easier to pursue. In this way, PDSs following this approach might lull users into a false sense of security through the rhetoric of greater ‘choice’, ‘control’, and ‘empowerment’—despite the evidence that these are flawed concepts in light of the structural and systemic nature of the concerns—while in practice facilitating the very data extraction and monetisation practices that users may be trying to escape. 6. Concluding remarks PDSs are nascent, but growing in prominence. Their proponents claim that PDSs will empower users to get more from their data, and to protect themselves against privacy harms by providing technical and legal mechanisms to enforce their choices around personal data processing. Though, as we have detailed, their ability to deal with the broader challenges associated with current data processing ecosystems appears limited. Regarding data protection, platforms, regulators and lawyers might together work on the specific data issues brought by PDSs, including how best to deal with issues concerning the rights of data subjects. However, despite any such efforts, and regardless of the purported benefits of PDSs, most of the issues inherent to the systemic information asymmetries and challenges in the current ecosystems remain. While PDSs might offer some helpful user-oriented data management tools, they are fundamentally grounded in the mistaken idea that with enough information presented in the right way, individuals will be able to overcome barriers that are ultimately structural and systemic in nature. References Anciaux, N. (2019). Personal Data Management Systems: The security and functionality standpoint. Information Systems, 21, 13 – 35. https://doi.org/10.1016/j.is.2018.09.002 Andrejevic, M. (2011). Surveillance and Alienation in the Online Economy. Surveillance & Society, 8(3), 270 – 287. https://doi.org/10.24908/ss.v8i3.4164 Article 29 Data Protection Working Party. (2007). Opinion 1/2010 on the concepts of ‘controller’ and ‘processor’. (WP169 of 16 February 2010). Article 29 Data Protection Working Party. (2010). Opinion 1/2010 on the concepts of ‘controller’ and ‘processor’. (WP169 of 16 February 2010). Article 29 Data Protection Working Party. (2014). Opinion 8/2014 on Recent Developments on the Internet of Things. (WP 223 of 16 September 2014). Article 29 Data Protection Working Party. (2016). Guidelines on the right to data portability (WP242 rev.01 13 December 2016). Barocas, S., & Nissenbaum, H. (2009). On Notice: The Trouble with 'Notice and Consent’. Proceedings of the Engaging Data Forum: The First International Forum on the Application and Management of Personal Electronic Information. Barth, S., & De Jong, M. (2017). The privacy paradox – Investigating discrepancies between expressed privacy concerns and actual online behavior – A systematic literature review’. Telematics and Informatics, 34(7), 1038 – 1058. https://doi.org/10.1016/j.tele.2017.04.013 Bietti, E. (2020). Consent as a Free Pass: Platform Power and the Limits of the Informational Turn. Pace Law Review, 40, 317 – 398. Binns, R. (2020). Human Judgement in Algorithmic Loops: Individual justice and automated decision-making. Regulation & Governance, 1 – 15. https://doi.org/10.1111/rego.12358 Blume, P. (2012). The inherent contradictions in data protection law. International Data Privacy Law, 2(1), 26 – 34. https://doi.org/10.1093/idpl/ipr020 Bolychevsky, I., & Worthington, S. (2018, October 8). Are Personal Data Stores about to become the NEXT BIG THING? [Blog post]. @shevski. https://medium.com/@shevski/are-personal-data-stores-about-to-become-the-next-big-thing-b767295ed842 Brochot, G. (2015). Personal Data Stores [Report]. Cambridge University. https://ec.europa.eu/digital-single-market/en/news/study-personal-data-stores-conducted-cambridge-university-judge-business-school Brown, W. (2015). Undoing the Demos: Neoliberalism’s Stealth Revolution. Zone Books. Burrell, J. (2016). How the machine “thinks”: Understanding opacity in machine learning algorithms. Big Data & Society, 3(1), 1–12. https://doi.org/10.1177/2053951715622512 Cate, F. H., & Mayer-Schönberger, V. (2013). Notice and consent in a world of Big data. International Data Privacy Law, 3(2), 67 – 73. https://doi.org/10.1093/idpl/ipt005 Chen, J. (2020). Who is responsible for data processing in smart homes? Reconsidering joint controllership and the household exemption. International Data Privacy Law. https://doi.org/10.1093/idpl/ipaa011 Christl, W. (2017). Corporate Surveillance in Everyday Life [Report]. Cracked Labs. https://crackedlabs.org/en/corporate-surveillance Cobbe, J. (2020). What lies beneath: Transparency in online service supply chains. Journal of Cyber Policy, 5(1), 65 – 93. https://doi.org/10.1080/23738871.2020.1745860 Cobbe, J., & Singh, J. (2019). Regulating Recommending: Motivations, Considerations, and Principles. European Journal of Law and Technology, 10(3), 1 – 37. http://ejlt.org/index.php/ejlt/article/view/686 Cohen, J. E. (2019). Between Truth and Power: The Legal Constructions of Informational Capitalism. Oxford University Press. https://doi.org/10.1093/oso/9780190246693.001.0001 ControlShift. (2014). Personal Information Management Services – An analysis of an emerging market: Unleashing the power of trust [Report]. ControlShift. Crabtree, A. (2018). Building Accountability into the Internet of Things: The IoT Databox Model. Journal of Reliable Intelligent Environments, 4, 39 – 55. https://doi.org/10.1007/s40860-018-0054-5 Crabtree, Andy, & Mortier, R. (2015). Human Data Interaction: Historical Lessons from Social Studies and CSCW. In N. Boulus-Rødje, G. Ellingsen, T. Bratteteig, M. Aanestad, & P. Bjørn (Eds.), ECSCW 2015: Proceedings of the 14th European Conference on Computer Supported Cooperative Work, 19-23 September 2015, Oslo, Norway (pp. 3–21). Springer International Publishing. https://doi.org/10.1007/978-3-319-20499-4_1 Directive 2002/58/EC of the European Parliament and of the Council of 12 July 2002 concerning the processing of personal data and the protection of privacy in the electronic communications sector. http://data.europa.eu/eli/dir/2002/58/oj Directive (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of personal data, and repealing Directive 95/46/EC, (2016). E-Privacy Directive – Directive 2002/58/EC of the European Parliament and the Council of 12 July 2002 concerning the processing of personal data and the protection of privacy in the electronic communications sector, (2002). http://data.europa.eu/eli/dir/2002/58/2009-12-19 Ericson, R. V., & Doyle, A. (2003). Risk and Morality. University of Toronto Press. European Commission. (2020). A European strategy for Data. European Union. https://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1593073685620&uri=CELEX%3A52020DC0066 European Data Protection Board. (2019). Opinion 5/2019 on the interplay between the ePrivacy Directive and the GDPR, in particular regarding the competence, tasks and powers of data protection authorities (Opinion No. 5/2019; pp. 38 – 40). European Data Protection Board. Fuchs, C. (2011). An Alternative view on the Privacy of Facebook. Information, 2(1), 140 – 165. https://doi.org/10.3390/info2010140 German Data Ethics Commission. (2019). Gutachten der Deutschen Datenethik Kommission [Expert opinion]. Datenethikkomission. https://datenethikkommission.de/wp-content/uploads/191015_DEK_Gutachten_screen.pdf Hannah-Moffat, K. (2001). Punishment in Disguise: Penal Governance and Canadian Women’s Imprisonment. University of Toronto Press. Janssen, H., Cobbe, J., Norval, C., & Singh, J. (2020). Decentralised Data Processing: Personal Data Stores and the GDPR [Forthcoming]. https://doi.org/10.2139/ssrn.3570895 Janssen, H., Cobbe, J., & Singh, J. (2019). Personal Data Stores and the GDPR’s lawful grounds for processing personal data. Data for Policy, Kings College London. https://doi.org/10.5281/zenodo.3234880 Kamara, I., & De Hert, P. (2018). Understanding the balancing act behind the legitimate interest of the controller ground: A pragmatic approach. (Working Paper No. 4/12; pp. 1 – 33). Brussels Privacy Hub. Kerber, W. (2016). Digital Markets, data, and privacy: Competition law, consumer law and data protection. Journal of Intellectual Property Law & Practice, 11(11), 855 – 866. https://doi.org/10.1093/jiplp/jpw150 Lodge, T. (2018). Developing GDPR compliant apps for the edge. Proceedings of the 13th International Workshop on Data Privacy Management, 313 – 328. https://doi.org/10.1007/978-3-030-00305-0_22 Lyon, D. (2017). Surveillance Studies: An Overview. Polity Press. Mantelero, A. (2014). Social Control, Transparency, and Participation in the Big Data World. Journal of Internet Law, 23 – 29. https://staff.polito.it/alessandro.mantelero/JIL_0414_Mantelero.pdf Myers West, S. (2019). Data Capitalism: Redefining the Logics of Surveillance and Privacy. Business & Society, 58(1), 20–41. https://doi.org/10.1177/0007650317718185 Ng, I., & Haddadi, H. (2018, December 28). Decentralised AI has the potential to upend the online economy. Wired. https://www.wired.co.uk/article/decentralised-artificial-intelligence Nissenbaum, H. (2011). A Contextual Approach to Privacy Online. Dædalus, 140(4), 32–48. https://doi.org/10.1162/DAED_a_00113 Palmås, K. (2011). Predicting What You’ll Do Tomorrow: Panspectric Surveillance and the Contemporary Corporation. Surveillance & Society, 8(3), 338 – 354. https://doi.org/10.24908/ss.v8i3.4168 Pearson, S., & Casassa-Mont, M. (2011). Sticky Policies: An Approach for managing Privacy across Multiple Parties. Computer, 44(9), 60 – 68. https://doi.org/10.1109/MC.2011.225 Poikola, A., Kuikkaniemi, K., & Honko, H. (2014). MyData – A Nordic Model for human-centered personal data management and processing [White Paper]. Open Knowledge Finland. Selbst, A. D., & Powles, J. (2017). Meaningful information and the right to explanation. International Data Privacy Law, 7(4), 233 – 243. https://doi.org/10.1093/idpl/ipx022 Silverman, C. (2019, April 14). Popular Apps In Google’s Play Store Are Abusing Permissions And Committing Ad Fraud. Buzzfeed. Singh, J. (2017). Big Ideas paper: Policy-driven middleware for a legally-compliant Internet of Things. Proceedings of the 17th ACM International Middleware Conference. https://doi.org/10.1145/2988336.2988349 Singh, J. (2019). Decision Provenance: Harnessing Data Flow for Accountable Systems. IEEE Access, 7, 6562 – 6574. https://doi.org/10.1109/ACCESS.2018.2887201 Sloan, R. H., & Warner, R. (2013). Beyond Notice and Choice: Privacy, Norms, and Consent (Research Paper No. 2013–16; pp. 1 – 34). Chicago-Kent College of Law. https://doi.org/10.2139/ssrn.2239099 Solove, D. (2013). Privacy Self-Management and the Consent Dilemma. Harvard Law Review, 126, 1888 – 1903. https://harvardlawreview.org/2013/05/introduction-privacy-self-management-and-the-consent-dilemma/ Solove, D. (2020). February 11. The Myth of the Privacy Paradox (Research Paper No. 2020–10; Law School Public Law and Legal Theory; Legal Studies). George Washington University. https://doi.org/10.2139/ssrn.3536265 The Royal Society. (2019). Protecting privacy in practice: The current use, development and limits of Privacy Enhancing Technologies in data analysis [Report]. The Royal Society. https://royalsociety.org/topics-policy/projects/privacy-enhancing-technologies/ Tolmie, P. (2016, February). This has to be the cats – personal data legibility in networked sensing systems. Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work. https://doi.org/10.1145/2818048.2819992 Urquhart, L. (2018). Realising the Right to Data Portability for the Domestic Internet of Things. Pers Ubiqui Comput, 22, 317 – 332. https://doi.org/10.1007/s00779-017-1069-2 Urquhart, L. (2019). Demonstrably doing accountability in the Internet of Things. International Journal of Law and Information Technology, 2(1), 1 – 27. https://doi.org/10.1093/ijlit/eay015 Wachter, S., & Mittelstadt, B. (2019). A Right to Reasonable Inferences: Re-Thinking Data Protection Law in the Age of Big Data and AI. Columbia Business Law Review, 2, 494 – 620. https://doi.org/10.7916/cblr.v2019i2.3424 Wachter, S., Mittelstadt, B., & Floridi, L. (2017). Why a right to explanation of automated decision-making does not exist in the general data protection regulation. International Data Privacy Law, 7(2), 76–99. https://doi.org/10.1093/idpl/ipx005 Wagner, B. (2019). Liable, but Not in Control? Ensuring Meaningful Human Agency in Automated Decision-Making Systems. Policy & Internet, 11(1), 104 – 122. https://doi.org/10.1002/poi3.198 Yeung, K. (2017). 'Hypernudge’: Big Data as a mode of regulation by design. Information, Communication & Society, 20(1), 118–136. https://doi.org/10.1080/1369118X.2016.1186713 Zuboff, S. (2015). Big other: Surveillance capitalism and the prospects of an information civilization. Journal of Information Technology, 30, 75 – 89. https://doi.org/10.1057/jit.2015.5 Footnotes 1. Note that Hub-of All-Things (HAT) recently changed its name into Dataswift Ltd; Dataswift Ltd represents the commercial enterprise that grew from the university-led HAT research project which was tasked to build the decentralised HAT infrastructure and the governance model. Where we refer in the text to Dataswift, both the HAT project and the commercial enterprise Dataswift are considered within our analysis. 2. Note that Solid offers the technical infrastructure, while Inrupt is the company offering services that are built on that infrastructure. Where we refer to Inrupt, both the technical infrastructure and the company services come within our analysis. 3. This article builds on our earlier comparative analysis of commercial PDS offerings and different PDS formulations, as focused on data protection concerns (Janssen et al., 2020). 4. Note that a 'device' is conceptual, and can be underpinned by a range of technical architectures. In describing the data and processing 'within' a device, we refer to that logically governed by the device. This means, for example, that the data and compute might not necessarily occur all within a single technical component, but could potentially occur in various locations, e.g. across a range of (managed) cloud services. 5. Note that the terminology varies by platform; not all platforms would describe processing as occurring through apps, though generally there is some conceptually similar construct. 6. Note that despite the similar terms (devices, apps, app stores), PDS differ from mobile ecosystems, in that PDSs are governance oriented, with far richer and granular controls. Moreover, the degree of resemblance will depend on the specific formulation of the PDS and its ecosystem – many different approaches are possible. 7. We use ‘collective computation’ simply to refer to computation that occurs across a range of user devices. There is potential for the methods facilitating such computation to employ privacy-enhancing mechanisms (e.g. The Royal Society, 2019). 8. Note that differences exist as to what PDSs require from app developers to describe in their manifests. Databox envisages to assess risks as to whether an app developer intends to share the data with third parties, while other platforms might not envisage any risk assessment on this aspect (or it i...
Towards platform observability
1. Introduction Platforms are large-scale infrastructures specialised in facilitating interaction and exchange among independent actors. Whether understood economically as two- or multi-sided markets (Langley & Leyshon, 2017) or with an eye on online media as services that ‘host, organize, and circulate users’ shared content or social interactions’ (Gillespie, 2018, p. 18), platforms have not only become highly visible and valuable companies but also raise important social challenges. While intermediaries have in one form or another existed for millennia, contemporary platforms are relying on digital technologies in (at least) two fundamental ways. First, platforms ‘capture’ (Agre, 1994) activities by channelling them through designed functionalities, interfaces, and data structures. Uber, for example, matches riders with drivers in physical space, handles payment, and enforces ‘good behaviour’ through an extensive review system covering both parties. This infrastructural capture means that a wide variety of data can be generated from user activity, including transactions, clickstreams, textual expressions, and sensor data such as location or movement speed. Second, the available data and large numbers of users make algorithmic matching highly attractive: ranking, filtering, and recommending have become central techniques for facilitating the ‘right’ connections, whether between consumers and products, users and contents, or between people seeking interaction, friendship, or love. Digital platforms host social exchange in ways that Lawrence Lessig (1999) summarised under the famous slogan ‘code is law’, which holds that technical means take part in regulating conduct and shaping outcomes. The combination of infrastructural capture and algorithmic matching results in forms of socio-technical ordering that make platforms particularly powerful. As Zuboff (2019, p. 15) discusses under the term surveillance capitalism, the tight integration of data collection and targeted ‘intervention’ has produced ‘a market form that is unimaginable outside the digital milieu’. The rising power of platforms poses the question of what kind of accountability is necessary to understand these processes and their consequences in more detail. Matching algorithms, in particular, represent ordering mechanisms that do not follow the same logic as traditional decision-making, leading to considerable uncertainty concerning their inner workings, performativities, and broader social effects. So far, most regulatory approaches to tackling these questions seek to create accountability by ‘opening the black box’ of algorithmic decision-making. A recent EU regulation on fairness in platform-to-business relations, for example, proposes transparency as its principal means. 1 The public debate about the upcoming EU Digital Services Act indeed shows that calls for transparency of algorithmic power have gained support across parliamentary factions and stakeholder groups. 2 The ‘Filter Bubble Transparency Act’—a US legislative proposal that seeks to protect users from being ‘manipulated by algorithms driven by user-specific data’ - focuses more specifically on platforms as media, but again relies on transparency as guiding principle. 3 The German Medienstaatsvertrag (‘State Media Treaty’), which has recently been ratified by all state parliaments, explicitly requires platform operators to divulge criteria for ranking, recommendation, and personalisation ‘in a form that is easily perceivable, directly reachable, and permanently available’. 4 This widespread demand for disclosure and explanation articulates not only justified concerns about the opacity of platforms but also testifies to the glaring lack of information on their conduct and its social, political, and economic repercussions. In this paper, we likewise take up the challenge posed by platform opacity from the angle of accountability but seek to probe the conceptual and practical limitations of these transparency-led approaches to platform regulation. Echoing the critical literature on transparency as a policy panacea (e.g., Etzioni, 2010; Ananny & Crawford, 2018), we propose the concept of observability as a more pragmatic way of thinking about the means and strategies necessary to hold platforms accountable. While transparency and observability are often used synonymously (e.g. August & Osrecki, 2019), we would like to highlight their semantic differences. Unlike transparency, which nominally describes a state that may exist or not, observability emphasises the conditions for the practice of observing in a given domain. These conditions may facilitate or hamper modes of observing and impact the capacity to generate external insights. Hence, while the image of the black box more or less skips the practicalities involved in opening it, the term observability intends to draw attention to and problematise the process dimension inherent to transparency as a regulatory tool. While observability incorporates similar regulatory goals to transparency, it also deviates in important respects, most importantly by understanding accountability as a complex, dynamic ‘social relation’ (Bovens, 2007, p. 450), which is embedded in a specific material setting. The goal is not to exchange one concept for the other but to sharpen our view for the specificities of platform power. At the risk of stating the obvious, regulatory oversight needs to take into account the material quality of the objects under investigation. Inspecting the inner workings of a machine learning system differs in important ways from audits in accounting or the supervision of financial markets. Rather than nailing down ‘the algorithm’, understood as a singular decision mechanism, the concept of observability seeks to address the conditions, means, and processes of knowledge production about large-scale socio-technical systems. In the everyday life of platforms, complex technologies, business practices, and user appropriations are intersecting in often unexpected ways. These platform dynamics result in massive information asymmetries that affect stakeholder groups as well as societies at large. Regulatory proposals need to take a broader view to live up to these challenges. Our argument proceeds in three steps. In the next section, we retrace some of the main problems and limitations of transparency, paying specific attention to technical complexity. The third section then discusses the main principles guiding the observability concept and provides concrete examples and directions for further discussion. We conclude by arguing for a policy approach to promoting observability, emphasising that institutional audacity and innovation are needed to tackle the challenges raised by digital platforms. 2. Limitations to transparency Much of the debate around our insufficient understanding of platforms and their use of complex algorithmic techniques to modulate users’ experience has centred on the metaphor of a ‘black box’. Although Frank Pasquale, whose Black Box Society (2015) has popularised the term beyond academia, prefers the broader concept of intelligibility, the talk of black boxes is often accompanied by demands for transparency. The regulatory proposals mentioned above are largely organised around mechanisms such as explanations, disclosures, and—more rarely—audits 5 that would bring the inner workings of the machine to light and thereby establish some form of control. But these calls for transparency as a remedy against unchecked platform power encounter two sets of problems. First, the dominant understanding of transparency as information disclosure faces important limitations. Second, the object under scrutiny itself poses problems. Platforms are marked by opacity and complexity, which effectively challenges the idea of a black box whose lid can be lifted to look inside. This section discusses both of these issues in turn. 2.1. Accountability as mediated process Transparency has a long tradition as a ‘light form’ (Etzioni, 2010) of regulation. It gained new popularity in the 1970s as a neoliberal governance method, promising better control of organisational behaviour through inspection (August & Osrecki, 2019). Transparency is seen as an essential means of oversight and of holding commercial and public entities to account: only if powerful organisations reveal relevant information about their actions are we able to assess their performance. This understanding of transparency implies a number of taken for granted assumptions, which link information disclosure to visibility, visibility to insight, and insight to effective regulatory judgement (Ananny & Crawford, 2018, p. 974). According to this view, transparency is able to reveal the truth by reflecting the internal reality of an organisation (Albu & Flyverbom, 2019, p. 9) and thereby creating ‘representations that are more intrinsically true than others’ (Ananny & Crawford, 2018, p. 975). Making the opaque and hidden visible, creates truth and truth enables control, which serves as a ‘disinfectant’ (Brandeis, 1913, p. 10) capable of eliminating malicious conduct. Transparency is considered crucial for the accountability of politics because seeing, just as in the physical world, is equated with knowing: ‘what is seen is largely what is happening’, as Ezrahi (1992, p. 366) summarises this view. These assumptions also inform current considerations on platform regulation. However, recent research on transparency has shown that transparency does more and different things than shedding light on what is hidden. The visibility of an entity and its procedures is not simply a disclosure of pre-existing facts, but a process that implies its own perspective. While transparency requirements expect ‘to align the behavior of the observed with the general interest of the observers’, empirical studies found that ‘transparency practices do not simply make organizations observable, but actively change them’ (August & Osrecki, 2019, p. 16). As Flyverbom (2016, p. 15) puts it, ‘transparency reconfigures - rather than reproduces - its objects and subjects’. The oversight devices used to generate visibility shape what we get to see (Ezrahi, 1992; Flyverbom, 2016), which puts into question the idea of direct, unmediated access to reality if only the disclosed information is accurate. From a social science perspective, transparency should not be regarded as a state or a ‘thing’ but as the practice ‘of deciding what to make present (i.e. public and transparent) and what to make absent’ (Rowland & Passoth, 2015, p. 140). Creating visibility and insights as part of regulatory oversight consists of specific procedures, which involve choices about what specifically should be exposed and how, what is relevant and what can be neglected, which elements should be shown to whom and, not least, how the visible aspects should be interpreted (Power, 1997). In their critique of transparency-led approaches to algorithmic accountability, Ananny & Crawford (2018) moreover argue that there is a distinct lack of sensitivity for fundamental power imbalances, strategic occlusions, and false binaries between secrecy and openness, as well as a broad adherence to neoliberal models of individual agency. In light of these criticisms, it may not come as a surprise that regulatory transparency obligations often fall short of their goals and create significant side-effects instead. Among the most common unintended outcomes are bureaucratisation, generalised distrust, and various forms of ‘window dressing’ designed to hide what is supposed to be exposed to external review. Informal organisational practices emerge and coexist with official reports, accounts, and presentations (August & Orecki, 2019, p. 21). While the critical literature on regulatory failures of transparency obligations is increasing, these insights have yet to have an impact on regulatory thinking. Most regulatory proposals resort to traditional ideas of external control through transparency and frame transparency as a straightforward process of disclosure. As a result, they are missing the mark on the complex and conflictual task of creating meaningful understanding that can serve as an effective check on platform power. Taken together, a social science perspective on this key ideal of regulation suggests that making platforms accountable requires a critical engagement with the achievements and shortcomings of transparency. It needs to take on board efforts to combine different forms of evidence, and above all, to become attentive to the selective and mediated character of knowledge-building. Similar to the flawed logic of ‘notice and consent’ in the area of privacy protection, which holds that informing individuals on the purposes of data collection allows them to exercise their rights, a superficial understanding of transparency in the area of platform regulation risks producing ineffective results (see Obar, 2020; Yeung, 2017). 2.2. Opacity, complexity, fragmentation A second set of complications for transparency concerns algorithms and platforms as the actual objects of scrutiny. Large-scale technical systems, in particular those incorporating complex algorithmic decision-making processes, pose severe challenges for assessing their inner workings and social effects. One obvious reason for this is indeed their opacity. As Burrell (2016, p. 2) argues, opacity may stem from secrecy practices, lack of expertise in reading code, and the increasing ‘mismatch between mathematical optimization in high-dimensionality characteristic of machine learning and the demands of human-scale reasoning’. The last point in particular introduces significant challenges to transparency understood as information disclosure or audit. Even if decision procedures behind automated matchmaking can sometimes still be meticulously specified, platforms nowadays mainly deploy statistical learning techniques. These techniques develop decision models inductively and ‘learn programs from data’ (Domingos, 2012, p. 81), based on an arrangement between data, feedback, and a given purpose (see Rieder, 2020). In the canonical example of spam filtering, users label incoming emails as spam or not spam. Learning consists in associating each word in these messages with these two categories or ‘target variables’. Since every word contributes to the final decision to mark an incoming message as spam or not spam, the process cannot be easily traced back to singular factors. Too many variables come into play, and these algorithms are therefore not ‘legible’ in the same way as more tangible regulatory objects. With regard to regulatory oversight, this means that transparency in the sense of reconstructing the procedure of algorithmic decision making ‘is unlikely to lead to an informative outcome’, as Koene et al. (2019, p. II) conclude. Audits are unable to find out ‘what the algorithm knows because the algorithm knows only about inexpressible commonalities in millions of pieces of training data’ (Dourish, 2016, p. 7). There is a large gulf between the disclosure of ‘fundamental criteria’ mandated by regulatory proposals like the Medienstaatsvertrag and the technical complexities at hand. Even if regulators were given access to data centres and source code, the process of sense-making would not be straightforward. Reading the gist of an algorithm from complex code may run into difficulties, even if no machine learning is involved. As Dourish (2016) shows, the presence of different programming languages and execution environments adds further complications, and so do the many subsystems and modules that concrete programmes often draw on. Algorithmic decision procedures ‘may not happen all in one place’ (Dourish, 2016, p. 4) but can be distributed over many different locations in a large programme or computer network. In the case of online advertising, for example, the placement of a single ad may entail a whole cascade of real-time auctions, each drawing on different algorithms and data points, each adding something to the final outcome. The result is a continuously evolving metastable arrangement. Thus, time becomes a crucial analytical factor, causing considerable difficulties for the ‘snapshot logic’ underlying most audit proposals. For these reasons, algorithms turn out to be difficult to locate. In his ethnographic study of a recommender system, Seaver (2017) observes that even in small companies it can be a challenge for staff members to explain where exactly ‘the algorithm’ is. As Bogost (2015) quips, ‘[c]oncepts like “algorithm” have become sloppy shorthands, slang terms for the act of mistaking multipart complex systems for simple, singular ones’. What is referred to as ‘algorithm’, i.e. the actual matchmaking technique, may thus only be a small component in a much larger system that includes other various instances of ordering, ranging from data modelling to user-facing interfaces and functions that inform and define what users can see and do. YouTube, for example, not only fills its recommendation pipeline with a broad array of signals generated from the activities of billions of users but actually uses two different deep learning models for ‘candidate generation’ (the selection of hundreds of potential videos from the full corpus) and ‘ranking’ (the selection and ordering of actual recommendations from the candidate list) (see Covington et al., 2016). The fuzzy, dynamic, and distributed materiality of contemporary computing technologies and data sets means that algorithmic accountability is harder to put into practice than the call for transparency suggests. Regulatory proposals such as disclosures, audits, or certification procedures seeking to establish effective control over their functionality and effects assume properties that algorithmic systems may often not meet. Suffice to say that technical complexity also facilitates the attempts at dissimulation and ‘window dressing’ mentioned above. Yet, as if this was not difficult enough, our understanding of platform accountability should extend beyond oversight of algorithms and platform conduct to be meaningful. The ordering power of platforms also encompasses shared or distributed accomplishments (see Suchman, 2007) to which platforms, users and content providers each contribute in specific ways. As Rahwan et al. (2019, p. 477) argue, machine behaviour ‘cannot be fully understood without the integrated study of algorithms and the social environments in which algorithms operate’. The actions of users, for example, provide the data that shape algorithmic models and decisions as part of machine learning systems. In the same vein, platform behaviour cannot be reduced to platform conduct, that is, to the policies and design decisions put in place by operators. It must include the evolving interactions between changing social practices and technical adjustments, which may, in turn, be countered by user appropriations. As use practices change, algorithmic decision models change as well. Platform companies are therefore neither fully in control of actual outcomes, nor fully aware what is happening within their systems. Finally, the effects of platforms can only be sufficiently addressed if we consider what is being ordered. For example, ranking principles considered beneficial in one culture domain, e.g. music recommendation, may have troubling implications in another, e.g. the circulation of political content. Accountability thus has to consider what is made available on platforms and how ordering mechanisms interact with or shape the content and its visibility. This again requires a broader view than what algorithm audits or broad technical disclosures are able to provide. Taken together, research on the properties of algorithms and algorithmic systems suggests that regulatory proposals such as ‘opening the black box’ through transparency, audit, or explainability requirements reflect an insufficient understanding of algorithms and the platform architectures they enable. Algorithms can neither be studied nor regulated as single, clear-cut, and stable entities. Rather, their behaviour and effects result from assemblage-like contexts whose components are not only spatially and functionally distributed but also subject to continuous change, which is partly driven by users or markets facilitated by platforms. Given the ephemeral character of algorithms on the one side and the enormous generative and performative power of algorithmic systems on the other, the question arises what concepts, strategies, and concrete tools might help us to comprehend their logics and to establish effective political oversight. Such an approach needs to take on board the critique of transparency as a regulatory tool and consider accountability as a continuous interaction and learning process rather than periodical undertakings. It should recognise that the legibility of algorithmic systems significantly differs from that of other objects or areas of regulation; and it should take into account that any form of review is not only selective but also shapes the object under investigation. Thus, the debate on platform regulation needs to become reflexive with regard to the specific materiality of the regulatory field and the constitutive effects of studying it. 3. Principles of observability This section seeks to flesh out an understanding of observability as a step toward tackling the problems platform accountability currently faces. While the term is regularly used in the literature on transparency (e.g., Bernstein, 2012; Albu & Flyverbom, 2015; August & Osrecki, 2019), we seek to calibrate it to our specific goals: the challenges raised by platforms as regulatory structures need to be addressed more broadly, beginning with the question of how we can assess what is happening within large-scale, transnational environments that heavily rely on technology as a mode of governance. Who gets treated how on large online platforms, how are connections between participants made and structured, what are the outcomes, and—crucially—who can or should be able to make such assessments? Rather than a binary between transparency and opacity, the question is how to foster the capacity to produce knowledge about platforms and ‘platform life’ in constructive ways. The increasingly technological nature of our societies requires not just penalties for law infringements, but a deeper and well-informed public conversation about the role of digital platforms. This includes attention to the larger impacts of the new kinds of ordering outlined above, as well as a sensitivity for the ideological uses of transparency, which may serve ‘as a tool to fight off the regulations opposed by various business groups and politicians from conservative parties’ (Etzioni, 2010, p. 2). We therefore position observability as an explicit means of, not an alternative to regulation. As van Dijck et al. (2018, p. 158) underline, ‘[r]egulatory fixes require detailed insights into how technology and business models work, how intricate platform mechanisms are deployed in relation to user practices, and how they impact social activities’. Our concept of observability thus seeks to propose concrete actions for how to produce these insights. While some of the more concrete strategies we discuss may come out of self-regulation efforts, effective and robust observability clearly requires a regulatory framework and institutional support. In what follows, we outline three principles that inform the concrete conceptual and practical directions observability seeks to emphasise. 3.1. Expand the normative and analytical horizon The first principle concerns the research perspective on platforms and argues that a broader focus is needed. This focus takes into consideration how digital platforms affect societies in general, ranging from everyday intimacy to economic and labour relations, cultural production, and democratic life. Given that platformisation transforms not only specific markets but ‘has started to uproot the infrastructural, organizational design of societies’ (van Dijck, 2020, p. 2), it seems crucial to develop knowledge capacities beyond critical algorithm studies and include platform conduct, behaviour, and effects across relevant social domains in our agendas. As Powles and Nissenbaum (2018) have recently argued for artificial intelligence systems, limiting our focus to the important yet narrow problems of fairness and biases means that ‘vast zones of contest and imagination are relinquished’, among them the question whether the massive efforts in data collection underlying contemporary platform businesses are acceptable in the first place. The ability to say no and prohibit the deployment of certain technologies such as political micro-targeting of voters or face recognition requires robust empirical and normative evidence on its harm for democracies. While investigations into misinformation and election tampering are important, there are other long-term challenges waiting to be addressed. Recent studies on surveillance capitalism (Zuboff, 2019), digital capitalism (Staab, 2019), informational capitalism (Cohen, 2019), the platform society (van Dijck et al., 2018), or the ‘dataist state’ (Fourcade & Gordon, 2020) aim to capture and make sense of the ongoing structural changes of societies and economies, including the power shifts these imply. EU commissioner Vestager recently evoked Michel Foucault’s notion of biopower when addressing novel data-based techniques of classifying, sorting, and governing (Stolton, 2019). While the term addresses a set of political technologies that emerged in the 19th century to manage the behaviour of populations by means of specific regimes of knowledge and power, digital platforms’ considerable reach and fine-grained ‘capture’ (Agre, 1994) of everyday activities invites comparison. The deep political and social repercussions these conceptual frames highlight require broader forms of social accountability (Bovens, 2007) than disclosures or audits are able to provide. How can researchers, regulators, and civil society expand their capacity to study, reflect and act on these developments? The concept of observability starts from the recognition of a growing information asymmetry between platform companies, a few data brokers, and everyone else. The resulting data monopoly deprives society of a crucial resource for producing knowledge about itself. The expanding data sets on vast numbers of people and transactions bear the potential for privileged insights into societies’ texture, even if platforms tend to use them only for operational purposes. AirBnB’s impact on urban development, Uber’s role in transforming transportation, Amazon’s sway over retail, or Facebook and Twitter’s outsized influence on the public sphere cannot be assessed without access to relevant information. It is symptomatic that companies refuse access to the data necessary for in-depth, independent studies and then use the lack of in-depth, independent studies as evidence for lack of harm. New modes of domination are unfolding as part of analytics-driven business models and the unprecedented information asymmetries they bring about. Powles and Nissenbaum (2019) therefore argue that we need ‘genuine accountability mechanisms, external to companies and accessible to populations’. An essential condition and experimental construction site for such accountability mechanisms would be the institutionalisation of reliable information interfaces between digital platforms and society—with a broad mandate to focus on the public interest. We propose the concept of public interest as a normative reference for assessing platform behaviour and regulatory goals. However, public interest is neither well defined nor without alternatives. 6 We prefer public interest over the closely related common good because the former refers to an internationally established mandate in media regulation and could thus inform the formulation of specific requirements or ‘public interest obligations’ for platforms as well (Napoli, 2015, p.4). Furthermore, the concept speaks to our specific concern with matters of governance of platform life. The use of public interest spans different disciplinary and regulatory contexts, and it is open to flexible interpretation. Yet, the often-criticised vagueness of the concept has the advantage of accommodating the broad range of existing platforms. As a normative framework it can be used to critically assess the design of multiple-sided markets as much as the impact of digital intermediaries on the public sphere. Approaches to defining and operationalising public interest depend on the context. In economic theory, public interest is suspected of functioning as a ‘weapon’ for justifying regulatory intervention into markets for the purpose of enhancing social welfare (Morgan & Yeung, 2007). Correcting failing markets constitutes a minimalist interpretation of public interest, however. In politics, public interest is associated with more diverse social goals, among them social justice, non-discrimination, and access to social welfare; or more generally the redistribution of resources and the maintenance of public infrastructures. With regard to the public sphere and the media sector, public interest refers to protecting human rights such as freedom of information and freedom of expression, fostering cultural and political diversity, and not least sustaining the conditions for democratic will formation through high quality news production and dissemination (Napoli, 2015). What these different understandings of public interest have in common is a focus on both procedural and substantial aspects. Obviously, public interest as a frame of reference for assessing and regulating digital platforms is not a given. Rather, the meaning and principles of public interest have to be constantly negotiated and reinterpreted. As van Dijck (2020, p. 3) reminds us, such battles over common interest do not take place in a vacuum, they are ‘historically anchored in institutions or sectors’ and ‘after extensive deliberation’ become codified in more or less formal norms. From a procedural point of view, public interest can also be defined as a practice, which has to meet standards of due process such as inclusiveness, transparency, fairness, and right to recourse (Mattli & Woods, 2009, p. 15). In terms of substance, the notion of public interest clearly privileges the collective common welfare over that of individuals or private commercial entities. In this respect, it entails a departure from the neoliberal focus on individual liberty toward collective freedoms. Thereby it also extends the space of policy options beyond ‘notice and consent’ to more far-reaching regulatory interventions (Yeung, 2017, p. 15). We see similar conceptual adjustments toward public interest in other areas such as the discourse on data protection. As Parsons (2015, p. 6) argues, it is necessary to recognise ‘the co-original nature of [...] private and public autonomy’ to understand that mass surveillance is not merely violating citizens’ individual rights, but ‘erodes the integrity of democratic processes and institutions’ (p. 1). To conclude, the concept of observability emphasises the societal repercussions of platformisation and suggests public interest as a normative horizon for assessing and regulating them. It problematises the poor conditions for observing platform life and its effects, and suggests levelling off, in institutionalised ways, the information asymmetry between platforms and platform research. Thus, we think of observability as one possible ‘counter power’ in the sense of Helberger (2020, p. 9) who calls for establishing ‘entirely new forms of transparency’. First and foremost, observability therefore seeks to improve the informational conditions for studying the broader effects of platformisation. Over the next two sections, we discuss the modalities for such an approach. 3.2. Observe platform behaviour over time Building on the arguments laid out in section two, the second principle of observability holds that the volatility of platforms requires continuous observation. While ex ante audits of technical mechanisms and ex post analysis of emblematic cases are certainly viable for more restricted systems, the dynamic and distributed nature of online platforms means that intermittent inspections or disclosures are insufficient, thwarted by the object’s transient character. Traditional forms of information sharing through transparency reports, legal inquiries, and regulated and structured disclosures, similar to those that exist for stock markets, can still be part of an observability framework, as can investigative reporting and whistleblowing. However, to tackle the specific challenges of digital platforms, more continuous forms of observation need to be envisaged. When terms of service, technical design, or business practices change, the ‘rules of the game’ change as well, affecting platform participants in various ways. Projects like TOSBack 7 use browser plugins and volunteer work to track and observe changes in platforms’ terms of service continuously, that is, while they are happening and not after some complaint has been filed. These are then distilled into more readable forms to accommodate wider audiences. The joint Polisis 8 and PriBot 9 projects pursue similar goals, drawing on artificial intelligence to interpret privacy policies and deal with the limitations of volunteer work. Such efforts should be made easier: a recent proposal by Cornelius (2019) suggests making terms of service contracts available as machine-readable documents to facilitate ongoing observation and interpretation. Similar approaches can be imagined for other areas of platform conduct, including technical tweaks or changes in business practices. However, to account for the distributed and dynamic character of platform life, as it emerges from the interaction between policies, design choices, and use practices, continuous observation needs to reach beyond legal and technical specifications. Bringing the space of distributed outcomes into view is by no means easy, but the importance of doing so is increasingly clear. In their discussion of algorithms as policies, Hunt and McKelvey (2020, p. 330) indeed argue that the ‘outcomes of these policies are as inscrutable as their intentions - under our current system of platform governance, it is beyond our reach to know whether algorithmic regulation is discriminatory or radicalizing or otherwise undermines the values that guide public policy’. Here, observability does not alter the underlying normative concerns but asks how platform reality can be sufficiently understood to make it amenable to normative reasoning in the first place. As platforms suck the bulk of online exchange into their increasingly centralised infrastructures, we need the capacity to probe not merely how algorithms work, but how fundamental social institutions are being reshaped. Answering these questions requires studying technical and legal mechanisms, use practices, and circulating units such as messages together. Given that our first goal is to understand rather than to place blame, there is no need to untangle networks of distributed causation from the outset. Entanglement and the wide variety of relevant questions we may want to ask mean that observability thus favours continuous and broad access to knowledge generating facilities. There are at least four practical approaches that align with what we are aiming at. First, platforms have occasionally entered into data access agreements with researchers, journalists, NGOs, and so forth. Facebook is a case in point. The company’s Data for Good 10 programme, which builds ‘privacy-preserving data products to help solve some of the world's biggest problems’, shares data with approved universities and civil society groups. The recently launched Social Science One initiative 11, a collaboration with the US Social Science Council, is supposed to grant selected researchers access to both data and funding to study ‘the impact of social media on elections and democracy’ (King & Persily, 2019, p. 1). While these initiatives are good starting points, they have been plagued by delays and restrictions. Scholars have rightfully criticised that the scope and modalities for access remain in the hands of platforms themselves (Hegelich, 2020; Suzor et al., 2019). The central question is thus how to structure agreements in ways that asymmetries between platforms and third parties are reduced. Without a legal framework, companies can not only start and stop such initiatives at will but are also able to control parameters coming into play, such as thematic scope, coverage, and granularity. Accountability interfaces providing continuous access to relevant data constitute a second direction. Facebook’s Ad Library 12, for example, is an attempt to introduce carefully designed observability, here with regard to (political) advertisement. Despite the limitations of the existing setup (see Leerssen et al., 2019), machine-readable data access for purposes of accountability can enable third-party actors to ask their own questions and develop independent analytical perspectives. While tools like Google Trends 13 are not designed for accountability purposes, a broader understanding of the term could well include tools that shed light on emergent outcomes in aggregate terms. There are already working examples in other domains, as the German Market Transparency Unit for Fuels 14, a division of the Federal Cartel Office shows. It requires gas stations to communicate current prices in real-time to make them available on the Web and via third-party Apps. 15 Well-designed data interfaces could both facilitate observability and alleviate some of the privacy problems other approaches have run into. One could even imagine sandbox-style execution environments that allow third parties to run limited code within platforms’ server environment, allowing for privacy-sensitive analytics where data never leaves the server. Developer APIs are data interfaces made available without explicit accountability purposes. These interfaces have been extensively repurposed to investigate the many social phenomena platforms host, ranging from political campaigning (e.g. Larsson, 2016) to crisis communication during disasters (e.g. Bruns & Burgess, 2014), as well as the technical mechanisms behind ranking and recommendation (e.g., Airoldi et al., 2016; Rieder et al., 2018). Depending on the platform, developer APIs provide data access through keyword searches, user samples, or other means. Twitter’s random sample endpoint 16, which delivers representative selections of all tweets in real time (Morstatter et al., 2014), is particularly interesting since it allows observing overall trends while reducing computational requirements. One of the many examples for exploiting a data interface beyond social media is David Kriesel’s project BahnMining 17, which uses the German railroad’s timetable API to analyse train delays and challenge the official figures released by Deutsche Bahn. But the so-called ‘APIcalypse’ (Bruns, 2019) that followed the Facebook-Cambridge Analytica scandal has led to restrictions in data access, rendering independent research much more difficult. Even before Facebook-Cambridge Analytica, working with developer APIs regularly created issues of reliability and reproducibility of results, research ethics, and privacy considerations (see Puschmann, 2019). Generally, developer interfaces are not designed for structured investigations into the layers of personalisation and localisation that may impact what users actually see on their screens. YouTube’s ‘up next’ column is a case in point: while the API does make so-called ‘related videos’ available, it leaves out the personalized recommendations that constitute a second source for suggested videos. Research on the YouTube’s recommender system, for example a study by PEW 18, is therefore necessarily incomplete. But the fact that developer APIs enable a wide variety of independent research on different topics means that in cases where privacy concerns can be mitigated, they are worth extending further. A structured conversation between platforms and research organisations about possible long-term arrangements is necessary and independent regulatory institutions could play a central role here. Finally, due to API limitations, researchers have been relying on scraping, a set of techniques that glean data from end-user interfaces. Search engines, price snipers, and a whole industry of information aggregators and sellers rely on scraped data, but there are many non-commercial examples as well. Projects like AlgoTransparency 19, run by former YouTube employee Guillaume Chaslot, regularly capture video recommendations from the web interface to trace what is being suggested to users. Roth et al. (2020) have recently used a similar approach to study whether YouTube indeed confines users to filter bubbles. Such high-profile questions call for empirical evidence, and since research results may change as quickly as systems evolve, continuous monitoring is crucial. While scraping does not demand active cooperation from the platforms under scrutiny, large-scale projects do require at least implicit acquiescence because websites can deploy a whole range of measures to thwart scraping. Although more precarious than API-based approaches, taking data directly from the user interface allows for the explicit study of personalisation and localisation. Data retrieved through scraping may also serve to verify or critique data obtained through the previously mentioned techniques. Not unlike the panels assembled by analytics companies like Nielsen for their online products 20, the most promising platform-centred crowd-sourcing projects ask volunteers to install custom-built browser plugins to ‘look over their shoulder’. The Datenspende project, a collaboration between several German state-level media authorities, the NGO AlgorithmWatch, the Technical University Kaiserslautern, and Spiegel Online, recruited 4,500 volunteers before the German parliamentary elections in 2017 to investigate what users actually see when they look for party and candidate names on Google Search and Google News. 21 The same approach was later used to scrutinise the SCHUFA 22, Germany’s leading credit bureau, and most recently Instagram 23. There are many other areas where scraping has been productively used. The $herrif project 24, for example, also deployed browser plugins to investigate price discrimination practices on retail websites like Amazon (Iordanou et al., 2017). Even regulators have to resort to scraping: a recent study by the French Conseil Supérieur de l’Audiovisuel used the accounts of 39 employees and four fictitious users to study YouTube’s recommendation system. 25 The City of Amsterdam already began scraping data from AirBnB in 2017 26, analysing consequences for the housing market and compliance by landlords with rules on short-term rentals. Given that sample quality, scale, and the dependence on platform acquiescence are significant disadvantages under current conditions, a legal framework regulating access to platform data would increase the practical viability of this approach. The current ambiguities risk creating chilling effects that discourage smaller research projects in particular. NYU’s Ad Observer 27, a tool that uses browser plugins and scraping to investigate ad targeting on Facebook to compensate for the limitations of the above-mentioned Ad Library, tells a cautionary tale. The researchers recently received a cease and desist letter from the company, putting the whole project in peril (Horwitz, 2020). However, it should be stated that not all forms of access to platform data further the public interest. Across all these four approaches we encounter serious privacy concerns. While there are areas where data access is unproblematic, others may require restricting access to certain groups, anonymise data, use aggregate statistics, or explore innovative models such as sandbox environments. These are not trivial problems; they raise the need for innovative and experimental approaches supported by institutional oversight. From a legal perspective, a recent interpretation of the GDPR by the European Data Protection Supervisor 28 clarified that research in the public interest must have leeway if done in accordance with ethical best practices. Still, concrete measures will need to be the subject of broader conversations about the appropriate balance to strike, which may lead, in certain cases, to more restrictions rather than fewer. 3.3. Strengthen capacities for collaborative knowledge creation In his analysis of accountability as a social relation, Bovens (2007, p. 453) argues that ‘transparency as such is not enough to qualify as a genuine form of accountability, because transparency does not necessarily involve scrutiny by a specific forum’. Given their deep and transversal impact, the question as to how knowledge about platforms is generated and how it circulates through society is crucial. In this section, we argue that effective accountability requires the participation of different actors and the generation of different forms of knowledge. Our argument starts from the fact that platform companies have largely treated information about their systems, what users are posting or selling, and which kind of dynamics emerge from their interactions as private assets. They heavily invest in sophisticated analytics to provide insights and pathways for corporate action. Product development, optimisation, and detection and moderation of all kinds of illegal or ‘undesirable’ content have become important tasks that fully rely on evolving observational capabilities. While platforms would be able to facilitate knowledge creation beyond such operational concerns, the existing information asymmetries between those collecting and mining private data and society at large make this highly unlikely. Instead, platforms provide businesses and individual users with deliberately designed ‘market information regimes’ (Anand & Peterson, 2000) consisting of analytics products and services that provide information about the larger market and one’s own standing. Creators on YouTube, for example, are now able to gauge how their videos are faring, how the choice of thumbnails affects viewer numbers, or how advertisers are bidding on keywords within the platform interface. But such interfaces are ‘socially and politically constructed and [...] hence fraught with biases and assumptions’ (Anand & Peterson, 2000, p. 270), privileging operational knowledge designed to boost performance over broader and more contextualised forms of insight. The narrow epistemological horizon of platform companies thus needs to be supplemented by inquiries that contextualise and question this business model. The problematic monopolisation of analytical capacities legitimises our demand for a more inclusive approach, which would open the locked-up data troves to qualified external actors. However, there simply is no one-size-fits-all approach able to cover all types of platforms, audiences, and concerns. Researchers, journalists, and activists are already engaged in ‘accountability work’, covering a range of questions and methods. Regulators add to this diversity: competition and antitrust inquiries require different forms of evidence than concerns regarding misinformation or radicalisation. We may therefore prefer to speak of ‘accountabilities’ in plural form. There are many approaches coming from the technical disciplines that promise to enhance understanding. Emerging research fields like ‘explainable AI’ (e.g. Doran et al., 2017) seek to make primary ordering mechanisms more accountable, even if the issue remains of what ‘explainable’ means when different audiences ask different questions. Other strategies like the ‘glass box’ approach (Tubella & Dignum, 2019) focus on the monitoring of inputs and outputs to ‘evaluate the moral bounds’ of AI systems. A particularly rich example for image classification from Google Researchers comes in the form of an ‘activation atlas’, which intends to communicate how a convolutional neural network ‘sees’. 29 But since platforms are much more than contained ordering mechanisms, the problem of how to make their complexity readable, how to narrate what can be gleaned from data (see Dourish, 2016), remains unsolved. However, researchers in the humanities and social sciences have long been interested in how to make sense of quantitative information. Work on ‘narrating numbers’ (Espeland, 2015), ‘narrating networks’ (Bounegru et al., 2017), or the substantial research on information visualisation (e.g. Drucker, 2014) can serve as models. But as Sloane & Moss (2019) argue in their critique of current approaches to AI, there is a broader ‘social science deficit’ and the one-sided focus on quantitative information is part of the problem. The marginalisation of qualitative methods such as ethnographic work that tries to elucidate both the context within which platforms make decisions and the meaning actors ascribe to practices and their effects, limits knowledge production. Journalists also have unique expertise when it comes to forms of knowledge generation and presentation. A recent example is the work by Karen Hao and Jonathan Stray 30 on the controversial KOMPASS project, 31 which questions the very possibility of fair judgements by allowing users to ‘play’ with the parameters of a simplified model. Likewise, NGOs have long worked on compound forms of narration that combine different data sources and methods for purposes of accountability. Greenpeace’s Guide to Greener Electronics, which includes a grade for companies’ willingness to share information, or the Ranking Digital Rights 32 project are good examples for the translation of research into concrete political devices. Accountability, understood as an inherent element of democratic control, cannot be reduced to a forensic process that transposes ‘facts’ from obscurity into the light. It needs to be considered as an ongoing social achievement that requires different forms of sense-making, asking for contributions from different directions and epistemological sensitivities. Access to machine-readable data, our focus in the last section, has limitations, but also allows different actors to develop their own observation capacities, adapting their analytical methods to the questions they want to ask. We are aware that increased understanding of platform life would prompt reactions and adaptations by different stakeholders gathering around platforms, including actors seeking to ‘game’ the system and even platform owners themselves. Making the constant negotiations between these actors more visible may have the advantage, however, that the process of establishing boundaries of acceptable behaviour could be engaged more explicitly. As Ziewitz (2019, p. 713) argues for the field of search engine optimisation (SEO), ‘the moral status of reactive practices is not given, but needs to be accomplished in practice’. Distributing this ‘ethical work’ over a wider array of actors could thus be a step toward some modest form of ‘cooperative responsibility’ (Helberger et al., 2018), even if fundamental power asymmetries remain. Observability thus raises the complicated question of how data and analytical capacities should be made available, to whom, and for what purpose. This clearly goes beyond data access. As Kemper & Kolkman (2019) note, ‘no algorithmic accountability without a critical audience’, and the capacity for critique requires more than a critical attitude. For this reason, frameworks for data access should ‘go hand-in-hand with the broader cultivation of a robust and democratic civil society, which is adequately funded and guaranteed of its independence’ (Ausloos et al., 2020, p. 86). And Flyverbom (2015, p. 115) reminds us that transparency, understood as a transformative process, cannot succeed ‘without careful attention to the formats, processes of socialization, and other affordances of the technologies and environments in which they play out’. Monitoring platforms on a continuous basis may thus call for considerable resources if done well. Governmental institutions, possibly on a European level, could play a central role in managing data access, in making long-term funding available for research, and in coordinating the exchange between existing initiatives. But given the complexity of the task, regulators will also have to build ‘in-house’ expertise and observational capacity, backed by strong institutional support. The capacity to make sense of large and complex socio-technical systems indeed relies on a number of material conditions, including access to data, technical expertise, computing power, and not least the capacity to connect data-analytical practices to social concerns. Such a capacity is typically produced as a collective effort, through public discourse. The quality of observability depends on such discourses to explore what kind of knowledge forms allow concerned actors to make actually meaningful interpretations. 4. Conclusion: toward platform observability This article developed the concept of observability to problematise the assumptions and expectations that drive our demands for transparency of platform life. Observability is not meant to be a radical departure from the call for transparency. Rather, it draws practical conclusions from the discrepancy we noted between the complexity of the platform machinery and the traditional idea of shedding light on and seeing as a way of establishing external oversight. In a nutshell, we are suggesting observability as a pragmatic, knowledge-focused approach to accountability. Observability stresses technical and social complexities, including the distributed nature of platform behaviour. Moreover, it regards continuous and collaborative observation within a normative framework as a necessary condition for regulating the explosive growth of platform power. We see three main directions where further steps are needed to move closer to the practical realisation of these principles. Regulating for observability means working toward structured information interfaces between platforms and society. 33 To account for quickly changing circumstances, these interfaces need to enable continuous observation. To allow for a broader set of questions to be asked, a broad range of data has to be covered. And to bring a wider variety of epistemological sensitivities into the fold, they need to be sufficiently flexible. What constitutes suitable and sufficient access will have to be decided on a per-platform basis, including the question of who should be able to have access in the first place. But the examples we briefly discussed in section 3.2—and the many others we left out—show that there is already much to build on. The main goal, here, is to develop existing approaches further and to make them more stable, transparent, and predictable. Twitter’s new API 34, which now explicitly singles out academic research use cases, is a good example for a step in the right direction, but these efforts are still voluntary and can be revoked at any time. Without binding legal frameworks, platforms can not only terminate such initiatives at will, they also control relevant modalities such as thematic scope and depth of access. Realigning the structural information asymmetries between platforms and society thus requires curtailing the de facto ownership over data that platforms collect about their users. Observability as part of regulation requires engaging with the specific properties of algorithmic systems and the co-produced nature of platform behaviour. The complex interactions between technical design, terms of service, and sometimes vast numbers of both users and ‘items’ mean that the concept of a singular algorithm steering the ordering processes at work in large-scale platforms is practically and conceptually insufficient. If techniques like machine learning are here to stay, regulatory approaches will have to adapt to conditions where the object of regulation is spread out, volatile, and elusive. The pressing questions are not restricted to how and what to regulate, but also encompass the issue of what platforms are doing in the first place. While normative concepts such as algorithmic fairness or diversity are laudable goals, their focus seems rather narrow considering the fundamental change of markets and the public sphere that platforms provoke. We therefore suggest the broader concept of public interest as a normative benchmark for assessing platform behaviour, a concept obviously in need of specification. But whatever set of norms or values are chosen as guiding principles, the question remains how to ‘apply’ them, that is, how to assess platform behaviour against public interest norms. Observation as a companion to regulation stresses the fact that we need to invest in our analytical capacities to undergird the regulatory response to the challenges platforms pose. Likewise, the existing approaches to studying platforms should be supplemented with specific rights to information. Together, these elements would constitute important steps towards a shared governance model (see Helberger et al., 2018), where power is distributed more equally between platforms and their constituencies. Institutionalising processes of collective learning refers to the need to develop and maintain the skills that are required to observe platforms. A common characteristic of the data collecting projects mentioned above is their ephemeral, experimental, and somewhat amateurish nature. While this may sound harsh, it should be obvious that holding platforms to account requires ‘institution-building’, that is, the painstaking assembly of skills and competence in a form that transposes local experiments into more robust practices able to guarantee continuity and accumulation. While academic research fields have their own ways of assembling and preserving knowledge, the task of observing large-scale platforms implies highly specialised technical and logistical feats that few organisations are able to tackle. Material resources are only one part of the equation and the means to combat discontinuity and fragmentation are at least equally important. One form of institutional incorporation of observability would therefore be something akin to ‘centres of expertise’ tasked with building the capacity to produce relevant knowledge about platforms. Such centres could act as an, ‘important bridge builder between those holding the data and those wishing to get access to that data’ (Ausloos et al., 2020, p. 83). Pushing further, a European Platform Observatory, 35 driven by a public interest mandate, equipped with adequate funding, and backed by strong regulatory support, could be a way forward to platform accountability. Holding platforms to account is a complex task that faces many challenges. However, given their rising power, it is quickly becoming a necessity. The concept of observability spells out these challenges and suggests steps to tackle them, taking a pragmatic, knowledge-based approach. The goal, ultimately, is to establish observability as a ‘counter power’ to platforms’ outsized hold on contemporary societies. Acknowledgements This work was, in part, inspired by discussions we had as members of the European Commission’s Observatory on the Online Platform Economy. We would also like to thank Joris van Hoboken, Paddy Leerssen, and Thomas Poell for helpful comments and feedback. References Agre, P. E. (1994). Surveillance and Capture: Two Models of Privacy. The Information Society, 10(2), 101–127. https://doi.org/10.1080/01972243.1994.9960162 Albu, O. B., & Flyverbom, M. (2019). Organizational Transparency: Conceptualizations, Conditions, and Consequences. Business & Society, 58(2), 268–297. https://doi.org/10.1177/0007650316659851 Anand, N., & Peterson, R. A. (2000). When Market Information Constitutes Fields: Sensemaking of Markets in the Commercial Music Industry. Organization Science, 11(3), 270–284. https://doi.org/10.1287/orsc.11.3.270.12502 Ananny, M., & Crawford, K. (2018). Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media & Society, 20(3), 973–989. https://doi.org/10.1177/1461444816676645 August, V., & Osrecki, F. (2019). Transparency Imperatives: Results and Frontiers of Social Science Research. In V. August & F. Osrecki (Eds.), Der Transparenz-Imperativ: Normen – Praktiken – Strukturen (pp. 1–34). Springer. https://doi.org/10.1007/978-3-658-22294-9 Bernstein, E. S. (2012). The Transparency Paradox: A Role for Privacy in Organizational Learning and Operational Control. Administrative Science Quarterly, 57(2), 181–216. https://doi.org/10.1177/0001839212453028 Bogost, I. (2015, January 15). The Cathedral of Computation. The Atlantic. https://www.theatlantic.com/technology/archive/2015/01/the-cathedral-of-computation/384300/ Bovens, M. (2007). Analysing and Assessing Accountability: A Conceptual Framework. European Law Journal, 13(4), 447–468. https://doi.org/10.1111/j.1468-0386.2007.00378.x Brandeis, L. D. (1913, December 20). What publicity can do. Harper’s Weekly. Bruns, A. (2019). After the ‘APIcalypse’: Social media platforms and their fight against critical scholarly research. Information, Communication & Society, 22(11), 1544–1566. https://doi.org/10.1080/1369118X.2019.1637447 Bruns, A., & Burgess, J. (2013). Crisis communication in natural disasters: The Queensland floods and Christchurch earthquakes. In K. Weller, A. Bruns, J. Burgess, M. Mahrt, & C. Puschmann (Eds.), Twitter and Society (pp. 373–384). Peter Lang. Burrell, J. (2016). How the machine “thinks”: Understanding opacity in machine learning algorithms. Big Data & Society, 3(1), 1–12. https://doi.org/10.1177/2053951715622512 Cohen, J. E. (2019). Between Truth and Power: The Legal Constructions of Informational Capitalism. Oxford University Press. https://doi.org/10.1093/oso/9780190246693.001.0001 Cornelius, K. B. (2019). Zombie contracts, dark patterns of design, and ‘documentisation’. Internet Policy Review, 8(2). https://doi.org/10.14763/2019.2.1412 Covington, P., Adams, J., & Sargin, E. (2016). Deep Neural Networks for YouTube Recommendations. Proceedings of the 10th ACM Conference on Recommender Systems, 191–198. https://doi.org/10.1145/2959100.2959190 Domingos, P. (2012). A few useful things to know about machine learning. Communications of the ACM, 55(10), 78–87. https://doi.org/10.1145/2347736.2347755 Doran, D., Schulz, S., & Besold, T. R. (2017). What Does Explainable AI Really Mean? A New Conceptualization of Perspectives. ArXiv. http://arxiv.org/abs/1710.00794 Douglass, B. (1980). The Common Good and the Public Interest. Political Theory, 8(1), 103–117. https://doi.org/10.1177/009059178000800108 Dourish, P. (2016). Algorithms and their others: Algorithmic culture in context. Big Data & Society, 3(2). https://doi.org/10.1177/2053951716665128 Espeland, W. (2015). Narrating Numbers. In R. Rottenburg, S. E. Merry, S.-J. Park, & J. Mugler (Eds.), The World of Indicators: The Making of Governmental Knowledge through Quantification (pp. 56–75). Cambridge University Press. https://doi.org/10.1017/CBO9781316091265.003 Etzioni, A. (2010). Is Transparency the Best Disinfectant? Journal of Political Philosophy, 18(4), 389–404. https://doi.org/10.1111/j.1467-9760.2010.00366.x Ezrahi, Y. (1992). Technology and the civil epistemology of democracy. Inquiry, 35(3–4), 363–376. https://doi.org/10.1080/00201749208602299 Flyverbom, M. (2016). Transparency: Mediation and the Management of Visibilities. International Journal of Communication, 10, 110–122. https://ijoc.org/index.php/ijoc/article/view/4490 Fourcade, M., & Gordon, J. (2020). Learning Like a State: Statecraft in the Digital Age. Journal of Law and Political Economy, 1(1), 78–108. https://escholarship.org/uc/item/3k16c24g Gillespie, T. (2018). Custodians of the Internet. Yale University Press. Hegelich, S. (2020). Facebook needs to share more with researchers. Nature, 579, 473–473. https://doi.org/10.1038/d41586-020-00828-5 Helberger, N. (2020). The Political Power of Platforms: How Current Attempts to Regulate Misinformation Amplify Opinion Power. Digital Journalism, 8(3). https://doi.org/10.1080/21670811.2020.1773888 Helberger, N., Pierson, J., & Poell, T. (2018). Governing online platforms: From contested to coop...
This article belongs to Concepts of the digital society, a special section of Internet Policy Review guest-edited by Christian Katzenbach and Thomas Christian Bächle. Introduction Cybersecurity 1 covers the broad range of technical, organisational and governance issues that must be considered to protect networked information systems against accidental and deliberate threats. It goes well beyond the details of encryption, firewalls, anti-virus software, and similar technical security tools. This breadth is captured in the widely used International Telecommunication Union (ITU) definition (ITU-T, 2008, p. 2): Cybersecurity is the collection of tools, policies, security concepts, security safeguards, guidelines, risk management approaches, actions, training, best practices, assurance and technologies that can be used to protect the cyber environment and organization and user’s assets. Organization and user’s assets include connected computing devices, personnel, infrastructure, applications, services, telecommunications systems, and the totality of transmitted and/or stored information in the cyber environment. Cybersecurity strives to ensure the attainment and maintenance of the security properties of the organization and user’s assets against relevant security risks in the cyber environment The importance of cybersecurity has increased as so many government, business, and day-to-day activities around the world have moved online. But especially in emerging economies, “[m]any organizations digitizing their activities lack organizational, technological and human resources, and other fundamental ingredients needed to secure their system, which is the key for the long-term success” (Kshetri, 2016, p. 3). The more technically-focused information security is still in widespread use in computer science. But as these issues have become of much greater societal concern as “software is eating the world” (Andreessen, 2011), cybersecurity has become more frequently used, not only in the rhetorics of democratic governments as in the 2000s, but also in general academic literature (shown in Figure 1): Figure 1: Academic articles with cybersecurity/cyber-security/cyber security versus information security, data security and computer security in title, keywords or abstract of Web of Science indexed publications over time. Small numbers of records exist for both information security and computer security in the database since 1969. Data from Web of Science. Barely used in academic literature before 1990 (except in relation to the Cray CYBER 205 supercomputer from the late 1970s), cyber became ubiquitous as a prefix, adjective and even noun by the mid-1990s, with Google Scholar returning results across a broad range of disciplines with titles such as ‘Love, sex, & power on the cyber frontier’ (1995), ‘Surfing in Seattle: What cyber-patrons want’ (1995), ‘The cyber-road not taken’ (1994) and even the ‘Cyber Dada Manifesto” (1991). It evolved from Wiener’s cybernetics, a “field of control and communication theory, whether in machine or in the animal” (1948)—derived from the Greek word for ‘steersman’—with an important intermediate point being the popular usage of cyborg, a contraction of cybernetic organism, alongside the Czech-derived robot (Clarke, 2005, section 2.4). The notion of a ‘governor’ of a machine goes back to the mid-19th century, with J. C. Maxwell (discoverer of the electron) noting in 1868 it is “a part of a machine by means of which the velocity of the machine is kept nearly uniform, notwithstanding variations in the driving-power or the resistance” (Maxwell, 1868, p. 270)—what Wiener called homeostasis. The use of cyberspace to refer to the electronic communications environment was coined in William Gibson’s 1982 short story Burning Chrome (“widespread, interconnected digital technology”) and popularised by his 1984 science fiction novel Neuromancer (“a graphic representation of data abstracted from the banks of every computer in the human system […] lines of light ranged in the nonspace of mind, clusters and constellations of data […] a consensual hallucination experienced by millions”). Cyberspace’s arrival in legal and policy discussions was spearheaded by John Perry Barlow’s Declaration of the Independence of Cyberspace (1996). But by 2000, Gibson declared cyberspace was “evocative and essentially meaningless ... suggestive ... but with no real meaning” (Neale, 2000). Despite its ubiquity in present-day national security and defence-related discussions, Wagner and Vieth found: “Cyber and cyberspace, however, are not synonymous words and have developed different meanings [...] Cyber is increasingly becoming a metaphor for threat scenarios and the necessary militarisation” (2016). Matwyshyn suggested the term is “the consequence of a cultural divide between the two [US] coasts: ‘cybersecurity’ is the Washington, D.C. legal rebranding for what Silicon Valley veterans have historically usually called ‘infosec’ or simply ‘security’” (2017, p. 1158). Cybersecurity issues have, to many whose interests are served by the interpretation, become national security issues (Clarke, 2016; Kemmerer, 2003; Nissenbaum, 2005). A review by Craigen et al. (2014) found cybersecurity used in a range of literature and fields from 2003 onwards, including software engineering, international relations, crisis management and public safety. Social scientists interacting with policymakers, and academics generally applying for research and translation funding from government sources and interacting with the defence and signals intelligence/information security agencies that are the cybersecurity centres of expertise in many larger governments, have further popularised the term, 2 which appears in similar form in many languages, as shown in Appendix 1. Looking beyond academia to literature more widely, Figure 2 shows computer security was most prevalent in the Google Books corpus from 1974, overtaken by information security in 1997, and cybersecurity in 2015 (with cyber security increasingly popular since 1996, but cyber-security negligible the entire period). Computer (Ware, 1970), system, and data (Denning, 1982) security were all frequently used as closely-related terms in the 1970s (Saltzer & Schroeder, 1975). 3 Figure 2: Google n-gram analysis (Lin et al., 2012) of the usage of variants of information security over time. Cybersecurity encompasses cybersecurity, cyber security and cyber-security. Retrieved using ngramr (Carmody, 2020). This trend is unfortunate, since “using the term ‘cybersecurity’ seems to imply that information security issues are limited to code connected to the Internet [but] physical security of machines and human manipulability through social engineering are always key aspects of information security in both the private and public sector” (Matwyshyn, 2017, p. 1156). Cybersecurity in early context In computer science, attacks on the security of information systems are usually concerned with: Breaching the confidentiality of systems, with data exposed to unauthorised actors; Undermining the integrity of systems, and disruption of the accuracy, consistency or trustworthiness of information being processed; Affecting the availability of systems, and rendering them offline, unusable or non-functional. Together, confidentiality, integrity and availability are called the CIA triad, and have been the basis of information security since the late 1970s (Neumann et al., 1977, pp. 11–14). Echoing this history decades later, the Council of Europe’s 2001 Budapest Convention on Cybercrime set out in its first substantive section “Offences against the confidentiality, integrity and availability of computer data and systems”. Cybersecurity across disciplines The study and practice of cybersecurity spans a range of disciplines and fields. In this article, we consider three of the main angles important to cybersecurity practice: technical aspects; human factors; and legal dimensions. This is necessarily an incomplete list—notably, the topic is also the subject of study by those who are interested in, for example, how it reconfigures organisational structures (information systems), or relationships between actors such as states (international relations), and significant non-state actors such as organised crime gangs (criminology). Technical aspects Many technical domains are of direct relevance to cybersecurity, but the field designed to synthesise technical knowledge in practical contexts has become known as security engineering: “building systems to remain dependable in the face of malice, error, or mischance” (Anderson, 2008, p. 3). It concerns the confluence of four aspects—policy (the security aim), mechanisms (technologies to implement the policy), assurance (the reliability of each mechanism) and incentives (of both attackers and defenders). Security engineers may be intellectually grounded in a specialised technical domain, but they require a range of bridging and boundary skills between other disciplines of research and practice. A daunting (and worsening) challenge for security engineers is posed by the complexities of the sociotechnical environments in which they operate. Technological systems have always evolved and displayed interdependencies, but today infrastructures and individual devices are networked and co-dependent in ways which challenge any ability to unilaterally “engineer” a situation. Systems are increasingly servitised, (e.g., through external APIs) with information flows not under the control of the system engineer, and code subject to constant ‘agile’ evolution and change which may undermine desired system properties (Kostova et al., 2020). Human factors and social sciences The field of human factors in cybersecurity grew from the observation that much of the time “hackers pay more attention to the human link in the security chain than security designers” (Adams & Sasse, 1999, p. 41), leaving many sensitive systems wide open to penetration by “social engineering” (Mitnick & Simon, 2002). It is now very problematic to draw cybersecurity’s conceptual boundaries around an organisation’s IT department, software vendors and employer-managed hardware, as in practice networked technologies have permeated and reconfigured social interactions in all aspects of life. Users often adapt technologies in unexpected ways (Silverstone & Hirsch, 1992) and create their own new networked spaces (Cohen, 2012; Zittrain, 2006), reliant on often-incomprehensible security tools (Whitten & Tygar, 1999) that merely obstruct individuals in carrying out their intended tasks (Sasse et al., 2001). Networked spaces to be secured—the office, the university, the city, the electoral system—cannot be boxed-off and separated from technology in society more broadly. Communities often run their networked services, such as a website, messaging group, or social media pages, without dedicated cybersecurity support. Even in companies, or governments, individuals or groups with cybersecurity functions differ widely in location, autonomy, capabilities, and authority. The complexity of securing such a global assemblage, made up of billions of users as well as hundreds of millions of connected devices, has encouraged a wider cross-disciplinary focus on improving the security of these planetary-scale systems, with social sciences as an important component (Chang, 2012). Research focussed on the interaction between cybersecurity and society has also expanded the relevant set of risks and actors involved. While the term cybersecurity is often used interchangeably with information security (and thus in terms of the CIA triad), this only represents a subset of cybersecurity risks. Insofar as all security concerns the protection of certain assets from threats posed by attackers exploiting vulnerabilities, the assets at stake in a digital context need not just be information, but could, for example, be people (through cyberbullying, manipulation or intimate partner abuse) or critical infrastructures (von Solms & van Niekerk, 2013). Moreover, traditional threat models in both information and cybersecurity can be limited. For example, domestic abusers are rarely considered as a threat actor (Levy & Schneier, 2020) and systems are rarely designed to protect their intended users from the authenticated but adversarial users typical in intimate partner abuse (Freed et al., 2018). The domain of cyber-physical security further captures the way in which cybersecurity threats interact with physically located sensors and actuators. A broader flavour of definition than has been previously typical is used in the recent EU Cybersecurity Act (Regulation 2019/881), which in Article 2(1) defines cybersecurity as “the activities necessary to protect network and information systems, the users of such systems, and other persons affected by cyber threats” [emphasis added]. The difficult interaction between information systems, societies and environments is rapidly gaining traction in the research literature. Research at the intersection of human–computer interaction and cybersecurity has also pointed to challenges of usability and acceptability in deploying approaches developed in fields such as security engineering. Consider the encryption of information flowing across the internet using Transport Layer Security (TLS), a protocol which is able to cryptographically authenticate the endpoints and protect the confidentiality and integrity of transmitted data. TLS raises usability challenges in relation to developers’ and administrators’ understanding of how it works and thus how to correctly implement it (Krombholz et al., 2017, 2019) as well as challenges with communicating its properties—and what to do in its absence—to end users in their web browsers (Felt et al., 2015; Reeder et al., 2018). Focusing on the user experience of the web browser, Camp (2013) suggests principles of translucent security: high security defaults, single-click override, context-specific settings, personalised settings, and use-based settings. Related challenges faced by both users and developers or other specialists are found widely across the cybersecurity field, including passwords (e.g., Naiakshina et al., 2019) and encrypted email (Whitten & Tygar, 1999). The field of usable security seeks a fit between the security task and the humans expected to interact with it (Sasse et al., 2001). Without an understanding of issues such as these, the techniques used can bring at best a false sense of security, and at worst, entirely new threat vectors. Legal dimensions While few laws explicitly state they are governing cybersecurity, cybersecurity–related provisions are found in an extremely wide array of instruments. Law might incentivise or require certain cybersecurity practices or standards; apply civil or criminal sanctions, or apportion liability, for persons experiencing or taking action which leads to cybersecurity breaches; mandate practices (such as information sharing or interoperability) that themselves have cybersecurity implications; or create public advisory or enforcement bodies with cybersecurity responsibilities. Data protection and privacy laws generally contain varied provisions with cybersecurity implications. They are, at the time of writing, present in 142 countries around the world (Greenleaf & Cottier, 2020) as well as promoted by the Council of Europe’s Convention 108+ and model laws from several international organisations, such as the Commonwealth (Brown et al., 2020). They often, although not always, span both the public and private sectors, with common stipulations including the creation of an independent supervisory authority; overarching obligations to secure ‘personal’ data or information, often defined by reference to its potential identifiability; data breach notification requirements; obligations to design in enforcement of data protection principles and appoint a data protection officer; and rights that can be triggered by individuals to access, manage and if they wish, erase identifiable data that relates to them. Other specific laws also contain cybersecurity breach notification (to users and/or regulators) and incident requirements scoped beyond personal data, such as the European eIDAS Regulation (Regulation 910/2014, concerning identity and trust providers) and Network and Information Security Directive (Directive 2016/1148, concerning essential infrastructure, including national infrastructure such as electricity and water as well as ‘relevant digital service providers’, meaning search engines, online marketplaces and cloud computing). While lacking an omnibus federal data protection law, all 50 US states have some form of data breach law, although their precise requirements vary (Kosseff, 2020, Appendix B). In the EU, the law that would seem the most likely candidate for a horizontal regime is the 2019 Cybersecurity Act (Regulation 2019/881).It however provides little of real substantive interest, mainly increasing the coordination and advisory mandates of ENISA, the EU’s cybersecurity agency, and laying the foundation for a state-supported but voluntary certification scheme. A grab-bag of highly specific cybersecurity laws also exists, such as the California Internet of Things Cybersecurity Law, aimed mostly at forbidding devices from using generic passwords (Cal. Civ. Code § 1798.91.04). These reactive, ad-hoc instruments are often not technologically neutral: they may have clarity and legal certainty in the current situation, but may not be sustainable as technologies change, for example, away from passwords (Koops, 2006). On the other hand, generic laws have also, over time, morphed into cybersecurity laws. The Federal Trade Commission in the US penalises companies for exceptionally poor data security practices under the prohibition of “unfair or deceptive practices” in the FTC Act (15 U.S.C. § 45). There are, however, limits to the ability of generic laws to morph into cybersecurity laws. Computer misuse laws emerged in legal regimes in part due to the limitations of existing frameworks in capturing digital crime. Before the mid-1980s, the main avenue to prosecuting computer misuse in the US was theft (Kerr, 2003), a rationale which proved strained and unpredictable. The UK saw unsuccessful attempts to repurpose the law of forgery against unauthorised password use (R v Gold  AC 1063), leading to the passing of the Computer Misuse Act 1990. The US has struggled with the concept of ‘unauthorised’ access in its law. Offences in the Computer Fraud and Abuse Act (CFAA) of 1984 typically occur when individuals enter systems without authorisation, or where they exceed authorised access, mimicking laws of trespass (Kerr, 2016). But the notion of authorisation in digital systems quickly becomes tricky. If a website is designed such that sensitive information is discoverable by typing in a long URL (a problematic “security through obscurity” approach), without any authentication mechanism, is there implicit authorisation? Is an address bar more like a password box—guessing someone else’s being telling about your motive to access unauthorised material; or a telephone keypad or map—and the user is simply exploring? The CFAA has also created tensions based on its interaction with a site’s terms of service (ToS). This tension centres on whether authorisation is revoked based on statements in these long, legalistic documents that only few read. For example, such documents often preclude web scraping in broad, vague language (Fiesler et al., 2020), and despite over sixty legal opinions over the last two decades, the legal status of scraping remains “characterized as something just shy of unknowable, or a matter entirely left to the whims of courts” (Sellars, 2018, p. 377). This becomes highly problematic for firms, researchers or journalists, as computer misuse law may effectively turn potential civil liability for breach of contract into criminal liability under the CFAA. As a consequence, scholars such as Orin Kerr have argued that only the bypassing of authentication requirements, such as stealing credentials, or spoofing a log-in cookie, should be seen as creating a lack of authorisation under CFAA (Kerr, 2016). This contrasts with messy existing case law, which includes prosecution on the basis that an IP address was changed (as it often does by design) to avoid a simple numeric IP block. Contingent and subjective social aspects of cybersecurity law will remain, both in computer misuse and in other areas, even if this argument was accepted. Legal instruments around cybercrime and cybersecurity more generally continue to develop—the Council of Europe’s Budapest Convention on Cybercrime was concluded in 2001, seeking to harmonise cybercrime legislation and facilitate international cooperation, and drawing on experiences and challenges of earlier cybersecurity and cybercrime law. It has been ratified/acceded to by 65 countries including the US, which has only ever ratified three Council of Europe treaties. However, the further development of legal certainty in areas of cybersecurity will require yet clearer shared norms of how computing systems, and in particular, the internet, should be used. Cybersecurity’s broader impact Here, we select and outline just two broader impacts of cybersecurity—its link to security-thinking in other domains of computing and society, and its effect on institutional structures. (Cyber)securitisation While computer security narrowly focussed on the CIA triad, the cybersecurity concept expanded towards both national security and the use of computers for societally harmful activities (e.g., hatred and incitement to violence; terrorism; child sexual abuse) and attacks on critical infrastructures, including the internet itself (Nissenbaum, 2005). The privileged role of technical experts and discourse inside computer security has given technical blessing to this trend of securitisation (Hansen & Nissenbaum, 2009, p. 1167). Security is not new to technification, as ‘Cold War rationality’ showed (Erickson et al., 2013). Yet not only have technical approaches arguably been able to take a more privileged position in cybersecurity than any other security sector (Hansen & Nissenbaum, 2009, p. 1168), their success in raising salience through securitisation has resonated widely across computing issues. For example, privacy engineering has a dominant strand focussing on quantitative approaches to confidentiality, such as minimising theoretical information leakage (Gürses, 2014); while algorithmic fairness and anti-discrimination engineering has also emerged as a similar (and controversial) industry-favoured approach to issues of injustice (Friedler et al., 2019; see Gangadharan & Niklas, 2019). Gürses connects the engineering of security, privacy, dependability and usability—an ideal she claims “misleadingly suggests we can engineer social and legal concepts” (Gürses, 2014, p. 23). These echoes may have their origins in the very human dimensions of these fast-changing areas, as organisations seek to apply or redeploy employees with security skill sets shaped by strong professional pressures to these recently salient problems (DiMaggio & Powell, 1983), as well as the hype-laden discourse of cybersecurity identified as fuelling a range of problems in the field (Lee & Rid, 2014). While these areas may not yet be able to be considered securitised, insofar as neither privacy nor discrimination is commonly politically positioned as an existential threat to an incumbent political community (Buzan et al., 1998; Cavelty, 2020; see Hansen & Nissenbaum, 2009), neither can they be said to be unaffected by the way cybersecurity and national security, and the forms of computing knowledge and practice considered legitimate in those domains, have co-developed over recent decades. Institutions Requirements of cybersecurity knowledge and practice have led states to create new institutions to meet perceived needs for expertise. The location of this capacity differs. In some countries, there may be significant public sector capacity and in-house experts. Universities may have relevant training pipelines and world-leading research groups. In others, cybersecurity might not be a generic national specialism. In these cases, cybersecurity expertise might lie in sector-specific organisations, such as telecommunications or financial services companies, which may or may not be in public hands. Some governments have set up high-level organisations to co-ordinate cybersecurity capacity-building and assurance in public functions, such as the Australian Cyber Security Centre, the Canadian Centre for Cyber Security, the National Cyber Security Centre (UK and Ghana—soon to become an Authority) and the Cyber Security Agency (Singapore). A new Cybersecurity Competence Centre for the EU is set to be based in Bucharest. Relatedly, and sometimes independently or separately, countries often have cybersecurity strategy groups sitting under the executive (Brown et al., 2020). Cybersecurity agencies can find themselves providing more general expertise than simply security. During the COVID-19 pandemic, for example, the first version of the UK’s National Health Service (NHS) contact tracing app for use in England had considerable broad technical input from the government’s signals intelligence agency GCHQ and its subsidiary body the National Cyber Security Centre, which was considered a data controller under UK data protection law (Levy, 2020). Relatedly, these agencies have also been called upon to give advice in various regimes to political parties who are not currently in power—a relationship that would be challenging in countries where peaceful transitions of power cannot be easily taken for granted, particularly given many of these institutions’ close links with national security agencies which may have politically-motivated intelligence operations (Brown et al., 2020). National Computer Security Incident Response Teams (CSIRTs) are a relatively recent form of institution, which act as a coordinator and a point of contact for domestic and international stakeholders during an incident. Some of these have been established from scratch, while others have been elevated from existing areas of cybersecurity capacity within their countries (Maurer et al., 2015). These expert communities, trusted clearing houses of security information, are found in many countries, sectors and networks, with 109 national CSIRTs worldwide as of March 2019 (International Telecommunication Union, 2019). CSIRTs can play important international roles, although as they are infrequently enshrined in or required by law, they often occupy a somewhat unusual quasi-diplomatic status (Tanczer et al., 2018). Under the EU’s Network and Information Security Directive however, all 27 member states must designate a national CSIRT, with ENISA playing a coordinating role under the NIS Directive. Some researchers have expressed a more sceptical view of CSIRTs, with Roger Clarke telling the authors: “Regrettably, in contemporary Australia, at least, the concept has been co-opted and subverted into a spook sub-agency seeking ever more power to intrude into the architecture and infrastructure of telecommunications companies, and whatever other ‘critical infrastructure’ organisations take their fancy. Would you like a real-time feed of the number-plates going under toll-road gantries? Easily done!” (personal communication, September 2020). Conclusion Understanding cybersecurity is a moving target, just like understanding computing and society. Exactly what is being threatened, how, and by whom are all in flux. While many may still look on with despair at the insecurities in modern systems, few computing concepts excite politicians more. It is hardly surprising to see the language of security permeate other computing policy concepts as a frame. Politicians talk of keeping the internet safe; dealing with privacy breaches, and defending democracies against information warfare. This makes cybersecurity an important concept for scholars to study and understand, and its legal and institutional adventures instructive for the development of neighbouring domains (although perhaps not always as the best template to follow). Its tools and methodological approach are also a useful training ground for interdisciplinary scholars to gain the skills required to connect and work across social, legal and technical domains. In a 2014 review, three Canadian Communications Security Establishment science and learning advisers (Craigen et al., 2014) concluded cybersecurity is “used broadly and its definitions are highly variable, context-bound, often subjective, and, at times, uninformative”. In 2017, Matwyshyn noted “‘cyberized’ information security legal discourse makes the incommensurability problems of security worse. It exacerbates communication difficulty and social distance between the language of technical information security experts on the one hand, and legislators, policymakers and legal practitioners on the other” (Matwyshyn, 2017, p. 1150). It is not clear the situation has since improved in this regard. Cybersecurity has become a catch-all term, attached to the prevention of a very wide range of societal harms seen to be related to computing and communications tools now omnipresent in advanced economies, and increasingly prevalent in emerging economies. There are concerns this has led to a militarisation (Wagner & Vieth, 2016) or securitisation of the concept and hence measures taken by states as a result. (The UK Ministry of Defence trumpeted the launch of its “first cyber regiment” in 2020.) And the large-scale monitoring capabilities of many cybersecurity tools have led to serious concerns about their impact on human rights (Korff, 2019). Meanwhile, many computer and social scientists publicly mock 4 the notion of cyber and cyberspace as a separate domain of human action (Graham, 2013). Rid (2016, chapter 9) noted even Wiener “would have disdained the idea and the jargon. The entire notion of a separate space, of cordoning off the virtual from the real, is getting a basic tenet of cybernetics wrong: the idea that information is part of reality, that input affects output and output affects input, that the line between system and environment is arbitrary”. Matwyshyn concluded “[s]ecurity experts fear that in lieu of rigorously addressing the formidable security challenges our nation faces, our legal and policy discussions have instead devolved into a self-referential, technically inaccurate, and destructively amorphous “cyber-speak,” a legalistic mutant called “cybersecurity”” (p. 1154). We have described now that notions relating to the protection of information systems—and all the societal functions those systems now support—are increasingly significant in both academic literature and the broader public and policy discourse. The development of the “Internet of Things” will add billions of new devices over time to the internet, many with the potential to cause physical harm, which will further strengthen the need for security engineering for this overall system (Anderson, 2018). There appears little likelihood of any clear distinctions developing at this late stage between information security and cybersecurity in practice. It may be that the former simply falls out of common usage in time, as computer security slowly has since 2010—although those with security capabilities (a.k.a. state hacking) still stick resolutely with cyber. Anderson suggests the continued integration of software into safety-critical systems will require a much greater emphasis on safety engineering, and protection of the security properties of systems like medical devices (even body implants) and automotive vehicles for decades—in turn further strengthening political interest in the subject (2021, p. 2). Martyn Thomas, a well-known expert in safety-critical system engineering, told us (personal communication, September 2020): Rather than attackers increasingly finding new ways to attack systems, the greater threat is that developers increasingly release software that contains well-known vulnerabilities – either by incorporating COTS (commercial off-the-shelf) components and libraries with known errors, or because they use development practices that are well known to be unsafe (weakly typed languages, failure to check and sanitise input data, etc.). So, the volume of insecure software grows, and the pollution of cyberspace seems unstoppable. Powerful states (particularly the US) have since at least the 1970s used their influence over the design and production of computing systems to introduce deliberate weaknesses in security-critical elements such as encryption protocols and libraries (Diffie & Landau, 2010), and even hardware (Snowden, 2019). The US CIA and NSA Special Collection Service “routinely intercepts equipment such as routers being exported from the USA, adds surveillance implants, repackages them with factory seals and sends them onward to customers” (Anderson, 2020, p. 40). It would be surprising if other states did not carry out similar activities. In the long run, as with most technologies, we will surely take the cyber element of everyday life for granted, and simply focus on the safety and security (including reliability) of devices and systems that will become ever more critical to our health, economies, and societies. Acknowledgements The authors thank Roger Clarke, Alan Cox, Graham Greenleaf, Douwe Korff, Chris Marsden, Martyn Thomas and Ben Wagner for their helpful feedback, and all the native speakers who shared their linguistic knowledge. References Adams, A., & Sasse, M. A. (1999). Users are not the enemy. Communications of the ACM, 42(12), 40–46. https://doi.org/10.1145/322796.322806 Anderson, R. (2008). Security Engineering: A Guide to Building Dependable Distributed Systems (2nd ed.). Wiley. Anderson, R. (2018). Making Security Sustainable. Communications of the ACM, 61(3), 24–26. https://doi.org/10.1145/3180485 Anderson, R. (2020). Security Engineering: A Guide to Building Dependable Distributed Systems (3rd ed.). Wiley. Andreessen, M. (2011, August 20). Why Software Is Eating The World. The Wall Street Journal. https://www.wsj.com/articles/SB10001424053111903480904576512250915629460 Baran, P. (1960). Reliable Digital Communications Systems Using Unreliable Network Repeater Nodes (P-1995 Paper). The RAND Corporation. https://www.rand.org/pubs/papers/P1995.html Barlow, J. P. (1996). A declaration of the independence of cyberspace. https://www.eff.org/cyberspace-independence Bell, D. E., & LaPadula, L. J. (1973). Secure Computer Systems: Mathematical Foundations (Technical Report No. 2547; Issue 2547). MITRE Corporation. Biba, K. J. (1975). Integrity Considerations for Secure Computer Systems (Technical Report MTR-3153). MITRE Corporation. Brown, I., Marsden, C. T., Lee, J., & Veale, M. (2020). Cybersecurity for elections: A Commonwealth guide on best practice. Commonwealth Secretatiat. https://doi.org/10.31228/osf.io/tsdfb Buzan, B., Wæver, O., & De Wilde, J. (1998). Security: A new framework for analysis. Lynne Rienner Publishers. Camp, P. L. J. (2013). Beyond usability: Security Interactions as Risk Perceptions [Position paper]. https://core.ac.uk/display/23535917 Carmody, S. (2020). ngramr: Retrieve and Plot Google n-Gram Data (1.7.2) [Computer software]. https://CRAN.R-project.org/package=ngramr Cavelty, M. D. (2020). Cybersecurity between hypersecuritization and technological routine. In E. Tikk & M. Kerttunen (Eds.), Routledge Handbook of International Cybersecurity (1st ed., pp. 11–21). Routledge. https://doi.org/10.4324/9781351038904-3 Chang, F. R. (2012). Guest Editor’s Column. The Next Wave, 19(4). https://www.nsa.gov/Portals/70/documents/resources/everyone/digital-media-center/publications/the-next-wave/TNW-19-4.pdf Clark, D. D., & Wilson, D. R. (1987). A Comparison of Commercial and Military Computer Security Policies. 184–194. https://doi.org/10.1109/SP.1987.10001 Clarke, R. (2005, May 9). Human-Artefact Hybridisation: Forms and Consequences. Ars Electronica 2005 Symposium, Linz, Austria. http://www.rogerclarke.com/SOS/HAH0505.html Clarke, R. (2016). Privacy Impact Assessments as a Control Mechanism for Australian National Security Initiatives. Computer Law & Security Review, 32(3), 403–418. https://doi.org/10.1016/j.clsr.2016.01.009 Clarke, R. (2017). Cyberspace, the Law, and our Future [Talk]. Issue Launch of Thematic Issue Cyberspace and the Law, UNSW Law Journal, Sydney. http://www.rogerclarke.com/II/UNSWLJ-CL17.pdf Cohen, J. E. (2012). Configuring the Networked Self: Law, Code, and the Play of Everyday Practice. Yale University Press. http://juliecohen.com/configuring-the-networked-self Craigen, D., Diakun-Thibault, N., & Purse, R. (2014). Defining Cybersecurity. Technology Innovation Management Review, 4(10), 13–21. https://doi.org/10.22215/timreview/835 Denning, D. E. R. (1982). Cryptography and data security. Addison-Wesley Longman Publishing Co., Inc. Diffie, W., & Landau, S. (2010). Privacy on the Line: The Politics of Wiretapping and Encryption. MIT Press. https://library.oapen.org/handle/20.500.12657/26072 DiMaggio, P. J., & Powell, W. W. (1983). The Iron Cage Revisited: Institutional Isomorphism and Collective Rationality in Organizational Fields. American Sociological Review, 48(2), 147. https://doi.org/10.2307/2095101 Erickson, P., Klein, J. L., Daston, L., Lemov, R. M., Sturm, T., & Gordin, M. D. (2013). How Reason Almost Lost its Mind: The Strange Career of Cold War Rationality. The University of Chicago Press. https://doi.org/10.7208/chicago/9780226046778.001.0001 Felt, A. P., Ainslie, A., Reeder, R. W., Consolvo, S., Thyagaraja, S., Bettes, A., Harris, H., & Grimes, J. (2015). Improving SSL Warnings: Comprehension and Adherence. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI, 15, 2893–2902. https://doi.org/10.1145/2702123.2702442 Fiesler, C., Beard, N., & Keegan, B. C. (2020). No Robots, Spiders, or Scrapers: Legal and Ethical Regulation of Data Collection Methods in Social Media Terms of Service. Proceedings of the International AAAI Conference on Web and Social Media, 14(1), 187–196. Freed, D., Palmer, J., Minchala, D., Levy, K., Ristenpart, T., & Dell, N. (2018). "A Stalker’s Paradise”: How Intimate Partner Abusers Exploit Technology. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 667, 1–667. https://doi.org/10.1145/3173574.3174241 Friedler, S. A., Scheidegger, C., Venkatasubramanian, S., Choudhary, S., Hamilton, E. P., & Roth, D. (2019). A comparative study of fairness-enhancing interventions in machine learning. Proceedings of the Conference on Fairness, Accountability, and Transparency - FAT*, 19, 329–338. https://doi.org/10.1145/3287560.3287589 Gangadharan, S. P., & Niklas, J. (2019). Decentering technology in discourse on discrimination. Information, Communication & Society, 22(7), 882–899. https://doi.org/10.1080/1369118X.2019.1593484 Global Cyber Security Capacity Centre. (2016). Cybersecurity Capacity Maturity Model for Nations (CMM) Revised Edition. Global Cyber Security Capacity Centre, University of Oxford. https://doi.org/10.2139/ssrn.3657116 Graham, M. (2013). Geography/internet: Ethereal alternate dimensions of cyberspace or grounded augmented realities? The Geographical Journal, 179(2), 177–182. https://doi.org/10.1111/geoj.12009 Greenleaf, G., & Cottier, B. (2020). 2020 ends a decade of 62 new data privacy laws. Privacy Laws & Business International Report, 163, 24–26. Grossman, W. (2017, June). Crossing the Streams: Lizzie Coles-Kemp. Research Institute for the Science of Cyber Security Blog. Gürses, S. (2014). Can you engineer privacy? Communications of the ACM, 57(8), 20–23. https://doi.org/10.1145/2633029 Hansen, L., & Nissenbaum, H. (2009). Digital Disaster, Cyber Security, and the Copenhagen School. International Studies Quarterly, 53(4), 1155–1175. https://doi.org/10.1111/j.1468-2478.2009.00572.x International Telecommunication Union. (2019, March). National CIRTs Worldwide [Perma.cc record]. https://perma.cc/MSL6-MSHZ I.T.U.-T. (2008, April 18). X.1205: Overview of cybersecurity. https://www.itu.int/rec/T-REC-X.1205-200804-I Kabanov, Y. (2014). Information (Cyber-) Security Discourses and Policies in the European Union and Russia: A Comparative Analysis (WP 2014-01. Centre for German and European Studies (CGES. https://zdes.spbu.ru/images/working_papers/wp_2014/WP_2014_1–Kabanov.compressed.pdf Kanwal, G. (2009). China’s Emerging Cyber War DoctrineJournal of Defence Studies, 3(3). Kemmerer, R. A. (2003). Cybersecurity. 25th International Conference on Software Engineering, 2003. Proceedings, 705–715. https://doi.org/10.1109/ICSE.2003.1201257 Kerr, O. S. (2003). Cybercrime’s Scope: Interpreting Access and Authorization in Computer Misuse Statutes. New York University Law Review, 78(5), 1596–1668. Kerr, O. S. (2016). Norms of Computer Trespass. Columbia Law Review, 116, 1143–1184. Koops, B.-J. (2006). Should ICT Regulation Be Technology-Neutral? In B.-J. Koops, C. Prins, M. Schellekens, & M. Lips (Eds.), Starting Points for ICT Regulation: Deconstructing Prevalent Policy One-liners (pp. 77–108). T.M.C. Asser Press. Korff, D. (2019). First do no harm: The potential of harm being caused to fundamental rights and freedoms by state cybersecurity interventions. In Research Handbook on Human Rights and Digital Technology. Elgar. Kosseff, J. (2020). Cybersecurity law (Second). Wiley. https://doi.org/10.1002/9781119517436 Kostova, B., Gürses, S., & Troncoso, C. (2020). Privacy Engineering Meets Software Engineering. On the Challenges of Engineering Privacy By Design. ArXiv. http://arxiv.org/abs/2007.08613 Krombholz, K., Busse, K., Pfeffer, K., Smith, M., & Zezschwitz, E. (2019). 'If HTTPS Were Secure, I Wouldn’t Need 2FA’—End User and Administrator Mental Models of HTTPS. 246–263. https://doi.org/10.1109/sp.2019.00060 Krombholz, K., Mayer, W., Schmiedecker, M., & Weippl, E. (2017). I Have No Idea What I’m Doing’—On the Usability of Deploying HTTPS. 1339–1356. https://www.usenix.org/conference/usenixsecurity17/technical-sessions/presentation/krombholz Kshetri, N. (2016). Cybersecurity and Development. Markets, Globalization & Development Review, 1(2). https://doi.org/10.23860/MGDR-2016-01-02-03 Lee, R. M., & Rid, T. (2014). OMG Cyber! The RUSI Journal, 159(5), 4–12. https://doi.org/10.1080/03071847.2014.969932 Levy, I. (2020). High level privacy and security design for NHS COVID-19 Contact Tracing App. National Cyber Security Centre. https://www.ncsc.gov.uk/files/NHS-app-security-paper%20V0.1.pdf Levy, K., & Schneier, B. (2020). Privacy threats in intimate relationships. Journal of Cybersecurity, 6(1). https://doi.org/10.1093/cybsec/tyaa006 Lin, Y., Michel, J.-B., Aiden, E. L., Orwant, J., Brockman, W., & Petrov, S. (2012). Syntactic annotations for the Google Books ngram corpus. Proceedings of the ACL 2012 System Demonstrations, 169–174. Matwyshyn, A. M. (2017). CYBER! Brigham Young University Law Review, 2017(5), 1109. https://digitalcommons.law.byu.edu/lawreview/vol2017/iss5/6/ Maurer, T., Hohmann, M., Skierka, I., & Morgus, R. (2015). National CSIRTs and Their Role in Computer Security Incident Response [Policy Paper]. New America; Global Public Policy Institute. http://newamerica.org/cybersecurity-initiative/policy-papers/national-csirts-and-their-role-in-computer-security-incident-response/ Maxwell, J. C. (1867-1868). On Governors. Proceedings of the Royal Society of London, Vol. 16 (1867 - 1868), pp. 270-283 Miller, B. (2010, March 1). CIA Triad [Blog post]. Electricfork. http://blog.electricfork.com/2010/03/cia-triad.html Mitnick, K. D., & Simon, W. L. (2002). The Art of Deception: Controlling the Human Element of Security. Wiley. Moyle, E. (2019). CSIRT vs. SOC: What’s the difference? In Ultimate guide to cybersecurity incident response [TechTarget SearchSecurity]. https://searchsecurity.techtarget.com/tip/CERT-vs-CSIRT-vs-SOC-Whats-the-difference Naiakshina, A., Danilova, A., Gerlitz, E., Zezschwitz, E., & Smith, M. (2019). 'If you want, I can store the encrypted password’: A Password-Storage Field Study with Freelance Developers. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 19, 1–12. https://doi.org/10.1145/3290605.3300370 Neale, M. (2000, October 4). No Maps for These Territories [Documentary]. Mark Neale Productions. Neumann, A. J., Statland, N., & Webb, R. D. (1977). Post-processing audit tools and techniques. In Z. G. Ruthberg (Ed.), Audit and evaluation of computer security (pp. 2–5). National Bureau of Standards. https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nbsspecialpublication500-19.pdf Nissenbaum, H. (2005). Where Computer Security Meets National Security. Ethics and Information Technology, 7(2), 61–73. https://doi.org/10.1007/s10676-005-4582-3 Reeder, R. W., Felt, A. P., Consolvo, S., Malkin, N., Thompson, C., & Egelman, S. (2018). An Experience Sampling Study of User Reactions to Browser Warnings in the Field. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 18, 1–13. https://doi.org/10.1145/3173574.3174086 Rid, T. (2016). Rise of the Machines: The lost history of cybernetics. Scribe. Saltzer, J. H., & Schroeder, M. D. (1975). The protection of information in computer systems. Proceedings of the IEEE, 63(9), 1278–1308. https://doi.org/10.1109/PROC.1975.9939 Sasse, M. A., Brostoff, S., & Weirich, D. (2001). Transforming the ‘Weakest Link’—A Human/Computer Interaction Approach to Usable and Effective Security. BT Technology Journal, 19(3), 122–131. https://doi.org/10.1023/a:1011902718709 Sellars, A. (2018). Twenty Years of Web Scraping and the Computer Fraud and Abuse Act. Boston University Journal of Science & Technology Law, 24(2), 372. https://scholarship.law.bu.edu/faculty_scholarship/465/ Silverstone, R., & Hirsch, E. (1992). Consuming Technologies: Media and Information in Domestic Spaces. Routledge. https://doi.org/10.4324/9780203401491 Snowden, E. (2019). Permanent Record. Pan Macmillan. Solms, R., & Niekerk, J. (2013). From information security to cyber security. Computers & Security, 38, 97–102. https://doi.org/10.1016/j.cose.2013.04.004 Tanczer, L. M., Brass, I., & Carr, M. (2018). CSIRTs and Global Cybersecurity: How Technical Experts Support Science Diplomacy. Global Policy, 9(S3), 60–66. https://doi.org/10.1111/1758-5899.12625 Wagner, B., & Vieth, K. (2016). Was macht Cyber? Epistemologie und Funktionslogik von Cyber. Zeitschrift für Außen- und Sicherheitspolitik, 9(2), 213–222. https://doi.org/10.1007/s12399-016-0557-1 Ware, W. (1970). Security Controls for Computer Systems: Report of Defense Science Board Task Force on Computer Security (Issues R609-1)) [Report]. The RAND Corporation. https://doi.org/10.7249/R609-1 Whitten, A., & Tygar, J. D. (1999). Why Johnny can’t encrypt: A usability evaluation of PGP 5.0. Proceedings of the 8th Conference on USENIX Security Symposium, 8. https://www.usenix.org/legacy/events/sec99/full_papers/whitten/whitten.ps Wiener, N. (1948). Cybernetics: Or Control and Communication in the Animal and the Machine. MIT Press. Zittrain, J. L. (2006). The Generative Internet. Harvard Law Review, 119, 1974–2040. http://nrs.harvard.edu/urn-3:HUL.InstRepos:9385626 Appendix 1 – Cybersecurity in other languages 5 Table 1: Terms for cybersecurity (via Google Translate on 14 September 2020, checked against by native speakers). Language Term Afrikaans kubersekuriteit Arabic الأمن الإلكتروني Bengali সাইবার নিরাপত্তা Bulgarian киберсигурност Chinese 网络安全 Danish computersikkerhed Dutch cyberbeveiliging Finnish Kyberturvallisuus Farsi امنیت شبکه (or امنیت سایبری/ امنیت رایانه) French la cyber-sécurité German Cybersicherheit (sometimes IT-sicherheit, Informationssicherheit, or Onlinesicherheit in Austria) Greek κυβερνασφάλεια Hindi साइबर सुरक्षा Bahasa Indonesia keamanan siber Italian sicurezza informatica Japanese サイバーセキュリティ Portuguese cíber segurança Marathi सायबर सुरक्षा Romanian securitate cibernetica Russian кибербезопасность Spanish ciberseguridad or (more popularly) seguridad informática Swahili usalama wa mtandao Swedish Cybersäkerhet (or, commonly, IT-säkerhet) Urdu سائبر سیکورٹی Xhosa ukhuseleko One important difference between European languages is that some (such as English) differentiate security and safety, while others (such as Swedish and Danish) do not. One sociologist of security noted: “it does frame how you understand the concepts, particularly structure. When you're talking about access control in Swedish it's a different logic than when you talk about it in Anglo-Saxon languages […] In the Scandinavian view of the world there is always a much more socio-technical bent for thinking about security” (Grossman, 2017). Footnotes 1. The authors use cybersecurity, not cyber security, throughout this text, as it is the one most in use in computer science, even in Britain. 2. The second author must admit he has not been immune to this. 3. Ware’s 1970 report begins: “Although this report contains no information not available in a well stocked technical library or not known to computer experts, and although there is little or nothing in it directly attributable to classified sources…” 4. See the Twitter hashtag #cybercyber and @cybercyber account, and Google search results for “cyber cyber cyber", for hundreds of thousands of further examples, and the “cyber song” and video Unsere Cyber Cyber Regierung - Jung & Naiv: Ultra Edition. 5. According to Google Translate, confirmed or updated by native speakers consulted by the authors, including the top-15 most spoken languages according to Wikipedia. With thanks to Eleftherios Chelioudakis, Francis Davey, Fukami, Andreas Grammenos, Hamed Haddadi, Werner Hülsmann, Douwe Korff, Sagwadi Mabunda, Bogdan Manolea, Matthias Marx, Veni Markovski, Grace Mutung'u, Yudhistira Nugraha, Jan Penfrat, Judith Rauhofer, Kaspar Rosager, Eric Skoglund, Anri van der Spuy and Mathias Vermeulen for many of these translations!...