Cracked Labs

Institute for Critical Digital Culture

How Companies Use Personal Data Against People

Automated disadvantage and personalized manipulation? A working paper on the societal ramifications of the commercial use of personal information, with a focus on automated decision-making, personalization, and data-driven behavioral change.

Working Paper by Wolfie Christl, Cracked Labs, October 2017.

The large-scale commercial exploitation of digital personal information raises major concerns about the future of autonomy, equality, human dignity, and democracy. Our previous report, published in June 2017, documented the massive scale and scope of how companies collect, disclose, trade, and utilize personal information about individuals today (Web, PDF).

Our new working paper further explores and examines how the corporate aggregation and use of personal data can affect individuals, groups of people, and society at large, in particular in the context of two partially overlapping areas of concern: automated decisions and data-driven persuasion.

Download

How Companies Use Personal Data Against People.
Automated Disadvantage, Personalized Persuasion, and the Societal Ramifications of the Commercial Use of Personal Information.
Working paper by Cracked Labs, October 2017. Author: Wolfie Christl. Contributors: Katharina Kopp, Patrick Urs Riechert.
» Download as PDF

Abstract

Today, companies aggregate, trade, and utilize personal information at unprecedented levels. Their unilateral and extensive access to data about the characteristics, behaviors, and lives of billions allows them to constantly monitor, follow, judge, sort, rate, and rank people as they see fit. Our previous report documented the massive scale and scope of today’s networks of digital tracking and profiling. It investigated relevant industries, business models, platforms, services, devices, technologies, and data flows, focusing on their implications for people – whether as individuals, consumers, or citizens – and society at large.

This working paper examines how the corporate use of personal information can affect individuals, groups of people, and society at large, particularly in the context of automated decisions, personalization and data-driven persuasion. After briefly reviewing our previous research’s findings and key developments in recent years, this paper explores their potential to be used against people in detail.

Systems that make decisions about people based on their data produce substantial adverse effects that can massively limit their choices, opportunities, and life-chances. These systems are largely opaque, nontransparent, arbitrary, biased, unfair, and unaccountable – even in areas such as credit rating that have long been regulated in some way. Through data-driven personalization, companies and other institutions can easily utilize information asymmetries in order to exploit personal weaknesses with calculated efficiency. Personalized persuasion strategies provide the means to effectively influence behavior at scale. As companies increasingly and unilaterally shape the networked environments and experiences that underlie and determine everyday life, manipulative, misleading, deceptive, or even coercive strategies can be automated and customized down to the individual level.

Based on the examination of business practices and their implications we conclude that, in their current state, today’s commercial networks of digital tracking and profiling show a massive potential to limit personal agency, autonomy, and human dignity. This not only deeply affects individuals, but also society at large. By improving the ability to exclude or precisely target already disadvantaged groups, current corporate practices utilizing personal information tend toward disproportionally affecting these groups and therefore increase social and economic inequality. Especially when combined with influencing strategies derived from neuroeconomics and behavioral economics, data-driven persuasion undermines the concept of rational choice and thus the basic foundation of market economy. When used in political campaigns or in other efforts to shape public policy, it may undermine democracy at large.

While this working paper does not directly offer solutions, it examines, documents, structures, and contextualizes today’s commercial personal data industries and their implications; further research will build on this basis. Hopefully, it will also encourage and contribute to further work by others.

The production of this report was supported by the Open Society Foundations.

„You have to fight for your privacy or you will lose it“

Eric Schmidt, Google/Alphabet, 2013

Our Previous Reports and Publications

Report, June 2017
Corporate Surveillance in Everyday Life.
How Companies Collect, Combine, Analyze, Trade, and Use Personal Data on Billions. A Report by Cracked Labs, Vienna, June 2017. Author: Wolfie Christl. Contributors: Katharina Kopp, Patrick Urs Riechert. Illustrations: Pascale Osterwalder.
» Web publication with infographics
» Info page, summary & full report as PDF

Report/Book, October 2016
Networks of Control.
A Report on Corporate Surveillance, Digital Tracking, Big Data & Privacy. Publisher: Facultas, Vienna, 2016. Authors: Wolfie Christl and Sarah Spiekermann. A collaboration between Cracked Labs and the Institute for Management Information Systems at the Vienna University of Economics and Business, chaired by Prof. Sarah Spiekermann.
» Info & full report as PDF
» Buy the Book at Facultas

Coverage / What Others Say

On “Networks of Control”:

“the most comprehensive study ever carried out on this aspect of the digital economy”
La Stampa, Italy

“a must-read for anyone who is interested in today's data-driven world”
Paul Nemitz, Director Fundamental Rights, DG Justice, EU Commission

“having such a collection of examples in one place is really exciting”
– Anna Fielder, Chair of Privacy International

On “Corporate Surveillance in Everyday Life”:

“Comprehensive report on what it means for surveillance to be the default business model for so much of the internet”
– Zeynep Tufekci, Writer and Techno-Sociologist

“Filled with a variety of colorful infographics about data collection, predictive analytics and the relation between major players in the personal data industry, the report is a useful overview of this incredibly complex collection of business relationships and data flows”
– Cobun Keegan, International Association of Privacy Professionals

“authoritative & chilling report on how big corporates track your every move”
– Carole Cadwalladr, The Guardian / Observer

“Ninety-three glorious pages”
– Bruce Sterling on wired.com

Wolfie Christl

Wolfie Christl is a technologist, researcher, writer, and digital rights activist based in Vienna, Austria. In 2013, he co-created Data Dealer, an award-winning online game on personal data and privacy. His 2016 book Networks of Control is, according to the “International Association of Privacy Professionals”, a “slow-burning horror film”, and, according to Paul Nemitz, Director Fundamental Rights at the Directorate-General Justice of the European Commission, “a must-read for anyone who is interested in today's data-driven world”. Wolfie Christl has presented his research in the European Parliament, has contributed to TV documentaries about digital tracking, works as a trainer for employee privacy, and writes for newspapers such as the German FAZ. From 2000 to 2006 he was a part of Public Netbase, a digital art platform, non-profit ISP, and somewhat of an early hackerspace in Vienna. He and his projects have been featured in the New York Times, Forbes, and many other media outlets around the world.

Cracked Labs

Cracked Labs is an independent research institute and creative laboratory based in Vienna, Austria. It investigates the socio-cultural impacts of information technology and develops social innovations in the field of digital culture. Cracked Labs is a non-profit organization. It was established in 2012 to strengthen a participatory and self-determined use of information and communication technology as well as the free access to knowledge and information – independently from commercial or governmental interests.