CHAPTER TEN

Human Dignity and Democracy

#human well-being  #privacy  #discrimination  #proportionality

The purpose of this book is to discuss the application of data protection law and marketing law to value extraction in the guise of data-driven marketing. In particular, we have addressed how business models create new forms of information asymmetry, placing consumers at greater disadvantage vis-à-vis traders. Empowerment and informed decisions require agency, transparency and the absence of manipulation. This may be difficult to ensure in a context where technology is used both to observe and to shape behaviour¹—at scale and in real time as well as across platforms.

1. Fundamental (human) rights

In addition to affecting the economic interests of consumers, data-driven marketing poses a risk for fundamental rights that protect human beings and democracy as such. We have dealt with the fundamental right to the protection of personal data, which is a subset of the broader concept of privacy. In this chapter, we add a constitutional perspective by including reference to guaranteed fundamental rights and freedoms that go beyond market regulation but are challenged by activities in markets.

Law is not an end in itself, but a means to protect certain values. It follows from Article 2 TEU that

‘The Union is founded on the values of respect for human dignity, freedom, democracy, equality, the rule of law and respect for human rights […].’

In essence, law is a matter of balancing interests, which becomes more complex with this broader perspective. This complexity can be illustrated by the aims pursued under the Treaty of the European Union (TEU):

Article 3

‘1.  The Union’s aim is to promote peace, its values and the well-being of its peoples.

[…]

3.  The Union shall establish an internal market. It shall work for the sustainable development of Europe based on balanced economic growth and price stability, a highly competitive social market economy, aiming at full employment and social progress, and a high level of protection and improvement of the quality of the environment. It shall promote scientific and technological advance.

It shall combat social exclusion and discrimination, and shall promote social justice and protection, equality between women and men, solidarity between generations and protection of the rights of the child.

It shall promote economic, social and territorial cohesion, and solidarity among Member States.

It shall respect its rich cultural and linguistic diversity, and shall ensure that Europe’s cultural heritage is safeguarded and enhanced.

[…].’

As mentioned in Chapter 3 (regulating markets), teleological interpretation necessitates that ‘every provision of [EU] law must be placed in its context and interpreted in the light of the provisions of [EU] law as a whole, regard being had to the objectives thereof and to its state of evolution at the date on which the provision in question is to be applied’.² In this vein primary law can be perceived as a lens through which secondary law should be observed. Thus, secondary law serves as an important vehicle for realising the objectives stated in the Charter, which must be interpreted with due regard to the accompanying explanations.³

As discussed in previous chapters, there is already a legal framework to ensure consumer empowerment in the context of data-driven business models. Here, the hypothesis is that the need for agency, transparency and absence of manipulation may be corroborated by fundamental rights laid out in the Charter. We focus on human dignity, privacy and discrimination, which must be balanced against the right to conduct a business. The argument is that this broader range of fundamental rights may play a significant role in consumer protection law, including the protection of personal data, in the context of data-driven business models.

If we analyse the application of individual legal fields (bottom-up) without considering the bigger picture, not only do we fail to place EU law in its context and interpret it in light of EU law as a whole; we may also fail to understand the potential for personal, social and societal harm that some aspects of data-driven business models pose. It makes a significant difference whether we balance data-driven marketing only against market interests, including competition and consumers’ economic interests, or consider other interests, such as human dignity and democracy.

The importance of fundamental rights in the context of AI has also been recognised by the United Nations, which has suggested a moratorium on the sale and use of AI systems until adequate safeguards are put in place.⁴ As discussed in the previous chapters, consumer protection law, including the protection of personal data, does provide safeguards for the use of AI to personalise marketing.

In his analysis of a possible constitutionalisation of consumer law, Hans-Wolfgang Micklitz asks whether there is a ‘fundamental and human right to be treated as a consumer’, which may ‘open up a new round of debate about the relationship between the Internal Market and Fundamental Rights’. He adds that:⁵

‘Academics will have the task of investigating whether the constitutionalised consumer is just another variant in the overall history of consumer protection or whether this concept will turn into a new page in the history of private law—of constitutionalised private law beyond the state through the Charter of Fundamental Rights and not forgetting the European Convention on Human Rights.’

We argue that this constitionalisation must be real. Firstly, this is so because consumers are also citizens, and their engagements in markets—when they are on the consumer side of data-driven business models—have significant effects not only on the individual user but also on social and societal relationships. Secondly, fundamental rights ‘form an integral part of the general principles of law the observance of which the Court ensures’,⁶ and because ignoring the Charter in market law would render the protection of citizens and democracy illusionary. Thirdly, if we accept that data protection law is part of consumer law, this in itself requires a constitutional perspective.

Thus, we argue that the protection of human dignity, privacy and non-discrimination is likely to corroborate consumer protection law, including data protection law.

1.1. Fundamental rights and business

A free market, together with individualism, human rights and democracy, is part of what can be described as the ‘liberal package’.⁷ From the preamble to the Charter, it follows that its aims include balanced and sustainable development and ensuring free movement of persons, services, goods and capital, and the freedom of establishment.

The Charter is not aimed at traders and markets, as it follows from Article 51(1) of the Charter that:

‘The provisions of this Charter are addressed to the institutions, bodies, offices and agencies of the Union with due regard for the principle of subsidiarity and to the Member States only when they are implementing Union law. They shall therefore respect the rights, observe the principles and promote the application thereof in accordance with their respective powers and respecting the limits of the powers of the Union as conferred on it in the Treaties.’

Rights of the Charter are the same with regard to meaning and scope as corresponding rights in the ECHR as interpreted by the ECtHR, without preventing more extensive protection under EU law.⁸ Interpretation of human rights is dynamic and ‘it is of crucial importance that the [ECHR] is interpreted and applied in a manner which renders its rights practical and effective, not theoretical and illusory.’⁹ It could be argued that practical and effective application of the Charter must entail that individuals and democracy as such should not be less protected when infringement of the envisaged fundamental rights and freedoms results from traders’ exploitation of human frailties, including by persuasive technology.

Even though the Charter is directed at states, fundamental rights may be directly applicable to businesses or impose an obligation for Member States to ensure that the aims are pursued,¹⁰ and it remains an important question how fundamental rights apply to non-state actors.¹¹ In the context of privacy, the ECtHR has ruled that in some situations states have positive obligations to secure respect for private life, by adopting measures—beyond an opportunity to claim compensation for damages—designed to secure this right; even in ‘the sphere of the relations of individuals between themselves’.¹²

The CJEU has established that the Charter’s prohibition of discrimination is ‘sufficient in itself to confer on individuals a right which they may rely on as such in disputes between them in a field covered by EU law’.¹³ The protection of human dignity and respect for privacy are not equally precise, but are also far from devoid of meaning.

A principle of sincere cooperation is expressed in Article 4(3) TEU, which says that Member States must (a) ensure fulfilment of the obligations arising out of the Treaties and (b) facilitate the achievement of the Union’s tasks and (c) refrain from any measure which could jeopardise the attainment of the Union’s objectives.¹⁴

1.2. Commercial practices and fundamental rights

Both the GDPR and the UCPD are secondary law, and as such they derive their authority from a higher level of the legal hierarchy, i.e. primary law. Because privacy, including the protection of personal data, is secured in the Charter, it is a straightforward approach to take a fundamental rights’ perspective in the interpretation of the ePrivacy Directive (privacy) and the GDPR (personal data).

When it comes to marketing law, the Charter (only) provides for ensuring a high level of consumer protection in ‘Union policies’ (Article 38 that is based on Article 169 TFEU), which flavour is ‘more political aspiration than independently enforceable legal norm’.¹⁵ Article 38 must be ‘observed’ but does not provide subjective rights that must be ‘respected’.¹⁶

The UCPD applies to protecting the economic interests of consumers, and does not address ‘legal requirements related to taste and decency’ to the extent that the economic interests of consumers are not affected. Thus, Member States may apply national rules to protect human dignity and prevent sexual, racial and religious discrimination and the depiction of nudity, violence and antisocial behaviour.¹⁷

It is clear that the UCPD applies in situations where a commercial practice infringes on both the economic interests of consumers and taste and decency, which may include human dignity, privacy and non-discrimination. Doorstep selling is an example of a commercial practice that may affect both economic interests and privacy. In a case (predating the UCPD) concerning the free movement of goods, the CJEU found that consumer protection could justify restrictions of doorstep selling of silver jewellery. The court mentioned that in the assessment of proportionality, it must be considered that such practices have a

‘potentially higher risk of the consumer being cheated due to a lack of information, the impossibility of comparing prices or the provision of insufficient safeguards as regards the authenticity of that jewellery and the greater psychological pressure to buy where the sale is organised in a private setting.’¹⁸

The court focused on both the psychological pressure and the lack of information, including by means of price comparison. Both aspects are relevant to data-driven marketing and fall within the scope of the UCPD. Even though the internet allows for easy price comparison of offers from different traders, personalised marketing means that consumers may find it difficult to compare their offers with offers provided (by the same trader) to other consumers. This may be a similarly important comparison, as we discuss below in the context of non-discrimination.

As there is no doubt that the UCPD applies in such situations, the real question concerns the extent to which ‘taste and decency’ may be included in the interpretation and assessment of professional diligence:

‘Professional diligence’ means ‘the standard of special skill and care which a trader may reasonably be expected to exercise towards consumers, commensurate with honest market practice and/or the general principle of good faith in the trader’s field of activity.’

The wording is sufficiently broad to include interests pertaining to, for instance, human dignity, privacy and discrimination, provided that the commercial practice in question is likely to affect (also) the economic interests of consumers. The question is not settled by the CJEU, but it seems well aligned with the aims of the Charter and the need for an effective interpretation of the UCPD to include taste and decency in determining professional diligence.

The opposite result—because of the full harmonisation nature of the directive—would mean that consumers would not be protected from taste and decency-aspects of marketing when the commercial practice in question would also be relevant to the economic interests of consumers.

2. Human dignity

Human dignity is the mother of all human rights, as it ‘constitutes the real basis of fundamental rights’.¹⁹ Article 1 of the Charter provides that ‘human dignity is inviolable’ and ‘must be respected and protected’. The concept is both important and vague, and has been explained in the following manner:

‘All in all, human dignity has its roots deep in the origins of a conception of mankind in European culture that regards man as an entity capable of spontaneity and self-determination. Because of his ability to forge his own free will, he is a person (subject) and must not be downgraded to a thing or object.’²⁰

Thus, human dignity can be said to protect ‘the authority of human beings to govern their own lives’.²¹ In the context of data-driven business models, human beings may be observed as programmable and hackable algorithms, as we react predictably (in probabilistic terms) to external stimuli that may be delivered through the technology we use.

Data-driven business models rely on the commercialisation and commodification of human attention and the human experience.²² When users’ attention and agency are being conditioned for behaviour modification, consumers become more valuable for traders.

‘Machine becomes manlike and man becomes machinelike, a convergence that may augment our capabilities yet reduce our humanity, however we define it.’²³ People may ‘no longer see themselves as autonomous beings running their lives according to their wishes, but instead will become accustomed to seeing themselves as a collection of biochemical mechanisms that is constantly monitored and guided by a network of electronic algorithms.’²⁴

Both persuasive and addictive technology and the commercialisation of all individual behaviour and experience—public and private—may be said to undermine human dignity to some degree. This is especially true to the extent that they negatively impact personal, social and societal integrity and cohesion, as we discuss below. Manipulation can be seen as an insult to both autonomy and dignity, the latter by being humiliating.²⁵ However, human dignity is a double-edged sword, as we should also be free to err and learn, recognising the antifragile nature of human beings.

Antifragility can be defined as ‘anything that has more upside than downside from random events (or certain shocks) […]; the reverse is fragile.’²⁶ To ensure the antifragile nature of human beings, the random events must be recognisable and not devastating in order to support healthy development (‘what does not kill me makes me stronger’²⁷) rather than resignation and apathy. Relying too heavily on technology may lead to ‘digital amnesia’.²⁸

Freedom in a market perspective entails the agency to make rational choices that are informed; reflect the consumer’s goals, values and preferences; and are restrained by available choices, bounded rationality and choice architecture. In order to cater to human antifragility in markets (and elsewhere), transparency and the absence of manipulation are important.

2.1. Paying with human dignity

Human dignity is found in Title I of the Charter, which consists of Articles 1–5 under the heading ‘dignity’. Article 3(1) provides that ‘everyone has the right to respect for his or her physical and mental integrity.’ As predatory business models can intrude upon the integrity of persons, including by means of invading privacy and agency, this provision could serve as inspiration for understanding human dignity in the context of data-driven marketing. This could include situations where Artificial Intelligence is used to increase engagement, with addiction, outrage and polarisation as side effects.

Article 3(2)(c) further provides for ‘the prohibition on making the human body and its parts as such a source of financial gain’ in the fields of medicine and biology. This provision is obviously not applicable to data-driven marketing, even though surveillance capitalism relies on body parts, including the brain,²⁹ for financial gains and though, at least some data can be said to fall within the field of biology, including by proxies.

In the context of patents, the CJEU has established—with reference to safeguards concerning human dignity—that the human body may not be appropriated in its natural environment, but it may be part of a product which is patentable.³⁰

It could be argued that the design of behaviour modification experiments that are carried out without democratic oversight or scientific approval is contrary to human dignity, especially considering that their purpose is financial gain. In essence, the question is whether payment for products should include a part of oneself.³¹

2.2. Human well-being

It follows from Article 3(1) TEU that the aims of the European Union include ‘Union values and the well-being of its peoples’.³² In the early stages of the development of EU consumer policy, mentioned that ‘the improvement of the quality of life is one of the tasks of the Community and as such implies protecting the health, safety and economic interests of the consumer’.³³ The U.S. president John F. Kennedy has also stated that:³⁴

‘[…] If consumers are offered inferior products, if prices are exorbitant, if drugs are unsafe or worthless, if the consumer is unable to choose on an informed basis, then his dollar is wasted, his health and safety may be threatened, and the national interest suffers. On the other hand, increased efforts to make the best possible use of their incomes can contribute more to the well-being of most people than equivalent efforts to raise their incomes.’

In his presentation of this ‘special message’ he also stated that that the aims of consumer protection were important to ‘the well-being of every American family’.

Attention is a scarce resource—ultimately a zero-sum game³⁵—that, along with memory, lies at the core of our identities³⁶ and is important for how we live and experience our lives. As observed by William James:

‘When we reach the end of our days, our life experience will equal what we have paid attention to, whether by choice or default. We are at risk, without quite fully realizing it, of living lives that are less our own than we imagine.’³⁷

Human well-being is a complex concept that may seem more philosophical than legal, and material wealth is primarily important for the well-being of poorer people, who, for instance, may be susceptible to payday loans.

If one were to point at a default aim for human well-being, it could be to live longer and richer lives with deep and meaningful human interactions. Having a purpose,³⁸ being understood and having agency are important for well-being;³⁹ manipulation is not. The quality of human life also depends on our capacity for presence and boredom,⁴⁰ as well as our relationship to the world.⁴¹

Psychologists seem to agree that well-being depends primarily on the quality of social relations⁴² and meaningful work⁴³ when certain ‘human needs’ are fulfilled, as described by Abraham Maslov.⁴⁴ Sigmund Freud has emphasised the important role of ‘taking responsibility’,⁴⁵ and it remains safe to assume that he did not have in mind the consumer reading through extensive terms and conditions.

Unsolicited distractions may undermine human well-being, as they may erode ‘our capacity for deep, sustained, perceptive attention’ that are the building blocks of ‘intimacy, wisdom, and cultural progress’.⁴⁶ The revelations by The Wall Street Journal in the ‘Facebook Files’ include the observations that Facebook (now Meta) has been conducting studies over several years to understand how Instagram affects its millions of young users; and that ‘32% of teen girls said that when they felt bad about their bodies, Instagram made them feel worse’; and that ‘among teens who reported suicidal thoughts, 13% of British users and 6% of American users traced the desire to kill themselves to Instagram’.⁴⁷

The UCPD is without prejudice to rules relating to the health and safety aspects of products (Article 3(3)), which means that the fairness of a commercial practice, according to the directive, does not affect such rules, but that a failure to comply with such a rule may be considered an unfair commercial practice.⁴⁸ It is clear from Article 3(4) UCPD that if there is a conflict between the directive and other community rules regulating specific aspects of unfair commercial practices, the latter is to prevail and will apply to those specific aspects.⁴⁹

In a case concerning a national provision restricting the marketing of oral and dental care services, the CJEU found that the UCPD did not preclude the provision, as it concerned the protection of public health and the dignity of the profession of dentist. However, the provision—which imposes a general and absolute prohibition of any advertising relating to the provision of oral and dental care services, inasmuch as it prohibits any form of electronic commercial communications, including by means of a website created by a dentist—was precluded by Article 8(1) of the E-Commerce Directive.⁵⁰

Data-driven business models at the predatory end of a benign–predatory continuum may infringe not only on the economic interests of consumers but also on the interests relating to health and safety. In addition, the more or less unintended and/or unanticipated consequences of these business models include negative effects on social coherence and the democratic debate. Article 38 of the Charter (consumer protection) explicitly mentions ‘the health, safety and economic interests of consumers’. However, the Product Liability Directive⁵¹ does not apply to services or situations where information which, by its nature, constitutes a service, is incorporated into a physical item.⁵²

2.3. Loneliness and social relations

Being alone is stressful, and ‘lonely people tend to have more of just about every kind of mental and physical illness than people who live in rich social networks’.⁵³ In terms of health, loneliness is worse than not exercising, as harmful as being an alcoholic and twice as harmful as being obese.⁵⁴ Human beings need close relationships to thrive, but without time, attention and empathy, relationships will not thrive.⁵⁵ Facebook’s own research show that about 12.5% of its users report ‘engaging in compulsive use of social media that impacts their sleep, work, parenting or relationships’.⁵⁶

Advertisements work best on lonely individuals.⁵⁷ Thus, optimising for profits may entail real-life consequences for human health, and amplify the trend over past decades in the direction of more loneliness and more shallow interactions between human beings.⁵⁸

Social influence and competitive behaviour ‘stem directly from the drive for self evaluation and the necessity for such evaluation being based on comparison with other persons’.⁵⁹ In terms of herding, behaviours seem ‘normal’ because other people are doing them,⁶⁰ which is a good driver for behaviour modification. In this vein, it could be argued that it is an unfair commercial practice to exploit loneliness and friendship in marketing (cf. also Cialdini’s liking principle). As expressed by Shmuel I. Becher and Sarah Dadush:

‘By encouraging consumers to behave emotionally, relationship as product lowers consumers’ defenses. It encourages consumers to overlook their self-interest and invest more money, attention, and time in buying products and services and interacting with firms. At a societal level, relationship as product can damage trust and decrease well-being.’⁶¹

The idea of ‘relationship as a product’ could be extended to ‘vulnerability as a product’ in behaviour modification, especially when AI is used to optimise for the most profitable influence.

Even though smartphones and social media allow for connecting people, their design and our use of these technologies add to the shallowness of social interaction,⁶² which again allows for outrage and polarisation in society, as discussed below.

2.4. Dopamine and addiction

In neuroscience, drugs, gambling, shopping and other addictions are linked to the neurotransmitter dopamine,⁶³ which plays an important role in relation to behaviour, cognition and learning. As discussed in Chapter 7 (persuasive technology), human–computer interaction can be designed to affect moods and produce rewards in the guise of releasing dopamine in the user’s brain.⁶⁴

Our urge for instant gratification coupled with ‘bounded willpower’⁶⁵ has provided fertile ground for a dopamine industry, in which content modification thrives. The addictiveness of a product is directly proportional to the amount and speed of the dopamine release. The development of tolerance is also important for addiction, and the scary part is that tolerance does not seem to decline when the product is not used, i.e. the brain is altered permanently.⁶⁶

The significance of addictions to, e.g., tobacco, alcohol, drugs and gambling, is that the user’s ‘real tastes have been taken over by a special monkey on his shoulder’;⁶⁷ and there is in general extensive regulation in place for those addictive products. Persuasive technology can be designed to dispense dopamine automatically, at a distance and at scale, and thus create addictions.

Self-binding strategies—such as alcoholics not having alcohol in the house—are important to cope with addictions. However, smartphones and social media are important for social and societal interactions, and it may thus be difficult to develop and implement effective self-binding strategies and tactics for digital addictions.⁶⁸

The Zeigarnik effect—which describes the fact that incomplete experiences occupy our minds far more than completed ones⁶⁹—is another way of explaining addictive behaviour. For instance, a posting on social media services is likely to keep the activity and service more readily present in your brain, and thus draws on cognitive resources. The tension is relieved when a task is completed, but many tasks online are often open-ended.

The ping from or presence of a smartphone may prompt the user to check in with technology.⁷⁰ The term in neuroscience is ‘cue-dependent learning’ or ‘classical (Pavlovian) conditioning’.⁷¹ It is found that anxious and fearful users check in more frequently.⁷² Finally, Cialdini’s reciprocity principle is also likely to create a drive to repay likes and accept invitations to connect.⁷³

To the extent that persuasive design is used to create addiction-like behaviour, it may lead to anxiety and stress, which even further deplete willpower and diminish the consumer’s ability to control emotions⁷⁴—an ability that is important for effective decision-making. This creates a vicious cycle in which the value extraction becomes even more efficient.

We may rationalise addictive behaviour by means of preference adaptation and herding to avoid cognitive dissonance. Our appetite for distractions seems infinite,⁷⁵ and some economists may say that our ‘revealed preferences’ signal that we are in love with our technologies, considering the amount of screen time and how often users check their phones. Or as Tim Wu asks:⁷⁶

‘What are the costs to a society of an entire population conditioned to spend so much of their waking lives not in concentration and focus but rather in fragmentary awareness and subject to constant interruption?’

2.5. Cognitive overload

Despite their efficiency, our brains have limitations. It is suggested that our working memory can typically hold seven (±two) pieces of information.⁷⁷ And the ‘Dunbar number’ suggests—based on a correlation between brain size of primates and average social group size—that human beings cannot maintain stable social relationships with more than 150 people.⁷⁸ For closer friends and family, the number is around 15.⁷⁹

The idea of a stressful world is not new: just consider Sigmund Freud’s account from the early twentieth century:

‘The enormous expansion of communications, due to the world-wide telegraph and telephone networks, has entirely transformed the conditions of trade and commerce. Everything is done in haste, at fever pitch. The night is used for travel, the day for business; even “holiday trips” put a strain on the nervous system. Great political, industrial and financial crises carry this excitement into far wider areas of the population than ever before. […] people are forced to engage in constant mental activity and robbed of the time they need for relaxation, sleep and rest […] Big-city life has become increasingly sophisticated and restless. The exhausted nerves seek recuperation in increased stimulation, in highly-spiced pleasures, and the result is even greater exhaustion.’⁸⁰

With complexity comes a broader range of probabilities,⁸¹ and technology augments our abilities to manage information and engage with people. Computers make things look easy, so we don’t have to think.⁸² And that may in fact threaten our antifragile nature and challenge both agency and human dignity. Complexity may lead to ego depletion and even resignation,⁸³ which makes defaults and nudges more effective—and important.

In a complex and stressful world with widely scattered attention,⁸⁴ we may lose ‘our means and ability to go beneath the surface, to think deeply’.⁸⁵ The ability to skim text may be as important as the ability to read deeply, but it seems ‘that skimming is becoming our dominant mode of reading’.⁸⁶ When we outsource our memory and thinking,⁸⁷ we also outsource important parts of our intellect and identity, and there is the risk that the internet becomes ‘a replacement for, rather than just a supplement to, personal memory’.⁸⁸ As expressed by Robert B. Cialdini:

‘I have become impressed by evidence indicating that the form and pace of modern life is not allowing us to make fully thoughtful decisions, even on many personally relevant topics. Sometimes the issues may be so complicated; the time so tight, the distractions so intrusive, the emotional arousal so strong, or the mental fatigue so deep that we are in no cognitive condition to operate mindfully. Important topic or not, we have to take the shortcut.’⁸⁹

A study shows that the mere presence of smartphones reduces available cognitive capacity, and that there is a direct correlation between these cognitive costs and smartphone dependence.⁹⁰

3. Privacy

We introduced privacy in Chapter 4 (data protection law) and emphasised that privacy plays an important role in both democracy and individual autonomy, where a balance must be struck between solitude and companionship:⁹¹ both are necessary for developing democracy and individual agency.

The right to privacy includes protection from surveillance and is not limited to the processing of personal data. It does not matter whether the information is sensitive or whether the persons surveilled have been inconvenienced in any way.⁹² In our context, this is relevant to the surveillance carried out in the context of data-driven business models. Under the heading ‘respect for private and family life’, Article 7 of the Charter provides that:

‘Everyone has the right to respect for his or her private and family life, home and communications.’

In contrast to the protection of personal data (Article 8), Article 7 does not explicitly mention consent as a means of interfering with this right. Thus, interference must be justified under Article 52(1) of the Charter, as we discuss below. The provision corresponds to Article 8 ECHR,⁹³ which in subsection 2 provides for the following legitimate limitations:

‘There shall be no interference by a public authority with the exercise of this right except such as is in accordance with the law and is necessary in a democratic society in the interests of national security, public safety or the economic well-being of the country, for the prevention of disorder or crime, for the protection of health or morals, or for the protection of the rights and freedoms of others.’

In that vein the ePrivacy Directive is of particular interest, as it also pursues this broader concept of privacy.⁹⁴ The protection of privacy is also likely to be a relevant factor in the interpretation of both the GDPR⁹⁵ and the UCPD, especially if we perceive technology in a broader perspective, including as a social, cultural and political phenomenon.⁹⁶

The rights enshrined in both Articles 7 and 8 of the Charter ‘must be considered in relation to their function in society’.⁹⁷ In this vein, surveillance may also be understood as infringing on human dignity. For the UCPD, it may only apply if consumers’ economic interests are at stake, which is likely to be the case in the context of data-driven business models.

The protection is clearly not absolute but subject to a proportionality test that requires interferences to be legitimate (in accordance with the law) and proportionate (necessary) to the aim pursued (interest). Measures must ‘remain a genuine derogation to the principle of confidentiality’.⁹⁸ The ‘economic well-being of the country’ could also include the protection of the internal market (European single market) as envisaged in article 3(3) TEU mentioned above.

3.1. The ePrivacy Directive

According to Article 1(1), the ePrivacy Directive harmonises national law to ensure ‘an equivalent level of protection of fundamental rights and freedoms, and in particular the right to privacy, with respect to the processing of personal data in the electronic communication sector’. Thus, the Directive does not apply to offline activities but may be relevant for, e.g. a rewards program that involves the use of a smartphone. Recitals 6 and 7 from this 2002 Directive state:

‘(6)  The Internet is overturning traditional market structures by providing a common, global infrastructure for the delivery of a wide range of electronic communications services. Publicly available electronic communications services over the Internet open new possibilities for users but also new risks for their personal data and privacy.

(7)  In the case of public communications networks, specific legal, regulatory and technical provisions should be made in order to protect fundamental rights and freedoms of natural persons and legitimate interests of legal persons, in particular with regard to the increasing capacity for automated storage and processing of data relating to subscribers and users.’

Member States must ensure the confidentiality of ‘communications and the related traffic data’, in particular by prohibiting third-parties’ ‘listening, tapping, storage or other kinds of interception or surveillance of communications and the related traffic data […]’ (Article 5(1)). The provision covers ‘any operation enabling third parties to become aware of communications and data relating thereto for purposes other than the conveyance of a communication.’⁹⁹

Interference may be justified by means of consent (as in the GDPR) of the users concerned. Another exception is found in Article 15(1), which allows for restrictions, including ‘data retention’, that constitutes ‘a necessary, appropriate and proportionate measure within a democratic society’, for, inter alia, the prevention, investigation, detection and prosecution of criminal offences.

The latter exception is relevant because there is a string of case law that may elucidate the implication of the protection of privacy. The cases concerned data retention laws requiring telecom operators to store and give access to metadata (traffic and location data) for the purposes of fighting serious crime. This traffic data is lawfully processed for the purpose of subscriber billing and interconnection payments.

The judgments—Digital Rights Ireland¹⁰⁰ (April 2014), Tele2 Sverige¹⁰¹ (December 2016), Ministerio Fiscal (October 2018),¹⁰² Privacy International¹⁰³ (October 2020) and La Quadrature du Net and Others¹⁰⁴ (October 2020)—were delivered in Grand Chambers (15 Judges) of the CJEU, which is usually a signal of complexity and/or importance.

Firstly, the CJEU established that the transmission of traffic data and location data is a particularly serious privacy interference, and that the information that can be thus derived, including by profiling, may be ‘no less sensitive than the actual content of communications’.¹⁰⁵ Secondly, ‘only the objective of fighting serious crime is capable of justifying such a measure’,¹⁰⁶ and safeguarding national security is capable of justifying ‘measures entailing more serious interferences’ than other objectives.¹⁰⁷ Thirdly, ‘general and indiscriminate transmission’ exceeds the limits of what is strictly necessary and cannot be considered to be justified within a democratic society.¹⁰⁸

3.2. Consent and transparency in the context of marketing

The above-mentioned cases concerned state interference with privacy, and it is not clear to what extent the findings apply to non-state actors, such as traders relying on data-driven marketing. However, it may seem reasonable to assume that fighting serious crime is more important from a societal perspective than marketing.

The ePrivacy Directive allows for limited use of traffic data for marketing electronic communications services and the provision of value-added services (Article 6(3)), as well as using location data other than traffic data (Article 9). In both cases, such use is dependant on (a) consent, which may be withdrawn at any time, and (b) proportionality, i.e. to the extent necessary for the purposes.

The CJEU has emphasised that users of electronic communications services should be able to expect, at least in principle, anonymity and privacy in electronic communication, ‘unless they have agreed otherwise’.¹⁰⁹ However, communication systems must be ‘designed to limit the amount of personal data necessary to a strict minimum’,¹¹⁰ and interference may not be the rule.¹¹¹

Online, it may seem as though surveillance is the default option, i.e. it requires effort (friction) to stay incognito. Danah Boyd observes that ‘there’s a big difference between being in public and being public’, and that the notion of ‘digital natives’ is ‘an effort to force the global elite to recognize the significance of an emergent mediated society’.¹¹² Similarly, Tim Wu suggests that

‘The most pressing question in our times is not how the attention merchant should conduct business, but where and when.’¹¹³

Further, the CJEU has observed that retaining data without informing the user is ‘likely to cause the persons concerned to feel that their private lives are the subject of constant surveillance’.¹¹⁴ This may, at least for access by ‘public authorities’, have a chilling effect on the freedom of expression.¹¹⁵

Despite consent and transparency, the processing of personal data in the context of data-driven business models may entail the risk of leaving the data subjects with a feeling of ‘constant surveillance’. In addition, the accumulation of large amounts of data is likely to increase the risk of abuse and unlawful access. Thus, it is not unlikely that the protection of privacy will limit the amount of data that can be lawfully processed under the GDPR and affect certain privacy-infringing commercial practices under the UCPD.¹¹⁶

The right to privacy corroborates the argument for keeping personal data from the primary product separate from ancillary uses, such as data-driven marketing. As a matter of transparency, contextual marketing is easier to understand than personalised marketing. Some people, for instance, may believe that anything appearing at the top of a Google search must be true, because ‘why else would it appear at the top?’¹¹⁷

Surveillance may include online behaviour, facial expressions (visual) and oral uttering (audio) as well as proxies for emotions. One of the reasons that information from primary products, such as search, social media and health devices, should not be used for marketing is that we need to be honest to engage with these services;¹¹⁸ the product simply does not work without the user’s being honest. For example, you cannot search for ‘cancer treatment’ on Google without revealing your interest in the subject.

As observed by BJ Fogg, computers have as a distinct advantage that they can go where (a) ‘humans cannot go’ and/or (b) ‘may not be welcome’. The latter, in particular, suggests a privacy focus. Computers’ ability to evoke feelings and be more persistent may also a play a role in understanding computer use in a marketing context.

Despite information and consent, it may seem counter-intuitive that we are under pervasive surveillance, and the framing of the internet as ‘a predominantly commercial enterprise seriously limits the privacy’.¹¹⁹ In connection with social media, it has been noted by the sociologist Zygmunt Bauman that ‘our fear of disclosure has been stifled by the joy of being noticed’.¹²⁰

Tracking, including by means of GPS location, cookies, fingerprinters and spy pixels, must in general be considered more of an infringement on privacy than the use of personal data for personalising advertising. The same is true for monitoring the contents of private conversations in individual texts, e-mail messages, voice-calls, etc.

For the use of personal data processed in these contexts, the question is, in essence, whether traders have ‘the right to judge our fitness for their services based on abstract but statistically predictive criteria not directly related to those services?’¹²¹

4. Non-discrimination

The Charter’s Article 21 concerning non-discrimination is found under the title ‘equality’.¹²² Discrimination is closely linked to human dignity, and in our context also to privacy, as surveillance may provide the data that can be used for discrimination.¹²³

Discrimination may be a natural consequence of personalisation—i.e. treating individuals differentially—but ‘personalisation’ is a nicer framing. Discrimination constitutes another example of possible cross-fertilization between data protection law and consumer protection law. Discrimination can be assessed by considering (1) the parameters, including personal data, used for personalisation, (2) the manifestation of personalisation and (3) the effect that this has on the individual. Article 21(1) of the Charter prohibits

‘any discrimination’ based on ‘any ground such as sex, race, colour, ethnic or social origin, genetic features, language, religion or belief, political or any other opinion, membership of a national minority, property, birth, disability, age or sexual orientation’ (emphasis added).

The risks to the rights and freedoms of natural persons under the GDPR comprise ‘physical, material [and] non-material damage, in particular: where the processing may give rise to discrimination […] or any other significant economic or social disadvantage’ (recital 75 GDPR, emphasis added).

The UCPD focuses on whether the personalisation is likely to materially distort the economic behaviour of the average consumer, regardless of whether personal data are processed. This means, in particular, that material information must not be omitted and that the discrimination must not amount to an aggressive commercial practice, including by means of the trader’s exploiting a position of power vis-à-vis the consumer.

Data-driven business models allow for personalising and segmentation at unprecedented scope and scale. Artificial Intelligence can ultimately be designed to tell different people different things in order to win all of them, and to identify consumers as ‘targets or waste’.¹²⁴ In that vein, it could be noted that poor people are easier to manipulate, constitute a new market opportunity,¹²⁵ and are particularly vulnerable, including with regard to their economic interests.¹²⁶

Personalised marketing may include surveillance of behaviour, targeted advertising, the creation and use of a persuasion profile, and adaptation of terms. The latter includes, in particular, prices that can be adjusted by means of price indications, offers and discounts.

The prohibition concerning discrimination on grounds of nationality is a general principle of European Union law and is, e.g., elaborated on in the Geo-Blocking Regulation.¹²⁷ The Regulation prohibits unjustified geo-blocking and other forms of discrimination based, directly or indirectly, on the user’s (1) nationality, (2) place of residence or (3) place of establishment.

4.1. Price discrimination

The Charter’s prohibition on discrimination must also apply to (unjustified) price discrimination, but there is no specific legal prohibition of price differentiation,¹²⁸ which seems widely accepted, e.g. in the flight industry,¹²⁹ where travellers pay different prices for the exact same service delivered at the exact same time. ‘Personalised discount’ is another and more benign framing of price discrimination.¹³⁰

We must accept that some people are better negotiators or have more time to pursue bargains. However, we may also have intuitive expectations that individuals—at least in principle—all have access to the same information and terms.

It could be argued that—given the less complicated reality when it was adopted—the 1998 Price Indications Directive¹³¹ was intended to establish equality through ‘transparent operation of the market’ (recital 1). In recital 6, the directive states that the obligation to provide price indications

‘contributes substantially to improving consumer information, as this is the easiest way to enable consumers to evaluate and compare the price of products in an optimum manner and hence to make informed choices on the basis of simple comparisons.’

When stores provide the same price information to all visitors, price indications realise the potential for ‘simple comparisons’, a comparison that becomes easier when price information is available online. However, when prices are individualised, an obligation to indicate ‘a’ price does not realise the goal of ‘simple comparisons’ and may hinder ‘transparent operation of the market’.

In this vein, one could argue that—at least under certain circumstances—price differentiation could be contrary to professional diligence with regard to the overarching aims of the Price Indications Directive and the general aim of transparent markets. To the extent that such differentiation undermines the user’s price comparison in the market, such a practice would likely amount to the distortion of the consumer’s economic behaviour. Thus, price differentiation could meet the requirements for being an unfair commercial practice.

One could similarly argue that price discrimination may amount to an aggressive commercial practice; this is corroborated by the fact that enforcement and online comparison become more difficult and that being barred from structured market information affects the economic behaviour of the consumer.

When personal data are used for price differentiation, the GDPR applies. This entails the trader’s consideration of the general principles as well as the legitimate basis for processing. In that vein, the reasonable expectations of the data subject play an important part. In the context of automated individual decision-making, including profiling, it is expressly mentioned that differential pricing based on personal data may have a significant effect if, e.g., ‘prohibitively high prices effectively bar someone from certain goods or services’.¹³² This is a clear example of privacy interferences that affect the economic behaviour of consumers.

Article 20 of the Services Directive¹³³ provides for non-discrimination based on the recipient’s nationality or place of residence ‘without precluding the possibility of providing for differences in the conditions of access where those differences are directly justified by objective criteria’.

The Geo-Blocking Regulation¹³⁴ clarifies certain situations where differential treatment cannot be justified. It is explicitly stated in Article 4(2) that the prohibition ‘shall not prevent traders from offering general conditions of access, including net sale prices, which differ between Member States or within a Member State and which are offered to customers on a specific territory or to specific groups of customers on a non-discriminatory basis’ (emphasis added).

With the New Consumer Deal Directive, an obligation to inform consumers when a price has been personalised on the basis of automated decision-making is inserted in the Consumer Rights Directive.¹³⁵ It follows from recital 45 that the provision is without prejudice to the GDPR, and that:

‘Traders may personalise the price of their offers for specific consumers or specific categories of consumer based on automated decision-making and profiling of consumer behaviour allowing traders to assess the consumer’s purchasing power. Consumers should therefore be clearly informed when the price presented to them is personalised on the basis of automated decision-making, so that they can take into account the potential risks in their purchasing decision. […]’

It is obvious that this information requirement cannot in itself justify price discrimination, which must be assessed under the GDPR, including the provisions on automated individual decision-making, including profiling, the UCPD (professional diligence) and the Charter. The provision provides a shallow transparency, as it will not allow the user to understand the logic involved, its significance and the envisaged consequences, as discussed in Chapter 9 (transparency).

4.2. Targeted advertising

John Wanamaker (1838–1922) is credited with saying that ‘Half the money I spend on advertising is wasted; the trouble is I don’t know which half.’ Thus, from the trader’s perspective, it may make a lot of sense to target advertising and measure its effects.

It is unclear to what extent the rules on automated individual decision-making, including profiling, apply to targeted advertising. Personalised marketing could in principle amount to discrimination, depending on the qualifying criteria, but it seems intuitively justified to e.g. direct marketing for dresses towards women rather than men.¹³⁶ In a joint opinion, the EDPB and EDPS have suggested that the future AI Regulation should prohibit any type of social scoring.¹³⁷

Using individual profiles to target advertising may—to some extent—be considered an aggressive commercial practice, as it inherently involves some kind of discrimination.¹³⁸ It could similarly be considered exercise of undue influence—exploiting a position of power as dealt with in Chapter 8 (manipulation)—where account should be taken of inter alia timing, location, nature and persistence (Article 9(1)(a) UCPD).

One way to create transparency in the context of targeted advertising is for the trader to be explicit about why a particular advertisement has been served. Facebook has adopted such a tool, which in a particular case made it clear that advertising for payday loans was targeted at people with an interest in gambling. The Danish Consumer­ombudsman found it to be an unfair commercial practice,¹³⁹ and it is difficult to see that serving such advertising to this particular (vulnerable) group should be necessary for the purposes of marketing and the performance of the contract entered into between Facebook and the user.

5. The freedom to conduct a business

Article 16 of the Charter recognises ‘the freedom to conduct a business in accordance with Union law and national laws and practices’, which entails the freedom to engage in an economic or commercial activity without ‘disproportionate restraints’.¹⁴⁰ The freedom to conduct a business is expressly mentioned in the proposed Digital Services Act,¹⁴¹ which aims, inter alia, to ‘mitigate discriminatory risks [… and contribute to …] the right to human dignity online’ and observes that ‘the costs incurred on businesses are offset by reducing fragmentation across the internal market’.

5.1. Proportionality

The freedom to conduct a business is not an absolute right, as it must (‘of course’) be exercised with respect for Community law and national legislation.¹⁴² Thus, this right must be considered in relation to its social function and restrictions, which must (a) correspond to objectives of general interest pursued by the European Union and (b) not constitute, with regard to the aim pursued, a ‘disproportionate and intolerable interference, impairing the very substance of those rights’.¹⁴³

As follows from Article 52(1) of the Charter, limitations to rights and freedoms must (1) be provided for by law, (2) respect the essence of the rights and freedoms and (3) be necessary (subject to proportionality) and genuinely meet objectives of general interest, including the rights and freedoms of others.¹⁴⁴

The right does not imply ‘any right to engage in any activity or to perform any act aimed at the destruction of any of the rights and freedoms recognised in this Charter or at their limitation to a greater extent than is provided for herein’.¹⁴⁵ The freedom to conduct a business must be reconciled with the requisites for protecting other fundamental rights, including aspects of consumer protection;¹⁴⁶ privacy, including personal data;¹⁴⁷ and health,¹⁴⁸ in order to strike a ‘fair balance between the various fundamental rights protected by the Community legal order’.¹⁴⁹ Thus, when claiming the right to conduct a business, this right must be understood with respect to EU and national law and be weighed against possible infringement of, inter alia, human dignity, privacy and non-discrimination.

When transposing and interpreting directives, such as the UCPD, Member States must allow for ‘a fair balance to be struck between the various fundamental rights’ and interpret their national law in a manner consistent with those directives, fundamental rights and other general principles, such as the principle of proportionality.¹⁵⁰

The CJEU has recognised that funding requirements go hand-in-hand with the freedom to conduct a business, but also that this freedom must be balanced against, inter alia, ‘the protection of the interest of consumers’ (as television viewers) and ‘excessive advertising’.¹⁵¹ In that vein, it must be considered whether the restraint affects the actual substance of the freedom to choose an occupation, for instance, by going beyond merely controlling marketing to prohibit the offering and marketing of products.¹⁵²

The freedom of expression and information—envisaged in Article 11 of the Charter—apply to commercial communication¹⁵³ and may corroborate the right to conduct a business in this context, but the trader’s freedom of commercial expression is not likely to carry much weight compared to the protection of human dignity and democracy.

The freedom to conduct a business does not extend to protecting ‘mere commercial interests or opportunities, the uncertainties of which are part of the very essence of economic activity’,¹⁵⁴ but it does recognise the freedom of contract,¹⁵⁵ which in this context is closely related to the consumers’ agency and right to self-determination.

In determining ethical issues with persuasive technology, BJ Fogg has identified three focal points: (a) the intentions of the trader, (b) the methods used to persuade and (c) the outcomes of using the technology, and he suggests as a first step ‘to take technology out of the picture’, and ‘simply ask yourself, “If a human were using this strategy to persuade me would it be ethical?”’¹⁵⁶ This is similar to applying the IT law dogma of functional equivalence (non-discrimination between online/offline activities).

As a ‘reality check’, it could be useful to imagine sales personnel in the real world following and observing consumers every hour of the day—across stores, at work, in schools, in bars, at home, etc.—in order to meticulously observe, analyse and record behaviours and emotions. This could include occasionally interacting with the consumer to observe and record responses. Based on these observations and interactions, a detailed profile could be developed with a view to offering the right products and terms, at the right time and in the most persuasive manner.

To justify interference with the right to conduct a business, it is obviously necessary to consider other interests, including those pertaining to human dignity and democracy, and to consider whether those objectives can be attained by less restrictive measures.¹⁵⁷ In this equation the possibility of consumer empowerment is important as well as whether there are meaningful, believable alternatives to the business model.

5.2. Agency and the right to self-determination

We have discussed above a concept of freedom in the context of consumers’ having a right to self-determination and being free from paternalism (in both narrow and broad senses).

It is easy to assume that more choices equals more freedom and well-being, but the opposite may be true, as providing people with more choice may paralyse them rather than liberating them.¹⁵⁸ This also relates to the scarcity of attention¹⁵⁹ and the cost of choices.¹⁶⁰ These constraints on decision-making do not refute Adam Smith’s idea that individual freedom of choice provides for the most efficient production and distribution of society’s goods,¹⁶¹ but it may be an argument for market intervention that will empower consumers to comply with their obligations in the market.

The argument is not to limit the number of products offered, but to make it easier for the consumer to actually take informed decisions by enhancing transparency. This can be achieved by both disclosure of information and removing practices that add unnecessary friction to the decision process. As observed by Yuval Noah Harari, censorship in the twenty-first century ‘works by flooding people with irrelevant information’; and therefore having power today means ‘knowing what to ignore’.¹⁶² Or as Seth Godin puts it:

‘Marketing is now so powerful that caveat emptor is no longer a valid defence.’¹⁶³

From the dawn of EU consumer policy, the focus has been on enabling ‘consumers, as far as possible, to make better use of their resources, to have a freer choice between the various products or services offered’. One of the main priorities has been to ensure protection against ‘forms of advertising which encroach on the individual freedom of consumers’.¹⁶⁴ Similarly, the European Commission emphasised in its 2020 consumer agenda the need to tackle ‘commercial practices that disregard consumers’ right to make an informed choice, abuse their behavioural biases, or distort their decision-making processes’.¹⁶⁵

Freedom of contract and the free price system are essential elements of the market economy . . . and so are privacy and agency. In essence, the regulation of data-driven business models from the perspective of marketing law and data protection law is a matter of (a) how much the individual/average user is expected to understand and (b) the limit for what users should be legally allowed to accept.

The user’s ability to understand the deal is a function of inter alia (1) the complexity, (2) the envisaged consequences (impact), (3) the user’s reasonable expectations,¹⁶⁶ and (4) the trader’s effort to establish genuine transparency, including (5) by sufficiently engaging or appealing to the user’s capacity for reflection and deliberation. Or as asked (rhetorically) by Eric K. Clemons:

‘Can consumers really judge the benefits and rewards, costs and risks, of using a platform? Would warnings on Internet software sites be sufficient, the way warning labels are supposed to be sufficient in limiting tobacco smoking?’¹⁶⁷

In that vein, the CJEU has recognised that the imbalance of information and expertise between consumer and trader may be more pronounced in ‘a sector as technical as the telecommunications services sector’.¹⁶⁸

In the context of consent, information about and the ability to refuse or withdraw consent without detriment is an important part of understanding the deal. In that vein, it must be clear that from a fundamental rights’ perspective, paying in money is profoundly different from ‘paying’ with human dignity, privacy and agency.

Both marketing law and privacy law (including the processing of personal data) establish a threshold for legitimate interference with privacy for the purposes of marketing. The principle of data minimisation entails that data must be ‘adequate, relevant and limited to what is necessary in relation to the purposes’;¹⁶⁹ and the accuracy principle could be used as an argument for limiting the inference and use of data of a probabilistic nature.¹⁷⁰

An important question to ask concerns the extent to which consumers—in the name of their right to self-determination—should be free to choose to relinquish their rights to human dignity, privacy and non-discrimination as consideration for using a commercial service.

5.3. Meaningful, believable alternatives

Charging admission is the obvious alternative to the re-selling of human attention,¹⁷¹ which include (a) contextual marketing and (b) personalised marketing, the latter being the more complex and intrusive version. DuckDuckGo is a privacy-focused alternative to Google Search that has been profitable since 2014.¹⁷² Similar competitors may be envisaged in the area of social media but due to network effects, market penetration is much more difficult: For search, you only need good search results; for social media, you need your connections.¹⁷³

Contextual advertising might work better for search than for social media, as the search queries in themselves indicate an interest in finding something and provide a good idea of what the user is looking for. Similarly, social media could be designed to let the user look up information, which would provide a better base for contextual advertising than the frictionless algorithmically mediated newsfeed.

5.4. The cost of ‘free’

There is good reason to believe that ‘free’ is an attractive—and even inevitable—price point,¹⁷⁴ but if this low price is reached by undermining human dignity and democracy,¹⁷⁵ ‘free’ may in fact be very expensive when externalities are also included in the calculations. Harm from these externalities may be felt on personal, social and societal levels. As expressed by Jamie Bartlett:

‘We need to be aware that cheap or free services have invisible costs: whether that’s your own rights or those of the workers who are employed by the companies that run them.’¹⁷⁶

From a consumer perspective, the cost of ‘free’ is much more difficult to calculate and compare when the consumer must make a prognosis for the impact of behaviour modification on pursuing his goals, values and preferences. As mentioned by George A. Akerlof and Robert J. Shiller:¹⁷⁷

‘Markets with externalities and manipulations do not work perfectly, even though people are free to choose, and there is a potential role for government.’

When we pay with attention, the price is ‘all the things you could have attended to, but didn’t: all the goals you didn’t pursue, all the actions you didn’t take, and all the possible yous you could have been, had you attended to those other things’.¹⁷⁸ When we are exposed to manipulation, we pay with agency (human dignity), which is linked to the opportunity cost of attention, but is more intrusive.

When products are offered free of charge, friction is removed, which means less appeal to our capacity for reflection and deliberation than if the consumer were to make a payment, as discussed in Chapter 8 (manipulation). Payment also have the advantage that it sends an implicit signal that the user is operating in the financial/commercial domain, not the social domain.¹⁷⁹

If the price point is ‘free’, competitors may have a hard time competing at a higher price point.¹⁸⁰ The competition may be more likely to revolve around the effectiveness of extracting value from surveillance and manipulation, while keeping users in an apathetic state with regard to their privacy. As mentioned above, it is possible to imagine business models relying on fees and contextual marketing, but this would require that consumers stop trading privacy for convenience and for being noticed,¹⁸¹ and the real question may be whether we have the time to wait yet another decade for a possible market solution.¹⁸²

Add to this that it has been argued that there is no real documentation for the effectiveness of targeted advertising (compared to, for instance, contextual advertising) and that there may be a possible economic bubble pertaining to the re-selling of attention which poses a potential threat to traders, including media, gig-workers, artists and others who rely on advertising (targeted or not) to fund their activities.¹⁸³

It may be impossible to reframe ‘free’ to mean ‘expensive’, so maybe it would make more sense to require that reality fit the current frame.

5.5. Aligned goals

From a market perspective, competition is supposed to benefit consumers. The idea of transparency is supposed to ensure that consumers can pursue their goals, values and preferences. In essence, the market failure pertaining to data-driven marketing rests in divergent objectives of the trader and consumer, and there may be a need for ‘working with the machines, rather than against them and certainly not for them’.¹⁸⁴

For instance, when an online bookstore suggests books based on what the user and others have bought, it is better aligned with the user’s likely preferences than when the same data are used to personalise the marketing of the suggested books to the user’s persuasion profile. Here, the distinction is between what is advertised and how and when it is advertised.

The aim of legislation and interpretation of existing law must be to secure a reasonable balance between these divergent goals, possibly aiming for ‘a high level of consumer protection’ by maximising our capacity for reflection and deliberation, which may require less information and fewer decision. As expressed by James Williams:

‘We can, however, describe the broad outline of our goal: it’s to bring the technologies of our attention onto our side. This means aligning their goals and values with our own.’¹⁸⁵

Where consumer law generally focuses on the (internal) market, the GDPR is ‘intended to contribute to the accomplishment of an area of freedom, security and justice and of an economic union, to economic and social progress, to the strengthening and the convergence of the economies within the internal market, and to the well-being of natural persons’.¹⁸⁶ Therefore, the GDPR may be our best hope for restoring human dignity in data-driven marketing. It actually follows from recital 4 GDPR that ‘the processing of personal data should be designed to serve mankind’.

6. Democracy

We also need to bring these traders onto the side of democracy to keep externalities from data-driven business models from hurting democracy, including its institutions and citizens.

Attention and agency, including our capacity for reflection, deliberation and empathy, are also important in a democratic perspective. The same is true for the infrastructures on which data-driven business models rely, including social media, search and the devices we use to dispense our attention. This is an additional argument for keeping digital media safe with regard to democratic values.¹⁸⁷

From a fundamental rights perspective, we would not accept unjustified infringement upon human dignity, privacy and non-discrimination from a state actor, and there are no compelling arguments for accepting similar interference from a non-state actor. For the user, it does not matter whether infringements come from states or a trader.

That behaviour modification is also readily available for political gains became particularly clear with the Facebook–Cambridge Analytica scandal, affecting the 2016 U.S. presidential election and the 2016 U.K. ‘Brexit’ referendum; and again in January 2021, when ‘big tech’—without any democratic oversight—decided to ‘cancel’ the U.S. president, who had previously benefited from products of theirs that were used in an attempt to ‘cancel’ the presidential election.

Even though technology provides significant benefits to society, it falls within the obligations of democratic oversight to protect citizens from activities that undermine human dignity and democracy. As expressed by Peter Hulsrøj and Marco Aliberti:

‘It is important to stress, however, that choice manipulation is not only commercial. With access to big data, political parties will customise their message to you, and might do so with scant regard to truth. Facebook is showing the way. Religion has not started yet, but big data in the hands of the evangelical movement or the Catholic Church opens fantastic possibilities (together with customized priest avatars).’¹⁸⁸

Autonomy is an important feature of AI systems, and unintended outcomes may cause harm to individuals and societies.¹⁸⁹ Unintended is not the same as unanticipated, and traders have the capacity to test and monitor their AI systems and an obligation to do so under the GDPR. It could be emphasised that ‘risk is in the future, not in the past’.¹⁹⁰ It is in the interest of democracy that market failures do not become personal, social and societal failures. And there are good reasons why we would not like the content of Wikipedia to be personalised¹⁹¹ or leave voting to algorithms,¹⁹² even when they know our goals, values and preferences better than we do ourselves.

There is an overarching growth agenda in the EU, as made clear in the Digital Content Directive,¹⁹³ which in its first recital states that ‘the growth potential of e-commerce in the Union has not yet been fully exploited’, and that ‘[…] making it easier for businesses to supply digital content and digital services, can contribute to boosting the Union’s digital economy and stimulating overall growth’. Growth that is based on undermining the values and aims of the EU Treaties and the Charter is not desirable. As expressed by Jamie Bartlett:

‘It’s clear that these technologies have, on balance, made us more informed, wealthier and, in some ways, happier. After all, technology tends to expand human capabilities, produce new opportunities, and increase productivity. But that doesn’t necessarily mean that they’re good for democracy.’¹⁹⁴

Consumer protection law exists in the context of markets where traders are offering (competing) products that are bought and consumed by individuals. In this part of European economic law, consumer protection is secondary to the goal of efficient markets.¹⁹⁵ Thus, consumer protection should be understood against this backdrop, and legislation is generally limited to situations where it can be justified due to a market failure, i.e. situations in which the allocation of goods and services is not efficient, as when traders’ pursuit of pure self-interest leads to results that are not efficient.

In modern societies, time and agency are some of the resources most precious to human beings, and therefore the freedom to dispose of our time has great importance. At the current stage of humanity, time for the individual is a finite resource. Consumers need time to comply with their obligations in the market, i.e. to inform themselves and make rational choices. Adding information and complexity (‘friction’) in markets will deplete consumers’ mental capacity and divert it from being used for other purposes, including engaging in democracy.

Trading on consumers’ time and attention has implications for the functioning of both democracy and markets. Even though everyone should not participate equally in politics,¹⁹⁶ democracy must be discussed and challenged in order to stay relevant,¹⁹⁷ and in that sense democracy is ‘antifragile’, as it gets stronger with volatility.¹⁹⁸

Empowerment must ensure that users have the capacity to take ownership of attention management in order to pursue their individual goals, values and preferences in both markets and democracy. The internet is an important common good that has been seized by commercial interests in a manner whereby surveillance and behaviour modification is the default.

Algorithmic mediation of information—in a broader sense than commercial marketing—will also affect democratic public debate and political processes,¹⁹⁹ especially when user attention and engagement are optimised for profit rather than informing citizens and public debate. Algorithmic mediation of information is less transparent than traditional curation due to opaque and proprietary (‘black boxed’) algorithms,²⁰⁰ and fragmented abated realities may undermine informed debate²⁰¹—bearing in mind that ‘surveillance is obviously a fundamental means of social control’.²⁰²

Because marketing is generally perceived as relatively harmless, ‘the legal system usually does not attempt to prevent it’.²⁰³ Data-driven marketing is far from harmless, however. For example, as a result of the 2018 changes to Facebook’s algorithms, political parties shifted their content to be more negative to increase the reach of their postings, because the new algorithm rewarded outrage.²⁰⁴

The proposed Artificial Intelligence Act focuses on physical and psychological harm but not economic harm as regulated in the UCPD. Given the wide range of interconnected potential harms from data-driven business models, there may be a need to rethink consumer protection law in order to ensure both horizontal coherence (between legal disciplines and different harms) and vertical coherence (with fundamental rights).

There is undoubtedly a huge societal potential for AI, and it could be argued that society is deprived of important growth possibilities when AI experts use their knowledge and skills to optimise marketing²⁰⁵ in abated realities rather than pursuing aims that further personal, social and societal goals, including challenges relating to social cohesion and climate.

In Chapter 3 (regulating markets) we used a computer’s operating system as a metaphor for constitutions. The reason we did not use hardware is that operating systems can be updated and replaced. Territory, humans and culture are important parts of the societal hardware on which our democracies run.