From demonstrations to the Dutch childcare benefits scandal, AI and algorithms are everywhere, but how are they being perceived?

Stijns, Konate, Schuilenburg, Brown en Strangio

On 8 and 9 April 2024, Erasmus School of Law organised a symposium where experts from different fields shared their views on the theme of 'AI Experiences'. Currently, much is being discussed about AI's technical and legal aspects, but remarkably little about the human factor. However, when humans and technology are intertwined, looking at how individuals relate to AI and algorithms in practice is highly necessary. "This is the first major conference where more than 40 national and international scientists discussed how individuals relate to AI and algorithms. It revealed how AI tools are experienced in practice, with all the emotions and feelings involved," stated Marc Schuilenburg, Professor of Digital Surveillance at Erasmus School of Law and co-organiser of the symposium. 

The use of AI in surveillance is becoming an increasingly relevant topic in the context of security and in the fight against crime. Think of the use of biometrics, anomaly detection, DNA banking, crowd control, facial recognition, violence detection, 24-hour surveillance and other forms of data-intensive surveillance. These phenomena raise crucial questions about their effects on the human experience. This symposium provided an opportunity to explore various facets of AI surveillance and its implications on public safety. This included not only the technological and legal aspects, but also the emotional, social and cultural aspects that come with the use of AI.  

The use of AI: from protest policing to the childcare benefits scandal 

Publiek AI symposium

The deployment of AI in surveillance practice takes many different forms. Among the topics discussed at the symposium were innovative ways of exploring AI surveillance deployment, the effects of AI surveillance during demonstrations, the use of AI surveillance at EU borders, how society perceives AI, the impact of the childcare benefits scandal, platform surveillance and AI experiences in the public domain. This article highlights two sessions.

Protest policing: a cat-and-mouse game 2.0 

Protest policing is playing an increasing role in EU debates, especially as AI is being used in enforcement and surveillance. This presentation answered questions such as: How do the police prepare for protests, and does this lead to new experiences for protesters and police officers? How does the integration of technologies and AI mediate the relationship between protesters and police? And does this AI-driven approach lead to different dynamics of public demonstrations and the use of social media? 

Continuously seeking the balance 

Majsa Storbeck, PhD researcher at Erasmus School of Law, states: "Protest enforcement seeks to find the right balance between public order and protection of fundamental rights, which includes the right to demonstrate. Adding new surveillance technologies such as AI adds new weight to the scales, and in doing so, a new balance must be found." "The deployment of technology increases the visibility and mobilisation of people, which can result in more demonstrations, demonstrations with larger crowds, and more diverse demonstrations. On the enforcement side, technology improves crowd management, proactive policing and efficiency", Storbeck said. 

But how does this affect the dynamic? 

Storbeck researched the interaction between police and XR protesters and described it as a cat-and-mouse game that has been evolved: "The mouse, in this context the protesters, can deploy technology to change the cat's field of view. Whereas the cat, in this context, the police, with the deployment of technology, has new claws to catch the mouse." However, this could have significant implications for the interaction and effectiveness of protest policing. Where it can have substantial benefits, there also appear to be adverse effects associated with deploying technology and AI. "Think of increased tension and reinforcement of mutual distrust. Moreover, technology blurs the boundaries of when a protest begins and ends. Thus, significant risks may be associated with police deployment of new technologies. Although these tools are primarily used for crowd management and are not necessarily intended to increase police power, they may inadvertently reinforce the perception of an all-powerful entity. I hope the results of my research will contribute to a wider public debate on the use of technology within protest policing, and its effects", Storbeck said.  

"That is why it is essential that, both in parliament and within legislation, more attention is paid to the use of AI and other technologies in demonstrations. Only by carefully considering the consequences and establishing clear rules and regulations can we ensure that the police can use these new technologies in a legally responsible manner. Practice must not take precedence over law", Storbeck stated. 

Affected by the childcare benefits scandal: how does it affect a victim?

Nadine Stijns- Kunstproject Mesmerizing...
Nadine Stijns- Kunstproject Mesmerizing...

During the keynote, visual artist Nadine Stijns expounded the childcare benefits scandal, in which tens of thousands of people were falsely accused of childcare benefit fraud due to the misuse of algorithms. Through the timeline of the benefits scandal and her art depicting the seriousness of this issue, she expressed the impact of the benefits scandal centred around the human experience of algorithms in both words and visuals. "It is important to engage with each other and make the effects of deploying AI and algorithms less abstract; we show who is affected by AI." Then, as moderator, Stijns engaged with Salama Konaté, researcher at the University of Amsterdam, Batya Brown, poet, activist and artist, and Melissa Strangio, filmmaker and protagonist of social change and maker of the documentary Batya, to discuss the seriousness and impact of the childcare benefits scandal. Brown and Strangio also spoke about their experiences as victims of the scandal. 

The impact is significant

The conversation quickly reveals the impact of the childcare benefits scandal. The moment Brown and Strangio were labelled as fraudsters had a big impact on their lives. "As soon as you get a message like that, you start doubting yourself and researching if it is possible that you really made a mistake. You don't assume that the government could be wrong", Strangio stated. But it does not stop at the shock of being labelled a fraud. Proving innocence takes a long time and causes a lot of stress which has a major impact on the lives of those affected and on the lives of their families. Stijns adds: "People lost their homes, relationships were destroyed, and for many, it had a major impact on their health. In some cases, the problems became so severe that children were even removed from their homes." "You are labelled and treated as a fraud and have no money to fight for your rights. Your life is literally taken away from you. I was blacklisted, which means many things become nearly impossible", Brown said. Strangio's story also painfully reveals the long-term impact: "I was always very entrepreneurial and had two successful businesses before the childcare benefits scandal. Eventually, I had to give up my business and work in a call centre to make a living. It really took away my drive to be an entrepreneur. Because why build a business if everything can be taken away from you like that?" 
Misrepresentation of those affected 

Konaté, who wrote her thesis on the childcare benefits scandal, stressed that there is often a completely wrong perception of those who were affected. "In the Netherlands, we have a strong norm of an ideal citizen. Who is hardworking, has their affairs in order and is independent from government reimbursements. The vast majority of those affected by the childcare benefits scandal are definitely not fraudsters but also do not appear to fit this norm completely. Seventy per cent of the group affected appears to consist of people with a migration background, and fifty per cent of them are single-person households, mostly mothers, and socioeconomic status also plays a role in this. And that while these are actually hard-working parents, as you will not receive childcare benefit if you are not working."  

Sprekers AI congres

The damage cannot be expressed in money 

It is almost impossible to express the impact the childcare benefits scandal has had on the lives of those affected in monetary terms. But compensation of 30,000 euros has been promised for those affected with a possible increase if the damages turn out to be higher. "It is now possible to sign a declaration allowing us to get that compensation, but we think it is important to fight for a civil procedure and a fair process", Brown and Strangio state. "After signing the agreement, all doors are closed. I will keep fighting until I am fairly compensated and I get my money back, but there are many parents who cannot take that fight anymore and sign the agreement so this fight can end for them", Brown explained. Compensation for all victims is expected to take until 2030. 

How can we do better in future? 

"In an ideal world, free childcare would obviously be the most beautiful solution. But I understand this is impossible. It is important that the system of mistrust disappears. We need to re-evaluate this policy and make room for dissenting voices", Konaté said. In addition, Strangio stresses the importance of working on building an algorithm that is free of biases. 
"The current proposal by the outgoing cabinet to channel childcare subsidies directly through the childcare organisations can ensure that this situation is not repeated, but only if parents are then also completely removed from the process", Brown states. "We also hope that the world will be a better place and that we can discuss how to resolve and prevent this from happening again. That is why we have set up citizen panels in collaboration with the State Commission on the Rule of Law to discuss what could be improved in our constitutional state. We will present these recommendations to the Lower House in July", Brown said. Konaté finished: "It is essential to keep thinking about the human experience on the other side of a decision, technology or policy document." 

A new perspective 

According to Schuilenburg, the AI story is currently mainly focused on the technical side of it, about the unprecedented possibilities AI offers. Or on the question of what should be understood as 'intelligence' and how 'intelligent' AI tools actually are. "Those are important topics", Schuilenburg argues. "But we should not forget that AI is also always a social practice. Therefore, the experiences of people interacting with AI, from immigrants and police officers to platform workers and protesters of Extinction Rebellion, should be included in talking and thinking about AI." 

If this does not happen, Schuilenburg argues, individuals will be reduced to abstract profiles and thus become invisible and disappear from view. Therefore, putting the AI experiences in the centre of the discussion answers a societal need to give a voice to all those who do not have it in the field of AI or have it insufficiently think especially of 'silenced voices' of weak, vulnerable or marginalised groups. 

"The conference has shown that there is a great need to conduct more research on the AI experiences of citizens and professionals in order to better understand the perspectives and meanings individuals give to AI and algorithms and include them in the design and application of AI tools. We need to make AI 'public'”, Schuilenburg states. 

More information

Click here to read more about the citizen panels. 

Compare @count study programme

  • @title

    • Duration: @duration
Compare study programmes