Renegotiating the relationship between citizens and the development of automated decision-making

We are told that artificial intelligence and the automated decisions it enables will offer us improved public governance. Artificial intelligence will flexibly provide consistent and quick decisions, and considerable savings can be achieved thanks to automation.

According to Finland's national AI strategy, the public sector must be modernized to cope with the pressure caused by aging demographics. Automated decision-making is already used by organizations such as the Tax Administration and Kela, the Social Insurance Institution of Finland.

However, efforts to digitalize public administration appear to overlook the perspective of ordinary citizens. In their study published in Big Data & Society, Associate Professor of Law Riikka Koulu and Postdoctoral Researcher Terhi Esko with her background in sociology combed through all the statements submitted during the drafting of general administrative legislation concerning automated decision-making. They were left with the impression of a legislative initiative carried out on administration's terms.

"We just wanted to see whether the citizen's perspective was present and how they were discussed as targets of automated decision-making. We concluded that the views of citizens were conspicuously absent," says Esko, summarizing the results.

The legislative initiative was based on the EU General Data Protection Regulation as well as the statements of Parliament's Constitutional Law Committee and the opinions of the supreme overseers of legality, according to which the automated procedures that various authorities had been using for several years lacked a legal basis. The researchers perceived a process that effectively aimed to ensure the continued use of previously adopted automated decision-making procedures. Hardly any critical discussion about the legislative initiative was heard.

For their study, the researchers examined the kinds of opinions and future visions of automated decision-making that the statements conveyed. Matters were mainly approached from the authorities' perspective, focusing on the nature of good governance and the benefits sought from digital administration and automation. The answers reflected the notion that citizens were not actually expected to understand the complex technology used in decision-making. Their role is primarily reduced to being the target of decision-making.

The forgotten but trusted citizen

The citizen, who is ignored at the law-drafting stage, is assigned great importance as a quality controller of information systems. Digitalization carries the risk of citizens becoming increasingly responsible, not only for safeguarding their own rights but also for ensuring that the system functions overall.

For example, users are increasingly asked for feedback on different systems and related problems. Many websites now offer a complaint form for reporting shortcomings in the site's accessibility. An easy way to submit a complaint is a positive development as such, but it may also lead to shortcomings not being detected if citizens do not actively report them.

The responsibility for lodging complaints about automated decision-making usually falls on the individuals who are personally affected by the decisions. However, if you do not understand how a decision concerning you has been made, how can you know when to complain about a decision?

"This raises the question about the skills and knowledge that people are assumed to possess," says Esko.

If our legal system works as it should, people complain about decisions that are wrong. This ensures that the materialized risks of digitalization are revealed and that problems can be addressed. However, as information systems become increasingly complex, there is a risk that fewer and fewer citizens complain about the systems and the decisions they make. This will render our legal system toothless.

Studies show that we tend to trust computers—we assume they make correct and objective decisions. This raises the question whether the threshold for complaining about, say, a tax decision will become higher if the decision is made by an algorithm.

Dare to question faith in digitalization

Finns have long held the conviction that we must seek a competitive edge by being innovative. Few have ventured to challenge this narrative or our technology policy.

"What if we didn't digitalize everything? This is a question that doesn't get much attention anywhere. There's no legal policy discussion in Finland about the limits of digitalization," says Koulu.

According to EU statistics, our society ranks first in Europe in the digitalization of public administration. This is naturally a fine achievement, but on the flip side, the one at the front of the pack is also the first one to meet any risks and problems.

"It means we can't copy best practices from other countries. In other words, other countries have not yet come across the problems we face. We're pretty blind to these risks."

Discussions about problems related to the accessibility of systems in digital administration easily give the impression that systems mainly cause challenges to seniors but not to young people, who have grown up with technology. The implication is that problems will be automatically resolved, as people who have been using digital devices since their childhood come to account for a larger share of the population.

However, the researchers caution against counting on young people being digital natives. Technical dexterity alone will not carry very far if the young user does not understand the underlying social system and does not know how to deal with the authorities.

"Digital skills are still treated as something different from civic skills. If you interact with digital administration, knowing how to use a computer isn't enough; you also have to understand how the authorities operate," says Koulu.

Rigid legislation may be a strength

In discussions about innovation and technology policy, Finnish legislation is often labeled as being an outdated impediment creating unnecessary obstacles to development. In these discussions, Estonia's enabling legislation is often cited as an example that Finland should also aim for.

Estonia's highly acclaimed agility is largely based on the country having reformed its legislation when it became independent and getting rid of statutes from the Soviet period.

"We have a constitutional tradition that is hundreds of years old, going back to the time of Swedish rule. The Finnish tradition may also be a protective factor, and I hope it can help challenge the idea that everything must be digitalised," says Koulu.

Finland's and Sweden's shared history and similar administrative structures lay the foundation for a joint Finnish–Swedish project dealing with automated decision-making in public administration.

The ADM-Gov project (The Automated Administration: Governance of ADM in the Public Sector) seeks to determine how automated public decision-making can be implemented without compromising good public administration or citizens' fundamental rights and trust in public institutions. The project also involves comparisons between Finland and Sweden, for example, about the tensions caused by the EU's future AI regulation in both countries. The neighboring countries lend themselves excellently to comparisons: while they have solved many things in the same way, they also exhibit differences.

In Sweden, automated decision-making was made possible in public administration by simply adding a provision to the Administrative Procedure Act stating that decisions can be made automatically. The parallel reform that happened in Finland was much more extensive and included several new sections to acts. The Swedish approach was quickly found to be too simplistic and in need of repair. As part of the project, the researchers hope to find straightforward answers to how well the authorities have adopted Finland's new legislation in practice.

A new branch of research: Legal user interface research

When students need help applying for student financial aid, they no longer navigate to the nearest Kela office but to the Kela website. Our public administration has gone digital, and this means that people no longer interact with a public official but a technical user interface.

User interfaces have become a key element of our daily lives and practically a necessity. The user interface is the part of the information system visible to the user. It can be a chatbot, online portal or any IT solution that citizens use to handle matters with public administration.

Since user interfaces are designed by people, they cannot be neutral. Successful design solutions offer users information about their rights and obligations and provide a well-functioning channel that citizens can use to submit their matters for processing by the authorities. Weaker technical solutions may violate the law and good governance.

"We believe that user interfaces and their design should also be discussed as legal phenomena because they have a direct impact on how people exercise their rights. We're interested in learning what assumptions about users, both citizens and the authorities, are made in user interfaces, and what action they require people to take in order to contact the authorities," says Koulu, describing the bases of their research.

Koulu heads the DARE (Digital Administration Redesigned for Everyone) research project, which explores interaction between technological design and the law. The project applies methods from various fields: legal sciences, social sciences and computer science.

Among other things, the research project aims to determine how well the technical user interfaces used by the authorities really comply with the law. The project also seeks to create a foundation for a new branch of research: legal user interface research.