Credit: Pixabay/CC0 Public Domain

Online misinformation is a critical societal threat. While fact-checking plays a role in combating the exponential rise of misinformation, little empirical research has been done on the work practices of professional fact-checkers and fact-checking organizations.

Existing research has covered fact-checking practitioner views, effectiveness of fact checking efforts, and professional and user practices for responding to political claims. While researchers are beginning to investigate challenges to fact-checking, such efforts typically focus on traditional media outlets rather than independent fact-checking organizations (e.g., Politifact). Similarly, such research has not yet investigated the entire misinformation landscape, including the dissemination of the outcomes of fact-checking work.

To address these shortcomings, a team including Nasir Memon of NYU Tandon and Nicholas Micalleff of NYU Abu Dhabi interviewed 21 professional fact-checkers from 19 countries, covering topics drawn from previous research analyzing fact-checking from a journalistic perspective. The interviews focused on gathering information about the fact-checking profession, fact-checking processes and methods, the use of computation tools for fact-checking, and challenges and barriers to fact-checking.

The study, "True or False: Studying the Work Practices of Professional Fact-Checkers," found that most of the fact-checkers felt they have a of correcting harmful information to provide "a service to the public," emphasizing that they want the outcome of their work to both educate and inform the public. Some fact-checkers mentioned that they hope to contribute to an information ecosystem providing a "balanced battlefield" for the discussion of an issue, particularly during elections.

The interviews also revealed that the fact-checking process involves first selecting a claim, contextualizing and analyzing it, consulting data and domain experts, writing up the results and deciding on a rating, and disseminating the report.

Fact-checkers encounter several challenges in achieving their desired impact because current fact-checking work practices are largely manual, ad-hoc, and limited in scale, scope, and reach. As a result, the rate at which misinformation can be fact-checked is much slower than the speed at which it is generated. The research points out the need for unified and collaborative computational tools that empower the human fact-checker in the loop by supporting the entire pipeline of fact-checking work practices from claim selection to outcome dissemination. Such tools could help narrow the gap between generation and fact-check dissemination by improving the effectiveness, efficiency, and scale of work and dissemination of its outcomes.

More information: Nicholas Micallef et al, True or False: Studying the Work Practices of Professional Fact-Checkers, Proceedings of the ACM on Human-Computer Interaction (2022). DOI: 10.1145/3512974