Expert’s Corner with Head of Research Isabelle Augenstein

This month we were very happy to sit down with one of the brains behind Checkstep who is also a recognized talent among European academics. She is the co-head of research at Checkstep and also an associate professor at the University of Copenhagen.

She currently holds a prestigious DFF Sapere Aude Research Leader fellowship on ‘Learning to Explain Attitudes on Social Media’, and was also recently admitted to the Young Royal Danish Academy of Sciences and Letters.

In this interview she explains how she got involved in the Trust and Safety space and the important role researchers play in finding solutions to all manner of online harms.

  1. What made you get involved in the online Trust and Safety space?

First and foremost, online harms present a substantial societal problem — platforms are rife with abusive language, sexism, misinformation etc. Short of bringing about a cultural change, what one can realistically do to improve the status quo is to semi-automatically moderate harmful content. This, in itself, is very challenging from a technical point of view, which, as a researcher, I find exciting.

2. People tend to trust content that resonates with their beliefs or generally helps address some of their doubts. Thus, making it easy for misinformation propagators to easily target potential “victims”. Given your extensive work around fact-checking, do you think debunking some of these claims is the best way to keep society better informed?

Yes, prior work in psychology shows that it is very difficult to change peoples’ core beliefs and values. Thus, it is important to detect disinformation as early as possible before it spreads, and to provide automatic fact checks to content moderators for this purpose.

3. Better understanding the context of certain conversations helps us to put things into perspective, especially when it comes to social media. Could you tell us a little bit about your research project EXPANSE, that aims to explain attitudes on social media?

EXPANSE is a research leader fellowship I recently obtained (more information here: https://dff.dk/en/grants/copy_of_research-leaders-2020/researchleader-14).The project itself started on October 1, 2021, so there are unfortunately not so many results to share yet. Very briefly about the goals of the project though: the overarching aim is to be able to detect attitudes automatically (also called stance detection), but do so in a much more fine-grained and transparent way than is possible today. The key innovation is to imbue stance detection models with sociological knowledge, as I hypothesize they can shed a light on why people hold certain attitudes, and thus lead to more insightful automatically generated explanations.

4. Content moderation is a growing concern, given the recent infodemic. However, some criticize it as a means to suppress freedom of speech. How should content moderation companies position themselves? What are some of the areas they should focus on, to ensure online safety?

The concept of freedom of speech has existed since the 6th century BC, long before social media or even print media were invented. Before social media, it was much more challenging to spread and weaponize information — whereas now, everyone with access to the internet can do so, anonymously and with few repercussions. This format, by design, brings out the worst in people — things people would never feel comfortable saying to someone’s face, they feel comfortable writing in an anonymous online forum. The filter bubble effect means people additionally receive backing on their opinions from like-minded individuals. This means, in this day and age, one needs to very carefully weigh freedom of speech up against the real harms it can cause. One area I find particularly concerning is the negative impact of social media, especially image-sharing platforms, on depression and suicide among teenagers due to the distorted views of reality presented by many users on these platforms, including related to body image and lifestyle. I think a careful audit of such platforms is needed to address this problem more holistically, but content moderation can at least help to identify particularly harmful information, such as posts glorifying anorexia.

5. How can AI be applied to help with the problem? How does AI explainability help?

AI-based content moderation solutions can help identify harmful content before it even reaches users. Explainable AI can be useful in two ways: one, it can help continually improve content moderation models by identifying why they sometimes make mistakes, and two, especially for knowledge-intensive tasks such as automatic fact checking, they can provide content moderators with more information about why a model arrived at a prediction to make it easier to manually verify if the prediction is reliable.

6. The news is full of how bad actors propagate misinformation and also how platforms seem to exacerbate the problem. What role do you see for academics in addressing these problems?

Academics can provide crucial insights into why this phenomenon occurs, as well as potential solutions to the problem. For misinformation specifically, academics from many different disciplines have important and complementary research findings, which should be taken into account — e.g. from psychology, about the perception of misinformation; from computer science, about how to develop automatic content moderation solutions; from law, about how legislation applies to online platforms in different countries.
Academics can thus help inform online platform developers on how to make platforms safer for everyone, content moderation companies on how to automatically detect harmful content, and, perhaps most crucially, decision makers in governments on how to develop new legislation related to online harms.

PS. Something to look out for –

Isabelle’s higher doctoral dissertation defence, for earning the title of Doctor Scientiarum, on “Towards Explainable Fact Checking” on 6 December: https://di.ku.dk/begivenhedsmappe/begivenheder-2021/doctorate-defence-by-isabelle-augenstein/

More posts like this

We want content moderation to enhance your users’ experience and so they can find their special one more easily.

Fake Dating Pictures: A Comprehensive Guide to Identifying and Managing 

In the world of online dating, fake dating pictures are harmful, as pictures play a crucial role in making a strong first impression. However, not all dating pictures are created equal. There is a growing concern about fake profiles using deceptive or doctored images.  To navigate the online dating landscape successfully, it's important to know
5 minutes

What is Content Moderation? 

Content moderation is the strategic process of evaluating, filtering, and regulating user-generated content on digital ecosystems. It plays a crucial role in fostering a safe and positive user experience by removing or restricting content that violates community guidelines, is harmful, or could offend users. An effective content moderation system is designed to strike a delicate
5 minutes

Transforming Text Moderation with Content Moderation AI

In today's interconnected world, text-based communication has become a fundamental part of our daily lives. However, with the exponential growth of user-generated text content on digital platforms, ensuring a safe and inclusive online environment has become a daunting task. Text moderation plays a critical role in filtering and managing user-generated content to prevent harmful or
4 minutes

Streamline Audio Moderation with the Power of AI

In today's digitally-driven world, audio content has become an integral part of online platforms, ranging from podcasts and audiobooks to user-generated audio clips on social media. With the increasing volume of audio content being generated daily, audio moderation has become a critical aspect of maintaining a safe and positive user experience. Audio moderation involves systematically
4 minutes

It’s Scale or Fail with AI in Video Moderation

In the digital age, video content has become a driving force across online platforms, shaping the way we communicate, entertain, and share experiences. With this exponential growth, content moderation has become a critical aspect of maintaining a safe and inclusive online environment. The sheer volume of user-generated videos poses significant challenges for platforms, necessitating advanced
4 minutes

Enable and Scale AI for Podcast Moderation

The podcasting industry has experienced an explosive growth in recent years, with millions of episodes being published across various platforms every day. As the volume of audio content surges, ensuring a safe and trustworthy podcast environment becomes a paramount concern. Podcast moderation plays a crucial role in filtering and managing podcast episodes to prevent the
4 minutes

Ready or Not, AI Is Coming to Content Moderation

As digital platforms and online communities continue to grow, content moderation becomes increasingly critical to ensure safe and positive user experiences. Manual content moderation by human moderators is effective but often falls short when dealing with the scale and complexity of user-generated content. Ready or not, AI is coming to content moderation operations, revolutionizing the
5 minutes

How to Protect the Mental Health of Content Moderators? 

Content moderation has become an essential aspect of managing online platforms and ensuring a safe user experience. Behind the scenes, content moderators play a crucial role in reviewing user-generated content, filtering out harmful or inappropriate materials, and upholding community guidelines. However, the task of content moderation is not without its challenges, as it exposes moderators
4 minutes

Scaling Content Moderation Through AI Pays Off, No Matter the Investment

In the rapidly evolving digital landscape, user-generated content has become the lifeblood of online platforms, from social media giants to e-commerce websites. With the surge in content creation, content moderation has become a critical aspect of maintaining a safe and reputable online environment. As the volume of user-generated content continues to grow, manual content moderation
4 minutes

Overhaul Image Moderation with the Power of AI

In today's digital world, visual content plays a significant role in online platforms, ranging from social media to e-commerce websites. With the exponential growth of user-generated images, ensuring a safe and inclusive user experience has become a paramount concern for platform owners. However, image moderation poses unique challenges due to the sheer volume, diverse content,
4 minutes

Outsourcing Content Moderation

Outsourcing content moderation has become an essential aspect of managing online platforms in the digital age. With the exponential growth of user-generated content, businesses are faced with the challenge of maintaining a safe and inclusive environment for their users while protecting their brand reputation. To address this, many companies are turning to outsourcing content moderation
4 minutes

Designing for Trust in 2023: How to Create User-Friendly Designs that Keep Users Safe

The Significance of designing for trust in the Digital World In today's digital landscape, building trust with users is essential for operating a business online. Trust is the foundation of successful user interactions and transactions, it is key to encouraging users to share personal information, make purchases, and interact with website content. Without trust, users
5 minutes

Content Moderation: A Comprehensive Guide

Content moderation is a crucial aspect of managing online platforms and communities. It involves the review, filtering, and approval or removal of user-generated content to maintain a safe and engaging environment. To navigate this landscape effectively, it's essential to understand the terminology associated with content moderation. In this article, we'll delve into a comprehensive glossary
7 minutes

How Predators Are Abusing Generative AI

The recent rise of generative AI has revolutionized various industries, including Trust and Safety. However, this technological advancement generates new problems. Predators have found ways to abuse generative AI, using it to carry out horrible acts such as child sex abuse material (CSAM), disinformation, fraud, and extremism. In this article, we will explore how predators
4 minutes

The Future of Dating: Embracing Video to Connect and Thrive

‍In a rapidly evolving digital landscape, dating apps are continually seeking innovative ways to enhance the user experience and foster meaningful connections. One such trend that has gained significant traction is the integration of video chat features. Video has emerged as a powerful tool to add authenticity, connectivity, and fun to the dating process. In
4 minutes

Prevent unwanted content from reaching your platform

Speak to one of our experts and learn about using AI to protect your platform
Talk to an expert