How Trustpilot achieved a 10x improvement in case resolution time
Trustpilot partnered with Checkstep to bring AI-powered precision to its 100+ person moderation team - cutting case resolution from minutes to seconds while safeguarding the authenticity of hundreds of thousands of reviews a day.
About Trustpilot
Trustpilot is a leading online review platform that enables millions of people worldwide to share their experiences with businesses. Its platform handles hundreds of thousands of new reviews each day, making it one of the most widely recognised and trusted sources for consumer feedback in the world.
To maintain its reputation for authenticity and integrity, Trustpilot has long been a pioneer in the market, setting the gold standard for trusted customer feedback. It invests heavily in moderation to ensure reviews are not only genuine, but also free from harmful or misleading content.
With a large, distributed moderation team and a mission centered on building trust between businesses and consumers, the platform consistently balances scale, accuracy and fairness. At the same time, it demonstrates a forward-looking mindset and a willingness to embrace innovative new technologies that support productivity, efficiency and trust.
How to reduce moderation backlogs?
Trustpilot’s platform processes hundreds of thousands of reviews daily, supported by a team of 100+ moderators, split between both internal and outsourced staff. It wanted to scale moderation with growth, while improving speed and accuracy.
Coupled with its existing workflows relying heavily on Zendesk, case handling was resource-intensive with multiple steps. Average handling times stretched to 3-5 minutes per case, and the team had a significant number of harmful content categories to keep up with, such as spam, advertising violations, hate speech, threats, and other trust and safety issues.
The BRIEF
What Trustpilot was looking for from a third-party vendor
Recognising this operational challenge, as well as the opportunity to lead the industry in adopting Al technology, Trustpilot decided to search for software that would not only solve its current challenges, but also future-proof its moderation strategy.
Speed
Improve average handling time
Flexibility
Adapt to evolving policies across different harm types and regional requirements
Precision
Maintain high precision to minimise appeals and false positives
Cost
Lower the financial cost of moderation
THE SOLUTION
A layered approach to Al-driven content moderation
Trustpilot chose to partner with Checkstep to tackle these challenges head on. The solution focuses on the introduction of a layered approach to Al-driven content moderation.
-
Top of funnel automation
Every incoming review is scanned using automation to flag harmful content at very high precision thresholds, ensuring minimal false positives.
The initial policies covered advertising, spam, gibberish, and later expanded into hate speech, violence, terrorism, and suicide ideation.
-
Policy definition support
Checkstep collaborates closely with the moderation team to clarify and refine policies.
Checkstep's expert team provided concrete examples of where policies needed adapting and pushed back where definitions were too vague.
-
Integration with moderator workflows
Moderators have been onboarded directly into the Checkstep platform to carry out moderation responsibilities.
The intuitive interface, which includes segregated workflows and case management capabilities, enables moderators to make decisions on violations. Moderators can choose to allow, remove and escalate content or suspend a user entirely.
-
Advanced automation with Modbot
Modbot, Checkstep's advanced Al reasoning model, has been deployed to handle decision making for complex scenarios.
Started with the "About another business" policy which aims to identify reviews that are referencing the wrong business. A large number of reviews that previously required manual intervention are now automated. Cascading workflow allows borderline cases to be escalated to advanced Al checks, balancing cost with precision.
-
Continuous improvement
As platform usage increases, so have precision levels. Checkstep now achieves precision comparable to, or often exceeding, human moderators.
Checkstep runs models in shadow mode to benchmark improvements before deployment, ensuring the model is always performing at its best.
More than just technology.
Checkstep's team of Trust & Safety and policy experts enabled Trustpilot to have confidence in the adoption of AI powered-moderation.
Checkstep has brought real policy expertise to the table, not just technology. That's what's made it possible for us to transform our moderation approach with AI - and trust it.
Senior Director Trust & Safety Operations, Trustpilot
THE IMPACT
Making content moderation more accurate, efficient and scalable
Overall, Trustpilot has been able to improve its ability to handle and process hundreds of thousands of reviews daily. Case resolution times have dropped from 3-5 minutes to 20 seconds - a 10x improvement - reducing backlog and helping maintain user trust.
Checkstep's automation not only runs on reported content, but also on detecting reviews that violate guidelines before they’re even posted on the platform, removing a significant portion of the hundreds of thousands of reviews submitted. This maintains platform integrity at high levels and reduces the need for users to report violating or harmful content. This is all done with high accuracy, reducing appeals in the process.
The adoption of Checkstep's Al content moderation platform has cemented Trustpilot’s position as industry leaders, showcasing its commitment to innovation in Trust and Safety.
Want to see our AI content moderation platform for yourself?
Book a demo to see how it can help you deliver safer, more inclusive content at scale.
Checkstep, trusted by global leaders in trust and safety
