In BrightLocal's 2026 survey of 1,002 US consumers about local business reviews, 63% said review platforms should be responsible for detecting and preventing fake reviews. User generated content moderation is the work of reviewing what users post, such as reviews, listings, comments, and uploads, against a platform's policies and the law, then acting on the items that break them.
User-generated content (UGC) refers to media, such as photos, videos, reviews, or text, that a platform's users create or share online. When people talk about it, they often cite influencers showing off shopping hauls on social media. It can also be a fundraiser someone created, a review of a shop on a review site, or a listing on eBay.
Because anyone can create and post UGC, content moderation is needed to prevent malicious users from abusing it.
User generated content moderation at a glance
- User-generated content comes from people who want to share something they love or discuss its quality. Customers, brand loyalists, and even employees create it.
- Most platforms make UGC easy by letting users leave reviews, add product photos, create listings, and more.
- UGC is powerful because it comes from real people; however, malicious users can exploit a UGC campaign by spamming it to farm engagement or hijacking it to cause harm.
- The EU's Digital Services Act (DSA) and the UK's Online Safety Act (OSA) both require platforms that host user-generated content to act on illegal content, and the OSA also requires every in-scope service to assess its risks.
Why does user-generated content need moderation?
Platforms submitted about 4.3 billion content moderation decisions to the EU's DSA Transparency Database in the six months before October 6, 2026, according to the database's own counter. Bad actors can fake user-generated content or use trends and events to spread harm or manipulate people.
Review platforms show how much rides on this. Checkstep's case study reports hundreds of thousands of reviews arriving on Trustpilot every day. A moderation team of at least 100 people, some in-house and some outsourced, works through policies covering spam, advertising, hate speech, threats, and similar harms.
After Trustpilot moved that work onto Checkstep's moderation platform, with automation screening each new review, the average time to resolve a case fell from 3 to 5 minutes to 20 seconds. That's a 10x improvement by the case study's measure.
What are the main UGC-specific risks?
While UGC is a powerful force, it also leaves platforms vulnerable to five main risks: fake reviews and fake profiles, campaign hijacking and spam, user-uploaded child sexual abuse material (CSAM), harassment and doxxing, and intellectual property (IP) and copyright exposure.
Fake reviews and fake profiles
A very common issue for UGC platforms is fake reviews and fake profiles. Trustpilot defines a fake review as one that doesn't reflect a genuine service or buying experience with a business. Fake reviews aim to mislead readers about a business, in its favor or against it.
Fake profiles are something the average user knows more about: accounts on dating apps and elsewhere that misrepresent who is behind them, whether through an invented persona or a stolen identity.
Campaign hijacking and spam
A well-known example is Coca-Cola's "Make It Happy" hashtag campaign. In February 2015, Coca-Cola invited Twitter users to reply to negative tweets with #MakeItHappy, and its automated bot turned those words into cute ASCII art.
Adam Pash of the media site Gawker built a counter-bot called @MeinCoke that fed the campaign lines from Hitler's Mein Kampf. Coca-Cola's automated account unintentionally tweeted those lines as ASCII art for a couple of hours before the campaign ended.
User-uploaded CSAM
User-uploaded CSAM is child sexual abuse material, meaning sexual images or videos of anyone under 18, that a user uploads to an online service or cloud storage, whether deliberately or by accident. Generative AI also lets abusers manipulate ordinary images of children, including family photos shared online, into sexual abuse material.
Harassment and doxxing
Malicious users can use platforms to harass or dox other users. Doxxing is publishing an individual's private or personally identifying information without consent to intimidate them or expose them to harm.
Harassment is repeated, unwanted contact meant to alarm or distress someone, often through a stream of tags and messages. Platforms can identify this using AI or human moderation, or users can report it.
IP and copyright exposure
By allowing UGC, platforms can face severe IP and copyright risks if users upload protected material without authorization. This involves posting unlicensed music, videos, photos, or text. Safe harbors, such as the one in the US Digital Millennium Copyright Act, limit that risk for platforms that meet their conditions.
Additionally, unauthorized commercial use of brand logos or names, or of a person's name or likeness, can infringe trademark or publicity rights.
How is UGC moderation applied across content formats?
UGC moderation is applied to text, images, video, and audio, using different AI techniques for each format to detect harmful content before it reaches users. Text moderation uses natural language processing to catch hate speech, harassment, spam, and scams in comments, posts, and chat. It weighs context and intent, with keyword lists as one supporting layer.
Image moderation applies computer vision to flag nudity, violence, weapons, and extremist symbols. It uses OCR to read text embedded in memes and screenshots.
Video moderation combines both approaches by sampling frames, analyzing captions, and transcribing speech. Because every minute of footage multiplies that work, scale is the central challenge for platforms handling long or live uploads.
Audio moderation transcribes speech and analyzes tone to surface abuse and threats in voice chat, podcasts, and livestreams.
Across all four formats, AI handles the volume, and trained human reviewers provide the oversight, judging the cases that need context and deciding appeals.
What do the DSA and Online Safety Act require for UGC?
The EU and UK generally share the same goal for UGC moderation. Both laws require platforms to act on illegal content, and they differ on risk assessment: the DSA requires it only of the largest platforms, and the OSA of every service it covers.
The EU Digital Services Act for UGC platforms
The Digital Services Act applies to any service that stores or shares user content for people in the EU, wherever the provider is based (Article 2). Its obligations are tiered:
- All UGC platforms must let users flag illegal content and explain their decisions. Platforms need clear notice-and-action mechanisms, and must give affected users a statement of reasons when content is restricted or removed (Articles 16 and 17).
- Online platforms, apart from micro and small enterprises, must be open about recommendations and handle complaints. They must explain the main parameters of their recommender systems in their terms and conditions (Article 27) and run an internal complaint-handling system so users can challenge moderation decisions (Article 20).
- Very Large Online Platforms and Search Engines carry the heaviest load. Services the European Commission designates as having 45 million or more average monthly active users in the EU must assess systemic risks and undergo independent audits at least once a year (Articles 33 to 37).
- The largest platforms must also give users more control over their feeds. They must offer at least one recommender option that isn't based on profiling (Article 38).
- They must be ready to act during crises and elections. During a crisis, the Commission can require them to apply response measures of their own choosing (Article 36), and they must assess and reduce election-related risks (Articles 34(1)(c) and 35), guided by the Commission's 2024 election guidelines.
The UK Online Safety Act for UGC platforms
The Online Safety Act covers user-to-user services and search engines with links to the UK, and is enforced by Ofcom, the UK's communications regulator.
- Every in-scope platform must assess its illegal content risks (section 9). This means looking at how priority harms set out in the Act, such as child sexual exploitation and abuse, terrorism, and fraud, could appear on the service.
- Services likely to be accessed by children must also assess risks to minors (section 11). They need to consider how children could encounter harmful content, including content that encourages or promotes self-harm, and put protections in place accordingly (section 12).
- Platforms must deal with illegal content quickly. Platforms need proportionate systems to identify and remove illegal content, scaled to their size and risk level (section 10).
- Users need clear ways to report, complain, and understand the rules. Platforms must offer easy-to-use reporting and complaints routes (sections 20 and 21) and clearly set out their terms of service (section 10).
- The biggest services face additional duties. Ofcom sorts services that meet thresholds set in regulations into Category 1, 2A, and 2B, with categorized services taking on extra obligations such as annual transparency reports (section 77). Category 1 services must also provide user empowerment tools that give adults more control over the content they see (section 15).
The UGC risk-to-duty map
Both the DSA and OSA ask the same question: what could go wrong on your platform, and what have you done about it? A risk-to-duty map lines the two up side by side.
Illegal content (CSAM, terrorism, hate speech, fraud) triggers notice-and-action under the DSA (Article 16) and takedown duties under the OSA (section 10). The DSA adds reporting suspected offenses threatening life or safety (Article 18), and the OSA adds reporting detected child sexual abuse content to the National Crime Agency (section 66).
Risks to minors trigger the DSA's minor-protection duty for online platforms minors can access, including a ban on profiling-based ads (Article 28). Under the OSA, services likely to be accessed by children need age assurance plus content-exposure controls (section 12).
Systemic risks (election interference, mental health harms) require annual risk assessments and audits for the largest platforms under the DSA (Articles 34 and 37). The OSA requires risk assessments for every in-scope service plus transparency reports for categorized services (sections 9 and 77).
Mapping risks this way turns two overlapping rulebooks into one workable compliance program.
How does UGC moderation differ by platform?
Moderation looks different depending on what a platform does. Social networks deal in high-volume, fast-moving text, images, and video, where speed and recommender-system risk dominate. Marketplaces face fraud, counterfeit goods, and scam listings, so identity verification and product-level review matter more than free-speech nuance.
Video and livestreaming platforms need real-time detection for graphic content and CSAM, with human moderators on hand while streams are live. Gaming and social apps with in-game chat or voice must moderate live, ephemeral interactions, frequently involving minors, which raises the stakes on age assurance.
Forums and niche communities rely heavily on volunteer community moderators, who also lean on automated tools, so platforms must give them tools, escalation paths, and support without losing context. The core duties apply across platforms; the practical response has to match the platform.
UGC moderation best practices
Good UGC moderation isn't one tool; it's automation and human judgment working together, backed by processes that hold up under scrutiny. None of this works as a one-off project. The practices below scale to each platform's volume and risk.
- Automated detection handles high-volume harms well: hash matching catches known CSAM and terrorist content, and classifiers catch spam and new material at scale.
- Human reviewers are still needed for anything that depends on context, like harassment or misinformation, where the wrong call is easy to make without understanding intent.
- Community guidelines need to be clear, public, and kept current, since regulators expect risk assessments, and the policies built on them, to keep pace with new harms.
- Users need a fast way to report content and a real path to appeal that reaches a person.
- Log every decision, risk assessment, and escalation, because regulators can ask to see those records.
- Moderators also need support, as the work involves repeated exposure to disturbing material, which peer-reviewed studies link to psychological distress.
- Track accuracy, turnaround time, and how often appeals get overturned, then feed that back into retraining and policy updates.
How Checkstep helps
Checkstep’s AI content moderation platform gives online services control over how they detect and moderate content. Rather than locking into a single AI model, Checkstep provides access to a marketplace of AI models, so platforms can choose and configure the models that fit their policies and content.
Online platforms can also test how those models perform on their own content, then track decision accuracy and quality-assurance results over time. That combination of model choice and measurable performance helps Trust and Safety teams feel confident their AI moderation is working.
Frequently asked questions about moderating UGC
What counts as user-generated content under the DSA and OSA?
User-generated content under the DSA and OSA can be text posts, images, video, livestreams, reviews, comments, forum threads, and even usernames or profile photos. If users can upload, post, or share something that other people might see, it's likely in scope.
The OSA exempts email and comments or reviews on a platform's own content, and the DSA's online platform rules exclude private messaging.
Do small platforms need to moderate UGC?
Small platforms do need to moderate user-generated content, but the bar is lower. Smaller services still need a way for users to report illegal content under both laws. Under the UK Online Safety Act, they must also run risk assessments and keep records of them.
The heaviest duties, like independent audits for the largest platforms under the DSA and transparency reports for categorized services under the OSA, kick in only once you cross certain size thresholds.
Whether a small platform builds or buys its moderation tools, the legal duties stay with the platform, and Checkstep's guide to build versus buy weighs both routes.
What happens if a platform doesn't comply?
If a platform doesn't comply with the DSA or OSA, penalties are significant under both regimes. The DSA allows fines of up to 6% of global annual turnover (Articles 52 and 74). The OSA allows Ofcom to fine up to £18 million or 10% of qualifying worldwide revenue, whichever is greater, under Schedule 13 of the Act.
Under the OSA, senior managers can face criminal liability in serious cases, for example if they fail to take reasonable steps to stop their company from giving Ofcom false information. Regulators have also started requesting risk assessments directly from platforms, as Ofcom did in March 2025, so gaps tend to surface quickly.
How much moderation has to be automated versus human?
Content moderation still needs human moderators alongside AI, and neither the DSA nor the OSA sets a fixed ratio of automated to human decisions. Of the decisions platforms filed to the EU's DSA Transparency Database in the six months before October 6, 2026, 40% were fully automated.
Automation is expected for scale and speed. The DSA requires qualified staff to supervise online platforms' appeal decisions (Article 20(6)) and platforms to tell users when automation was used in a decision (Article 17(3)(c)).
Make UGC safe to host
Hosting user-generated content will always carry some risk. What matters is having a system that catches problems early, treats users fairly, and gives you a clear record when regulators ask. Get the moderation right, and UGC becomes one of your platform's biggest strengths.
Checkstep helps you build that system, starting from a single API call, and its review tools blur sensitive media for moderators and show it in grayscale by default. To see that system on your own content, book a demo with our team.
If you want support with DSA and OSA compliance, check out our compliance checklist or blogs like "DSA Compliance Isn't Optional Anymore".
