AI content moderation is the use of machine learning models to automatically detect, assess, and manage online content to ensure it complies with content policies, community guidelines, legal requirements, and global regulations. Instead of relying solely on human moderators, AI models are trained to identify harmful or non-compliant content - such as hate speech, misinformation, harassment, or explicit material - at scale and in real time.
At Checkstep, we believe the most effective content moderation solutions combine best in class AI models with human expertise. Our platform automatically classifies text, images, audio, and video for various risk types, helping platforms and moderation teams to maintain safe and inclusive online spaces. We also provide transparency and auditability tools that comply with regulations such as a Digital Services Act (DSA) so that moderation decisions can be explained, appealed, and continuously improved.
 

Some categories include: suicide/self-harm, explicit (incl. adult content, nudity), spam, aggressive (incl. violence, visually disturbing), hate (incl. bullying, threat, toxicity), drugs and illicit goods (incl. alcohol and tobacco).

Specialist harms: terrorism, CSAM (Child Sexual Abuse Material), personal identifiable information (PII), intellectual property (IP) infringement, fact-checking etc.

We add, adapt and tailor models to our client’s needs and thresholds and hence the above list is non-exhaustive. Also note that within some of these categories, there are options to tailor into different sub-categories depending on your individual requirements.
 

Yes, Checkstep supports large volumes in terms of throughput and individual case size. Depending on content requirements, customers can pick and choose the AI providers to best cover their large volumes at acceptable costs.

Our technology has sub-50 second millisecond latency, which means we can scan content in real time to provide an immediate assessment of whether content violates your policies. 

Yes, Checkstep integrations require a small amount of engineering work to send Checkstep content via an API and to process the response from your AI scanning and policy. Customers can submit content without a technical integration but most systems require some integration to support end to end moderation.

Our standard pricing model uses a simple matrix of set-up, volume tiers and operator seats. There are lots of variables with regards to volume, media types, abuse types, latency and accuracy needed.

As the Checkstep platform integrates all leading vendors. We will work with you to find the right blend of scanning technologies at a price point that works for you.

Yes – Checkstep has a User Reporting feature where users can report content and the content will be scanned by Checkstep, then the platform can decide whether to automatically action the content or send it for human review.

Additionally, Checkstep also allows to scan User Profiles for certain violations. This is done by de-composing a profile into it’s different elements (profile picture, URLs, bio, PII, …) and scan each component and then aggregate this into a user-profile detection strategy.

The client decides on the risk thresholds using the harm category and the AI scores. Checkstep can provide guidance. These risk thresholds can be adjusted in the moderation interface.

We currently process 100 languages natively and we can support every language through translation. 

Yes, we offer Age Verification services. We can apply age estimation algorithms and also integrate end-to-end age and/or identity verification flows.

Yes, we have live streaming capabilities.

We can help with preparing for the DSA and OSB implementation, whilst remaining consistent with data protection legislation. We can work directly with your governance team if required to run gap analysis and formulate a risk profile comfortable to your individual business. We will then help drive the implementation of the strategies through Checkstep tooling, your current current stack integrations and overall trust and safety operation to streamline legislative requirements and drive down compliance costs.

Yes, we fully support a mix of:

• Different queues can be set-up based on different detection policies, regions, regulations;
• Escalations within the platform to ensure content can get seen quickly by the right team;
• Different queues can be manned by different teams of moderators;
• Queues can be ranked according to needs: i.e. first-in-first-out, minimising SLAs, optimising for severe harms etc.