When you’re building a product, the “safety by design” conversation usually centers on three things: anticipating risks, having the tools to address them, and shipping core safety features like content reporting and appeals. 

That’s a good starting point for leaders who know they need to build trust (and compliance) into their platforms. But it’s only a starting point. Great safety-by-design products go further. 

They make deliberate decisions about the internal systems and processes behind every safety decision, and about what the end user experiences while those decisions are being made.

 

The starting point

The standard best practices in most safety-by-design guides are a solid foundation. Every product team should:

  • Engage Trust and Safety in the product early. Bring T&S into the design of a new feature or product, not the launch review. It makes safety easier to deliver, avoids technical rework, and reduces unexpected harm. It also keeps safety from feeling like an afterthought in your user experience.
  • Document and assess risks. Anticipate where harm can occur before launch, and know where you’ll need trust & safety resources. If you allow public image uploads, for example, you need to be ready for everything from spam to child sexual abuse material (CSAM).
  • Give users core T&S features. Users need control and transparency: the ability to report violating content, notifications when action is taken on their content, and the right to appeal. These are increasingly more than best practice; regulation like the EU’s Digital Services Act requires them.
  • Match safeguards to your risks. Put the right mix of detection, automation and human review in place, and plan to keep evolving it as your product and the threats against it change. Trust and Safety tooling is a constantly evolving system.
  • Make the business case: Align stakeholders by showing that safety by design doesn’t just reduce risk; it improves the product. Users stay, buy, and engage where they feel safe.

But this only scratches the surface of what building product with Trust & Safety looks like. 

To get to great, product leaders need to understand their T&S policies and implement features in a way that balances safety with the user experience itself.

 

Common mistakes with the best intentions

These are the mistakes we see most often, even from product teams who already know the basics.

  1. Leaving policy complexity to your T&S Team

    A lucky product leader has a great Trust & Safety team that defines the terms of service, community guidelines, and broader policies. 

    However good that team is, a product leader still needs to get into the weeds of what the policy says and what it should enforce. Capturing the technical requirements for safety features isn’t enough. 

    When product doesn’t know the policy well, we see two mistakes. 

    First, teams miss chances to pass extra context to T&S tools and automated workflows, so those workflows make lower-quality decisions. 

    Second, teams don’t build the product to support complex enforcement. For example, if your bullying policy requires repeated, targeted abuse, your tooling can’t enforce it by looking at single messages or posts in isolation.
     

  2. Ignoring the models you need and their limits

    AI and LLM tools make automated Trust & Safety feel easier to deploy, and they’re smarter than ever. 

    But they aren’t magic, especially if you need cost-efficiency at scale, very low-latency decisions, or strong performance in nuanced policy areas. Your product will have to make cost-quality-speed trade-offs. 

    If you don’t understand where some models fall short, you’ll make design decisions that lock you out of the accuracy you need.

    Take live chat. A product leader who wants moderation decisions in under 100ms has to accept that deep, contextual analysis isn’t possible at that speed.

     That may change the design itself: a fast path for clear-cut decisions and a slower path for the ones that need more thought.
     

  3. Using one approach for all safety decisions

    Applying the same moderation approach across your whole product keeps integration simple, but at a huge cost to your safety capabilities. 

    There are three broad patterns: 1) synchronous automated decisions, where content is approved or rejected before it’s posted (common in live chat); 2) publish by default, and remove content if it’s found to be harmful; and 3) hold by default, and publish once content is found to be safe.

    The right pattern depends on the content, where it appears, and your risks. Product and T&S should choose it together for each feature. 

    Choose wrong, and you limit both your ability to bring humans into the loop for your hardest decisions and the AI workflows you can use.

     Synchronous-only workflows, for example, rule out escalation to human reviewers and, depending on your latency requirements, can rule out the latest frontier models.
     

  4. Treating flexible workflows as a nice-to-have

    In Trust & Safety, most of your content will probably follow a single workflow. But building your system to support only that default is a mistake. 

    A product leader building safety by design should set up a user experience pattern that covers at least three paths: 1) immediate approval; 2) immediate enforcement; and 3) a gray-area path that escalates to slower models, to moderators, or to other review processes. 

    In practice, this means investing not just in moderation decisions but in one or more pending states, to preserve the content and author experience while a decision is still being made.
     

  5. Not deciding a failure state

    Assuming that the trust and safety system will never have a ‘bad day’ can lead to a lot of manual recovery work.

    Even with 99.999% uptime, teams need to plan for how things go wrong. Without a playbook to re-run or recover from incidents, crises add stress when the team is already dealing with angry users or press inquiries.

     

A checklist for doing safety by design better

 

  1. 1. Read the policy with examples before you start.

    For each feature, know which policies it will enforce and what context each decision needs, such as user history, relationships, or the surrounding conversation. Then build the product to pass that context through.

  2. 2. Build for policies that span more than one item.

    When a policy depends on patterns (repeated, targeted, coordinated), it needs a data model and tooling that can see across messages, users, and time.

  3. 3. Understand your latency-quality trade-offs and document them.

    Decide how fast each decision has to be and be honest about the impact. Even if low latency is a requirement, consider implementing a fast path (for most cases) and a slow path instead of forcing one speed on every item.

  4. 4. Choose a moderation pattern for each feature or piece of content.

    Decide what will be the best experience for your end users: Synchronous decisions, publish-then-review, hold-then-publish, keep-private-then-publish, etc. Make these decisions for each content type and feature.

  5. 5. Design for the gray area (not just approve and enforce).

    All three paths need a defined user experience.

  6. 6. Build “pending” as a real state; get creative on the experience.

    Decide what your end-user sees when items need further review. Determine how long things can wait and what happens if review never comes.

 

The standard safety-by-design advice is still the right foundation. What separates good from great is the set of decisions underneath it: how fast each decision needs to be, which moderation pattern fits each feature, what users see while content is pending, and what happens when your systems have a bad day. These decisions get made in your T&S integrations whether or not anyone makes them deliberately. Product leaders who make them on purpose, with their Trust & Safety team, build products that are both safer and better to use. The ones who don't usually discover them in bad-user experiences (that could have been avoided).