Trust & Safety
Specialists in this role develop detection systems and enforcement strategies to identify and mitigate emerging abuse patterns across AI products, working at the intersection of data science, policy, and operations. They balance competing priorities—detecting sophisticated threat actors while maintaining platform usability—by building scalable detection pipelines, conducting rapid investigations, and collaborating with policy and engineering teams to implement mitigations. Unlike policy-focused roles, these positions emphasize technical implementation and quantitative analysis; unlike pure engineering roles, they require deep domain expertise in specific abuse vectors and threat actor behavior. These analysts typically sit within dedicated Trust & Safety or Safeguards teams that operate cross-functionally with research, product, and legal to stay ahead of evolving misuse techniques.
Skills
What companies are looking for in this role.
Identifying, investigating, and analyzing abuse patterns and harmful behaviors across digital platforms and ecosystems
Designing and maintaining analytical tools, evaluation methodologies, and quality metrics for intelligence workflows
Developing detection signals, tracking strategies, and classification systems to identify malicious activity and policy violations at scale
Monitoring and analyzing emerging threats, trends, and attack vectors to inform proactive risk mitigation strategies
Querying, organizing, and transforming complex datasets using structured data analysis techniques
Collaborating with cross-functional teams including engineering, product, policy, and leadership to shape safety decisions
Designing and executing capability evaluations and threat modeling to assess AI system vulnerabilities and misuse potential
Creating comprehensive policy frameworks, taxonomies, and enforcement workflows for content moderation and safety
Translating domain expertise into actionable safety requirements, guardrails, and technical safeguards for AI systems
Building reusable playbooks, frameworks, templates, and lightweight tools to operationalize and scale recurring analyses
Conducting rapid response investigations into escalations and high-impact safety incidents in ambiguous situations
Designing and refining safety systems to detect harmful AI model outputs and prevent misuse by sophisticated threat actors
Analyzing behavioral and psychological signals to understand user vulnerability, risk patterns, and human-AI interaction risks
Detecting and investigating autonomous, agentic, or self-improving behaviors in AI systems that may introduce safety risks
Applying frontier risk scanning, horizon scanning, and strategic foresight methodologies to identify emerging threats
Developing account-level signals, identity-linking strategies, and graph-based approaches to detect coordinated abuse
Conducting open-source research, dark web monitoring, and cross-platform threat analysis to understand threat actor behavior
Producing decision-ready intelligence briefs, risk assessments, and narratives for technical and non-technical stakeholders
Communicating complex technical findings and domain expertise clearly to diverse teams and leadership
Working cross-functionally and collaboratively with diverse technical and non-technical partners
Synthesizing raw signals, multiple data sources, and qualitative information into structured, actionable insights
Operating with strong judgment and resilience in high-pressure environments with sensitive or disturbing content
Identifying gaps in existing safeguards, evaluations, and monitoring systems and proposing improvements
Leading, mentoring, and managing teams focused on safety operations and content moderation
Technology
The tools and technologies that define this role.
Open Jobs
39 open Trust & Safety jobs across 5 companies.
Other Security roles
Identifies and mitigates security vulnerabilities in applications and products.
Secures cloud infrastructure, networks, and systems.
Generalist security engineering role spanning multiple security domains. For security engineers who work across application, infrastructure, and cloud security without a single dominant specialization. The default home for "Security Engineer" titles when the function is clearly Security.
Builds detection systems, investigates security incidents, and leads incident response efforts.
Conducts offensive security assessments including red teaming, penetration testing, and adversarial simulation.