---
title: 'Research Engineer, Evals at Variance'
canonical: 'https://feeny.ai/job/research-engineer-evals-variance-san-francisco-geadd7cbpjyz'
type: 'job'
last_seen: '2026-09-06'
---

# Research Engineer, Evals at Variance

- **Company:** Variance
- **Location:** San Francisco, CA
- **Compensation:** $250k–$300k
- **Employment:** full-time
- **Work type:** onsite
- **Posted:** 2026-03-31
- **Last confirmed live:** 2026-09-06
- **Apply:** https://jobs.ashbyhq.com/intrinsic-safety/66dfd37e-6df5-4f89-8454-b3005bd6e1aa

## Job description

## ROLE

At Variance, we are teaching machines to make the hardest judgment calls at scale. That means building AI agents for the high-stakes gray area of risk investigations, fraud, and identity reviews.

We’re a small, talent-dense team in San Francisco working on a problem at the edge of what AI systems can reliably do: making good decisions in messy, adversarial, real-world environments. We focus on building, high-consequence systems problems where the edge cases matter most.

We’re looking for a Research Engineer to help define how we measure and improve model quality. You’ll build the benchmarks, datasets, tooling, and evaluation loops that tell us whether our systems are actually getting better on the tasks that matter. This role sits at the center of research, product, and engineering. It is about creating rigorous, domain-specific evaluations that reflect real customer workflows, expose meaningful failure modes, and drive the next generation of model and agent improvements.

## YOU’RE A FIT IF YOU:

- Care deeply about craftsmanship and have strong opinions about model quality, measurement, and experimental rigor
- Want to work on core model and agent behavior, not just surface-level product metrics
- Are excited by the challenge of defining what “good” looks like in messy, high-stakes environments
- Think in tight loops: hypothesis, benchmark design, evaluation, failure analysis, iteration
- Have strong engineering fundamentals and like building robust systems around ambiguous research problems
- Thrive in environments where success criteria are initially underspecified and need to be sharpened through work
- Are willing to do the work in the trenches: reviewing outputs, grading edge cases, curating datasets, and refining tasks until the evaluation actually measures what matters
- Care deeply about building systems that protect people from fraud, scams, and abuse

## WHAT YOU’LL DO

- Build proprietary benchmarks and datasets to evaluate models and model systems on fraud, identity, and risk workflows
- Design and run offline and online evals that measure model performance on real customer tasks, not just abstract benchmarks
- Define quality metrics for judgment systems, including precision, calibration, consistency, abstention, and failure handling
- Study where models and agents break, and turn those failures into better evals, better datasets, and better training loops
- Build reusable evaluation tools and quality building blocks that can be used across different product surfaces and workflows
- Partner closely with research, engineering, product, and design to improve system quality through rigorous experimentation
- Help create a strong culture of scientific experimentation, clear measurement, and continuous iteration
- Push the boundary of how AI systems are evaluated in regulated, adversarial, and high-consequence environments

## WHAT SUCCESS LOOKS LIKE

- We have a clear, trusted view of how our systems perform across the workflows that matter most
- Our evals predict real-world quality better than generic benchmarks
- We identify meaningful failure modes earlier and improve system behavior faster
- We develop differentiated datasets, benchmarks, and quality loops that compound over time
- Research and engineering teams use your work to make better decisions about what to train, ship, and improve next
- Variance becomes known for rigorous, domain-specific evaluation of judgment systems

## PREFERRED BACKGROUND

- Experience training, evaluating, or improving modern ML systems
- Strong programming skills and comfort working in research-heavy codebases
- Experience building benchmarks, datasets, evaluation pipelines, or quality systems
- Familiarity with LLMs, agent systems, retrieval, post-training, or adjacent areas
- Ability to design clean experiments and draw reliable conclusions from noisy results
- Strong engineering judgment and a bias toward building
- Interest in fraud, risk, trust and safety, compliance, or other regulated and adversarial domains

## OUR CULTURE

We believe in ownership, urgency, and craft. We enjoy spirited debate, wild ideas, and building things we’re proud of. We’re fully in-person in San Francisco.

## WHAT WE OFFER

- Competitive salary and meaningful equity
- Platinum-level medical, dental, and vision insurance
- Unlimited PTO, sick leave, and parental leave
- Up to $100 per month in reimbursement for personal health and wellness expenses
- 401(k) plan

## About Variance

## Company Overview
- **One-liner**: Variance builds AI agents that automate complex risk investigations, such as sanctions screening and fraud review, for Fortune 500s and regulated financial institutions.
- **Entity Type**: Private (Series A, $21.5M raised in March 2026)
- **Headquarters**: San Francisco, California, USA
- **Founded**: 2022
- **Founders**: Michael Lin and Karine Mellata

## Core Business
- **Primary industry/industries**: Enterprise AI, Compliance Technology, Cybersecurity, Risk Management
- **Target customers**: Fortune 500 companies, marketplaces, and regulated financial services (B2B, Enterprise)
- **Mission or purpose statement**: "To build technology that helps good prevail in highly adversarial environments." [variance.com](https://www.variance.com/company)

## Products & Services
- **Variance AI Agents**: A suite of AI agents designed to automate the full investigative workflow for compliance and risk teams. Agents can handle tasks from Level 1 to Level 3 reviews, including sanctions screening, fraud review, and identity verification. They extract insights from messy data (scanned documents, images, websites) and provide a complete, auditable evidence trail for every decision. [variance.com](https://www.variance.com/)

## Market Standing
- **Valuation/Market Cap**: Not publicly disclosed.
- **Key Metric**: Total Funding of $24.6M ($3.1M Seed in 2024 + $21.5M Series A in March 2026). [ycombinator.com](https://www.ycombinator.com/companies/variance)
- **Notable Investors/Partners**: Y Combinator (Winter 2023 batch), with the Series A led by a prominent venture firm (specific lead investor not named in the provided search results, but the round was reported by Axios and Business Wire).
- **Growth Signals**: Raised a $21.5M Series A in March 2026, signaling strong market traction. The company claims its agents can reduce investigative cycles by 10x and collect 90% of evidence for each case. Trusted by Fortune 500s.

## Competitive Advantages
- **Auditability & Explainability**: Unlike many "black box" AI systems, Variance is "built for regulators and auditors." Every decision comes with a complete, cited evidence trail, making it defensible in a highly regulated environment.
- **Adversarial Focus**: The company is explicitly designed for "highly adversarial environments" where threats evolve constantly, giving it a sharper focus than general-purpose automation tools.
- **Speed to Production**: The company emphasizes a culture of velocity, measuring timelines from prototype to production in "days, not months," allowing them to adapt quickly to new fraud patterns.

## Strategic Focus
- **Deepening Enterprise Sales**: The company is actively hiring Enterprise Account Executives and Sales Engineers, indicating a push to expand its customer base among large, regulated institutions.
- **Product Development**: Open roles for Research Engineers (Evals, Judgment Systems) and Senior Software Engineers suggest a continued focus on improving the accuracy, reliability, and reasoning capabilities of their AI agents.
- **Scaling the Team**: With a Series A close in early 2026, the company is in a clear growth phase, expanding its engineering and go-to-market teams from a small base (12-person team per YC profile).

## Why Work Here
- **Culture & Values**: The company emphasizes a culture of **Ownership** (every employee is an owner from day one), **Velocity** (fast iteration), **Craftsmanship** (deep care and intentionality in building), and **Impact** (building systems trusted for mission-critical decisions). [variance.com](https://www.variance.com/company)
- **Work Policy**: In-person in San Francisco.
- **Compensation & Perks**: Offers competitive salary and meaningful equity. Benefits include platinum-tier medical/dental/vision insurance, a 401(k) program, unlimited PTO/sick leave/parental leave, daily catered lunches and dinners, a monthly wellness stipend, commuter benefits, and company events. [variance.com](https://www.variance.com/careers)
- **Engineering Culture**: A high-agency environment where engineers own end-to-end responsibility for systems. The work is technically challenging, involving building AI that operates in real-time against sophisticated adversaries.

## Sources
1. [variance.com](https://www.variance.com/)
2. [variance.com/company](https://www.variance.com/company)
3. [variance.com/careers](https://www.variance.com/careers)
4. [ycombinator.com](https://www.ycombinator.com/companies/variance)

## Other roles at Variance

- [Software Engineer, Agent](https://feeny.ai/job/software-engineer-agent-variance-san-francisco-dza9223d58q9) — San Francisco, CA
- [Enterprise Account Executive](https://feeny.ai/job/enterprise-account-executive-variance-san-francisco-0sebfqrpdc9k) — San Francisco, CA
- [Agent Product Manager](https://feeny.ai/job/agent-product-manager-variance-san-francisco-7kwq2pbz1mkv) — San Francisco, CA
- [Product Designer](https://feeny.ai/job/product-designer-variance-san-francisco-whvwasrtj9m7) — San Francisco, CA
- [Product Marketing Lead](https://feeny.ai/job/product-marketing-lead-variance-san-francisco-8qk1v7zqt80f) — San Francisco, CA
- [Enterprise Account Executive](https://feeny.ai/job/enterprise-account-executive-variance-new-york-yd980aaksyfc) — New York, NY
- [Enterprise Sales Engineer](https://feeny.ai/job/enterprise-sales-engineer-variance-san-francisco-wz60q5thfqf1) — San Francisco, CA
- [Research Engineer, Judgment Systems](https://feeny.ai/job/research-engineer-judgment-systems-variance-san-francisco-3czxbrkkqtvr) — San Francisco, CA
- [Software Engineer](https://feeny.ai/job/software-engineer-variance-san-francisco-qpk136xty44m) — San Francisco, CA
- [Senior Software Engineer](https://feeny.ai/job/senior-software-engineer-variance-san-francisco-vd07ecwmbygz) — San Francisco, CA