---
title: 'Evaluations - Member of Technical Staff at Simile'
canonical: 'https://feeny.ai/job/evaluations-member-of-technical-staff-simile-san-francisco-6d6h2mv7zeh2'
type: 'job'
last_seen: '2026-09-10'
---

# Evaluations - Member of Technical Staff at Simile

- **Company:** Simile
- **Location:** San Francisco, CA
- **Employment:** full-time
- **Work type:** onsite
- **Posted:** 2026-06-30
- **Last confirmed live:** 2026-09-10
- **Apply:** https://jobs.ashbyhq.com/simile/33d75074-c23b-4a1f-bfdb-129bcc5be662

## Job description

## About the Company

Simile is The Simulation Company. We simulate human behavior to keep people at the center of the decisions that shape the world. With AI, anyone can create a product, a campaign, a policy, or a script — the bottleneck has moved upstream. The hard question is no longer whether you can create something, but what to create, for whom, and how to bring it to life. Those are fundamentally human decisions, and they shouldn't be left to chance or handed off to an algorithm. We're building the infrastructure to understand human behavior at scale and to represent humans in an increasingly agentic world. Our mission is to simulate all eight billion people on earth.

We launched five months ago. Since then we've grown revenue 5x, built a new foundation model for human behavior that has run tens of millions of simulations for F100 enterprises, trained a first-of-its-kind confidence model that predicts the accuracy of every simulation, and released the first product that lets organizations verifiably predict the future. The world's leading companies use Simile to make business-critical decisions — from consumer leaders like CVS Health and Wealthfront to professional services organizations like Deloitte and Gallup — strategizing product launches, entering new markets, and forecasting earnings calls.

We've raised over $200M at a $2B post-money valuation led by Greenoaks, with Index Ventures, Hanabi, A*, Bain Capital Ventures, and CVS Health Ventures. We've grown from a small home in Palo Alto to a global team of 50+, and we're building a team of the best researchers, engineers, designers, and operators in the world. The future is too important to be left to chance.

## About the Role

As a Member of Technical Staff, Model Evaluations at Simile, you will build the measurement systems that determine whether our simulations of human behavior are accurate, trustworthy, and useful enough to guide real-world decisions. You will help shape what Simile measures, the quality bars we defend, and how evaluation evidence guides model, product, and customer decisions.

Evaluation at Simile brings together model evals, statistics, behavioral science, research methodology, product quality, and human judgment. Our models simulate people, populations, markets, and groups, which means our evals must reason about distributions, noisy human ground truth, uncertainty, qualitative outputs, behavioral data, and customer decision-making. You will work with unusually rich data about human behavior, including surveys, long-form interviews, customer studies, qualitative research, and behavioral signals such as transactions, product interactions, and other real-world traces.

We are hiring across several forms of expertise. Some candidates may be deep in LLM evaluation, model training, and research engineering. Others may bring exceptional strength in statistics, behavioral science, survey methodology, human data, product evaluation, or experimentation. Across backgrounds, we are looking for people who can reason clearly, build quickly, use agentic coding tools fluently, and take hands-on ownership of ambiguous evaluation problems.

The core question for this role is simple: How do we know when a simulation of human behavior is good enough to trust?

In this role, you will:

- Build the measurement layer for behavioral simulation: Design evals, metrics, rubrics, datasets, dashboards, and workflows that measure whether Simile’s models are accurately predicting human behavior across customer use cases, populations, question types, and decision contexts.
- Partner with modeling to improve models: Evaluate new model versions, diagnose regressions, identify priority areas for model-improvement cycles, and maintain stable eval suites that represent capabilities customers actually care about.
- Contribute to product and applied evals: Build evals for qualitative responses, retrieval, survey generation, AI-generated research reports, customer-facing outputs, and other product surfaces where model quality directly shapes customer trust. Turn subjective quality concerns into concrete rubrics, labeled data, automated graders, release criteria, and model-improvement signals.
- Make ground truth and uncertainty legible: Develop rigorous ways to compare simulated responses against human data, customer studies, Simile-collected ground truth, and behavioral datasets. Help the company reason about sampling error, uncertainty, calibration, margin of error, representativeness, and what “ground truth” means when human behavior is inherently noisy.
- Automate evaluation workflows: Use modern agentic coding tools to rapidly build internal tools, inspect model outputs, create labeling workflows, validate evals, and turn fuzzy evaluation questions into working systems. We value people who can compress long, ambiguous projects into fast, useful prototypes without losing sight of rigor or reliability.
- Help define the future of behavioral simulation evals: Prototype ways to evaluate behavioral predictions using diverse sources of data, including transaction or purchase behavior, product interactions, intervention response, first-party experiments, and eventually multi-agent group settings.

## Requirements

Must Haves

- Evaluation Taste: You have strong intuition for what makes an eval meaningful, robust, and decision-relevant. You can explain what an eval measures, what it does not measure, how it can be gamed, and why it should or should not affect a model or product decision.
- LLM and Model Fluency: You understand the basics of modern LLM training, post-training, model evaluation, and hill-climbing. You do not need to be a modeling specialist, but you can read model outputs, understand modeling team needs, and reason about whether a model change actually improved the thing we care about.
- Statistical Judgment: You are comfortable reasoning about noisy data, uncertainty, sampling, distributions, calibration, confidence intervals, measurement validity, bias, variance, and the difference between an observed result and the underlying population quantity it estimates.
- Technical and Agentic Execution: You can build internal tools, scripts, dashboards, labeling workflows, analyses, or automated eval pipelines quickly. You are comfortable working with data and automation tools such as Python, SQL, R, notebooks, LLM APIs, and agentic coding tools such as Codex, Claude Code, Cursor, or equivalent systems. You know how to move quickly while still validating outputs, catching errors, and planning for the long-term..
- Hands-On Ownership: You can independently drive a workstream while still doing the work yourself. You are willing to build the first version, inspect the data, debug the workflow, write the rubric, revise the metric, and keep going until the evaluation system is useful.

Nice to Haves We do not expect one person to have all of these. We are hiring a team with complementary strengths.

- Modeling / Model-Quality Dashboards: Experience building model evaluation dashboards, regression suites, release gates, benchmark sets, model comparison workflows, or systems that help ML teams decide where to focus and when to ship.
- LLM-as-Judge and Human Data: Experience designing rubrics, automated graders, pairwise comparisons, expert review workflows, labeling interfaces, grader calibration, or human/model hybrid evaluation systems.
- Survey Methodology and Statistics: Experience with sampling, weighting, margin of error, power analysis, uncertainty quantification, Bayesian modeling, causal inference, psychometrics, polling, or measurement theory.
- Behavioral Simulation: Experience evaluating behavioral predictions beyond self-reported survey responses, such as transaction data, purchase behavior, mobility data, product interactions, or other passively collected behavioral signals.
- Behavioral Economics / Experimentation: Experience designing RCTs, A/B tests, survey experiments, vignette studies, field experiments, behavioral games, or intervention studies.
- Multi-Agent or Group Behavior: Interest or experience in modeling group conversation, deliberation, focus groups, juries, committees, polarization, collective decision-making, or social influence.

You might be a great fit if you have worked in LLM evals, applied ML research, data science, research engineering, human data, market research, UXR, polling, behavioral science, computational social science, or behavioral economics. You might also be a recent graduate or self-directed builder with unusually strong taste in evaluation, statistics, and AI tools.

You do not need to match every bullet. If you do not perfectly see yourself in this JD but believe you would be exceptional at building the measurement layer for behavioral simulation, we would love to hear from you.

## Compensation & Benefits

At Simile, we provide competitive compensation packages that include base salary, equity, and comprehensive benefits.

- Salary Range: $200,000 – $400,000 USD
- Note: Final offers are based on experience, specialized skills, interview performance, and relevant training.
- Equity: Grants are available for eligible roles, subject to board approval.
- Health & Wellness: Comprehensive medical, dental, and vision coverage.
- Time Off: Flexible time off policies to support work-life balance.

Our Process We prioritize thoughtful conversations and clear examples of past work. Our hiring journey is designed to help both sides align on fit, working style, and expectations.

Reapplication Policy: To ensure a fair and thorough evaluation for all applicants, Simile observes a 90-day waiting period before reconsidering candidates for the same role.

Commitment to Diversity & Inclusion Equal Opportunity: Simile is an equal opportunity workplace. We welcome applicants of all backgrounds and identities, valuing an environment where everyone can contribute authentically.

Accommodations: If you require support or reasonable accommodations during the application process due to a disability, please let us know. We are happy to assist.

## About Simile

## Company Overview
- **One-liner**: Simile builds AI-powered simulations of human behavior, creating digital twins of real people to help enterprises predict outcomes and de-risk decisions.
- **Entity Type**: Private (Series A)
- **Headquarters**: Palo Alto, California, United States
- **Founded**: 2025 (spun out of Stanford research)
- **Founders**: Joon Sung Park (CEO), Michael Bernstein, Percy Liang

## Core Business
- **Primary industry/industries**: Enterprise AI Simulation, Decision Intelligence, Market Research Technology
- **Target customers**: B2B, Enterprise (Fortune 500 companies, including CVS Health, Gallup, Telstra, Wealthfront, Suntory)
- **Mission or purpose statement**: "Simulate our uncertain world" — to give organizations the ability to preview the effects of decisions before triggering the cause.

## Products & Services
- **Simile Simulation Platform**: An AI-powered platform that creates high-fidelity digital twins of real people, enabling enterprises to simulate human behavior at scale. Used for product testing, focus groups, earnings call rehearsal, litigation forecasting, and policy change modeling. The platform can predict ~80% of analyst questions on earnings calls and allows companies to test user interface changes, marketing campaigns, and policy shifts before real-world rollout.

## Market Standing
- **Valuation/Market Cap**: Not disclosed
- **Key Metric**: Total Funding — $100M (Series A, February 2026)
- **Notable Investors/Partners**: Index Ventures (lead), Hanabi, A*, Bain Capital Ventures (BCV), Fei-Fei Li, Andrej Karpathy, Adam D’Angelo (Quora), Guillermo Rauch (Vercel), Scott Belsky (A24). Customers include CVS Health, Gallup, Telstra, Wealthfront, and Suntory.
- **Growth Signals**: Named to the inaugural Forbes AI 50 Brink List (April 2026); 26 employees with +12.5% monthly headcount growth; website traffic of ~815K monthly visits growing +41.2% month-over-month; LinkedIn following of 5,696 growing +8.7% monthly.

## Competitive Advantages
- **Pioneering Research Roots**: Founded by the Stanford team behind the landmark "Smallville" experiment (2023), which demonstrated that generative AI agents could simulate realistic human social behavior — giving Simile a deep research moat.
- **High-Fidelity Digital Twins**: Unlike generic LLM-based personas, Simile builds models based on real human data (interviews, transaction logs, scientific texts), creating agents that closely mirror actual human decision-making.
- **First-Mover in Enterprise Simulation**: Already deployed with Fortune 50 clients (CVS, Gallup, Telstra) for high-stakes use cases like earnings call rehearsal, litigation forecasting, and product testing — a nascent but rapidly growing category.
- **World-Class Backing**: $100M from top-tier investors (Index, BCV, A*) and angel investors including AI luminaries Fei-Fei Li and Andrej Karpathy, providing both capital and credibility.

## Strategic Focus
- **Scaling the Simulation Model**: Expanding from hundreds of thousands of real human data points toward a goal of simulating the world's entire population (~8 billion people).
- **Foundation Model Development**: Building a general-purpose foundation model that predicts human behavior in any situation, at any scale.
- **Enterprise Adoption**: Deepening partnerships with Fortune 500 companies across industries (healthcare, telecom, financial services, consumer goods) to embed simulation into core decision-making processes.
- **Product Expansion**: Moving beyond current use cases (focus groups, earnings calls, litigation forecasting) into broader enterprise planning and policy simulation.

## Why Work Here
- **Cutting-Edge AI Research**: Work at the intersection of generative AI, human behavior modeling, and large-scale simulation — a field pioneered by the founding team at Stanford.
- **High-Impact, Early-Stage**: Join a well-funded ($100M) Series A startup with only 26 employees, offering significant ownership and influence from day one.
- **World-Class Team & Advisors**: Backed by AI legends Fei-Fei Li and Andrej Karpathy, with a founding team of Stanford CS professors. Colleagues come from top AI companies (Scale AI, Figma, Cresta, Hebbia).
- **Real-World Impact**: Your work directly helps Fortune 500 companies make better decisions — from improving healthcare at CVS to shaping public opinion research at Gallup.
- **Culture & Location**: Palo Alto-based (hybrid likely given the team is 72% US-based). Technical team makes up 28% of headcount. Fast-growing with +12.5% monthly employee growth.
- **Notable Perks**: Work on one of the most technically ambitious problems in AI (simulating human behavior at global scale) with a team of world-class researchers and engineers.

## Sources
1. [simile.ai](https://simile.ai)
2. [jobs.ashbyhq.com/simile](https://jobs.ashbyhq.com/simile)
3. [LinkedIn - Simile](https://www.linkedin.com/company/simile-ai-inc)
4. [Observer - Fei-Fei Li and Andrej Karpathy’s New A.I. Bet](https://observer.com/2026/02/simile-100m-startup-backed-fei-fei-li-andrej-karpathy/)
5. [SiliconANGLE - AI digital twin startup Simile raises $100M](https://siliconangle.com/2026/02/12/ai-digital-twin-startup-simile-raises-100m-funding/)

## Other roles at Simile

- [Brand Designer](https://feeny.ai/job/brand-designer-simile-san-francisco-683j2ynwfcrd) — San Francisco, CA
- [Product Marketing Manager](https://feeny.ai/job/product-marketing-manager-simile-san-francisco-j2cvd28qab7v) — San Francisco, CA
- [Product Designer](https://feeny.ai/job/product-designer-simile-san-francisco-syyjp8x4epr6) — San Francisco, CA
- [Storytelling](https://feeny.ai/job/storytelling-simile-san-francisco-p6t130pe6nqp) — San Francisco, CA
- [Talent Partner, GTM](https://feeny.ai/job/talent-partner-gtm-simile-new-york-3330qppp0d0r) — New York, NY
- [Strategic Account Director](https://feeny.ai/job/strategic-account-director-simile-new-york-btncrcyx6yzt) — New York, NY
- [Infrastructure Engineer - Member of Technical Staff](https://feeny.ai/job/infrastructure-engineer-member-of-technical-staff-simile-san-francisco-pf23pesce6fc) — San Francisco, CA
- [ML Systems - Member of Technical Staff](https://feeny.ai/job/ml-systems-member-of-technical-staff-simile-san-francisco-aabgzy98579g) — San Francisco, CA
- [Member of Data Staff](https://feeny.ai/job/member-of-data-staff-simile-new-york-04e1tjg4n4ps) — New York, NY
- [Security Engineer - Member of Technical Staff](https://feeny.ai/job/security-engineer-member-of-technical-staff-simile-san-francisco-2hh02krn2xy0) — San Francisco, CA