---
title: 'Human Evaluation Researcher at Nuance Labs'
canonical: 'https://feeny.ai/job/human-evaluation-researcher-nuance-labs-seattle-gvf6yh22ba2e'
type: 'job'
last_seen: '2026-09-04'
---

# Human Evaluation Researcher at Nuance Labs

- **Company:** Nuance Labs
- **Location:** Seattle, WA
- **Posted:** 2026-08-31
- **Last confirmed live:** 2026-09-04
- **Apply:** https://job-boards.greenhouse.io/nuancelabs/jobs/4384186009

## Job description

## About Nuance Labs

Nuance Labs is building photorealistic, real-time AI avatars with emotional intelligence: a full-duplex audiovisual system that can listen, speak, react, interrupt, and respond like a real person.

We're a research company, with PhDs from MIT, UW, Oxford, CMU, and Johns Hopkins, and industry experience from Apple, Meta, Amazon AGI, and more. Backed by Accel, Lightspeed, South Park Commons, and NVIDIA, we combine frontier research with ruthless engineering needed for consumer-grade, real-time systems. The team is small, the work is real, and the problems are unsolved.

How Nuance Differentiates

Most conversational AI avatars today are hacks — a face slapped on a speech-to-speech pipeline, stuck in the uncanny valley: emotionless, mechanical, one-turn-at-a-time. Current systems take 2–5 seconds to respond; natural conversation requires sub-500ms. That's a 10x improvement, and it demands rethinking the entire stack.

That rethinking starts with full-duplex: an AI that listens and speaks simultaneously, perceives emotion in real time, and responds with a face that actually reflects it. It's an extremely hard problem, and we're developing foundation models designed for it from the ground up.

## Why this role exists

"Does this avatar feel human?" is the question our whole company is organized around — and no automated metric can answer it. Lip-sync error and video quality scores say nothing about whether a smile landed as sincere or unsettling, whether a conversation felt warm or hollow, or whether someone would want to talk to our avatar again tomorrow.

Your job is to turn human judgment into a reliable signal our researchers can train and ship against. You'll take the most ambiguous problems in our field (naturalness, emotional resonance, trust, presence) and design studies whose results people actually agree on. When two models differ, your study is the tiebreaker. When a model "feels off" and nobody can say why, your investigation finds the cause.

This is a hands-on IC role and our first hire dedicated to human evaluation. You'll own it end to end (what to measure, how to measure it, who rates it, and what the results mean) working directly with the founders and the modeling team. Your findings will decide which models ship and what we train next.

## What you'll do

- Design and run qualitative and quantitative studies of our AI avatars: side-by-side comparisons, controlled rating experiments, in-depth interviews, think-alouds, diary studies, and longitudinal panels.
- Turn ambiguous judgments into instruments people can align on (rubrics, anchored scales, annotation guidelines) then measure and improve inter-rater agreement without flattening real signal.
- Use ethnographic techniques (observation of live conversations, contextual inquiry, field work) to understand how people actually experience face-to-face AI, not just what they report in a survey.
- Build the human-eval pipeline itself: participant panels, rater training and calibration, tooling, and an evaluation cadence tied to model releases.
- Calibrate automated and model-based metrics (including LLM-as-judge) against human judgment, so the team knows when to trust them and when not to.

You may be a good fit if you have

- 5+ years designing and running human-subjects research in industry or academia (UX research, HCI, experimental psychology, behavioral science, or a related field).
- Examples of study designs you can walk us through, especially ones where you got humans to converge on an ambiguous judgment (tone, emotion, quality, trust), including the rubrics, anchors, and protocols that made alignment possible.
- Strong grounding in both qualitative methods (interviews, ethnography, contextual inquiry) and quantitative methods (survey and psychometric design, experimental design, statistics for rating and pairwise-comparison data).
- Fluency with agreement and reliability. For example: You know your Cohen's kappa from your Krippendorff's alpha, and more importantly, how to raise them.
- A bias toward running a scrappy, sound study this week over a perfect one next quarter, and the ability to explain findings crisply to ML researchers.

Strong candidates may also have

- Experience evaluating generative AI: avatars or digital humans, speech or video generation, conversational agents, or emotion expression and recognition.
- A background in perceptual science or psychophysics (how people perceive faces, voices, motion, and emotion). MS/PhD in a related field welcome.
- Familiarity with human evaluation at scale: crowdsourcing platforms, annotation tooling, golden datasets.
- Enough statistics and scripting (Python or R) to analyze your own data.

No candidate checks every box. If the "good fit" list sounds like you but your background is unconventional, we'd like to hear from you anyway.

## Compensation

$160,000 – $190,000 base salary, plus meaningful equity. We think long-term ownership matters and structure equity accordingly.

Logistics

- Location: In-person in Seattle, five days a week — we believe in the compounding value of working shoulder-to-shoulder.
- Visa sponsorship: We sponsor visas (O-1, H-1B, green card, etc.) from day one.
- AI-native tooling: Do your best work with the best tools, including unlimited tokens.

## Benefits

- Health: We offer a variety of plans that meet your needs, including an HDHP with ~$2,000 in annual HSA contributions by the company (roughly 2x what most big tech companies put in).
- Time off: 15 days of PTO, 10 public holidays, and we close the office for a full week at year-end.
- Food: Lunch, drinks, and snacks on us every workday. We observe boba tea Tuesdays and Thursdays.
- Commuter benefits: Utilize pre-tax money (up to $340/month) for parking and transportation.
- 401(k): 4% match (100% of 1st 3% + 50% of next 2% contributions).

Nuance Labs is an equal opportunity employer. We believe diverse teams build better AI.

## About Nuance Labs

## Company Overview
- **One-liner**: Nuance Labs is building a real-time, full-duplex audiovisual AI model that can see, hear, reason, speak, and express emotion simultaneously, enabling face-to-face interaction with machines.
- **Entity Type**: Private (Seed stage; raised $10M in a seed round led by Accel)
- **Headquarters**: Seattle, Washington, United States
- **Founded**: Not publicly disclosed (likely 2024 or 2025 based on funding date)
- **Founders**: Fangchang Ma (CEO), Edward Zhang (CTO), Karren Yang (Chief Scientist) – all PhDs from MIT with prior experience at Apple

## Core Business
- Primary industry: Artificial Intelligence / Software Development
- Target customers: B2B (enterprise and consumer product integration) and potentially B2C (direct-to-consumer AI interfaces)
- Mission or purpose statement: “We are creating a world where people can finally talk to every product, face to face.” – building a “human foundation model” with social and emotional intelligence.

## Products & Services
- **The Full-Duplex Engine**: A real-time audiovisual model that perceives, reasons, and responds within 500ms – handling interjections, nodding, backchanneling, tone, and expression. The system is designed to cross the uncanny valley with warm, engaging personality.
- **Human Foundation Model**: An AI that learns human behavior by predicting the next audio and visual token (similar to how transformers predict the next word). It reads tone, expression, hesitation, and responds in real time.

## Market Standing
- **Valuation**: Not disclosed (private company)
- **Key Metric**: Total funding – $10M in a seed round closed on October 9, 2025
- **Notable Investors/Partners**: Lead investor Accel (with 3 total investors in the seed round)
- **Growth Signals**: 
  - Small team of ~10 employees with 9.1% monthly headcount growth
  - Strong hiring momentum: 6+ active job postings for research and engineering roles
  - Team includes PhDs from MIT, UW, Oxford, CMU, Johns Hopkins, and alumni from Apple, Meta, Amazon AGI, and Discord
  - High LinkedIn follower growth (+4.5% monthly)

## Competitive Advantages
- **Full-duplex real-time capability**: 500ms latency floor – a hard requirement for natural conversation that most current AI systems cannot achieve.
- **Multimodal emotional intelligence**: Reads tone, expression, and hesitation, not just text.
- **Elite founding team**: PhDs from top institutions with deep experience shipping ultra-low-latency ML products at Apple and other tech giants.
- **First-mover in “human foundation model”**: Applying next-token prediction to audio-visual human behavior, not just language.

## Strategic Focus
- Current priorities: Advance the Full-Duplex Engine, scale the model, and build the infrastructure for real-time audiovisual interaction. The company is hiring across pretraining infrastructure, RL research, model optimization, speech synthesis, video diffusion, and MLLM training.
- Long-term direction: Enable face-to-face AI interaction for every product, moving computing interfaces closer to human communication.

## Why Work Here
- **Culture**: “Small, fast-moving research team” with an “exceedingly high bar” – only the very best talent. Every member has massive ownership, deep trust, and the opportunity to shape both the technology and the company from the ground up.
- **Location**: Based in Seattle, WA. The company appears to be primarily in-office (all listed roles are in Seattle, and the team is small).
- **Notable perks**: Not explicitly mentioned, but the emphasis on research autonomy, high-impact work, and a founding team from top AI labs suggests a strong engineering culture with cutting-edge projects.
- **Open roles** (as of mid-2026): Research Scientist (Speech Synthesis, Video Diffusion, MLLM Training), Machine Learning Research Engineer, ML Infra Engineer, Systems Engineer (Real-Time Engine), AI Product Engineer, and more.

## Sources
1. [nuancelabs.ai](https://www.nuancelabs.ai/)
2. [nuancelabs.ai/careers](https://www.nuancelabs.ai/careers)
3. [nuancelabs.ai/about](https://www.nuancelabs.ai/about)
4. [job-boards.greenhouse.io/nuancelabs](https://job-boards.greenhouse.io/nuancelabs)
5. [linkedin.com/company/nuance-labs-ai](https://www.linkedin.com/company/nuance-labs-ai)

## Other roles at Nuance Labs

- [Nuance Research Fellow](https://feeny.ai/job/nuance-research-fellow-nuance-labs-seattle-hge7hy4qpd3t) — Seattle, WA
- [Member of Technical Staff — RL Research (New PhD Grad)](https://feeny.ai/job/member-of-technical-staff-rl-research-new-phd-grad-nuance-labs-seattle-wmvfxyd7gw98) — Seattle, WA
- [Member of Technical Staff — Model Optimization and Inference (New Grad)](https://feeny.ai/job/member-of-technical-staff-model-optimization-and-inference-new-grad-nuance-labs-ech27rtf16gn) — Seattle, WA
- [Member of Technical Staff — ML Infra (Data)](https://feeny.ai/job/member-of-technical-staff-ml-infra-data-nuance-labs-seattle-efnr92fwhx14) — Seattle, WA
- [Member of Technical Staff — Model Optimization and Inference (Experienced)](https://feeny.ai/job/member-of-technical-staff-model-optimization-and-inference-experienced-nuance-5gazzjwgrhm1) — Seattle, WA
- [Member of Technical Staff — RL Research (Experienced)](https://feeny.ai/job/member-of-technical-staff-rl-research-experienced-nuance-labs-seattle-r895bhr1agh6) — Seattle, WA
- [Member of Technical Staff — Pretraining Infra (Experienced)](https://feeny.ai/job/member-of-technical-staff-pretraining-infra-experienced-nuance-labs-seattle-vbdypnk5fzcv) — Seattle, WA