---
title: 'Software Engineer, AI Evaluation at Nuna'
canonical: 'https://feeny.ai/job/software-engineer-ai-evaluation-nuna-san-francisco-w3nr46tax8s6'
type: 'job'
last_seen: '2026-09-09'
---

# Software Engineer, AI Evaluation at Nuna

- **Company:** Nuna
- **Location:** San Francisco, CA
- **Compensation:** $148k–$232k
- **Employment:** full-time
- **Work type:** hybrid
- **Posted:** 2026-08-13
- **Last confirmed live:** 2026-09-09
- **Apply:** https://jobs.ashbyhq.com/nuna/6d313b04-d9ad-4db7-98d4-cb16d59c3ca1

## Job description

Chronic disease isn't managed in a clinic. It is managed at home, in relationships, in the everyday. What's on the dinner table, what gets said, and who notices when someone's struggling. For the 130 million Americans managing a chronic condition, the healthcare system has offered the same answer for decades: a 15-minute doctor's visit, a pamphlet, and a portal login they'll never use.

At Nuna, we are building an AI health coach that shows up like a person who actually has time: available at 3am, infinitely patient, and never behind a waiting room. We use motivational interviewing to help patients and their families see themselves clearly, design experiments that fit their real lives, and navigate a system that has not historically been on their side. We are building from the ground up around a simple belief: patients don't want to be healthy; they want their lives back.

We're not competing with other health apps. We're competing with the moment a person gives up on getting better. If that's a problem you want to work on, we'd like to talk.

## YOUR TEAM

We are a small, interdisciplinary team - engineers, data scientists, designers, product managers, and clinicians - building Nuna's AI health coach. Our products are only as good as the care and science behind them, and your piece is how we know the coach is safe and working. You'll own the evaluation system for the team building the coach: a data scientist partners with you on the science, clinicians and designers supply the ground truth, and the engineers shipping the agents depend on the signal you produce to decide what ships.

## THE ROLE

This is a net-new, build-first role for someone who wants to own how we evaluate our AI agents end to end. You'll build the harnesses, datasets, judges, and release gates that tell us whether the coach is safe and good, and you'll own both that infrastructure and the evals that run on it. This is not a test-execution role - you write the code and own the system, rather than running tests someone else designed. You'll make the day-to-day calls on standards, methods, and trade-offs, often with incomplete information and the freedom to define the right answer yourself. Comfort with ambiguity is part of the job.

## WHAT YOU'LL DO

- Build testing harnesses and evaluation infrastructure for our agentic products and our internal agentic tooling
- Own our evals end to end - both the architecture and the content - with support from data science and clinical partners
- Make every agentic deployment run through the testing apparatus before it ships, and own the release gates that keep unsafe or low-quality behavior from reaching patients
- Build the ground truth, judges, and metrics, and validate that the evaluation itself can be trusted: calibration to human labels, reliability, and honest confidence on every number, in partnership with our data scientist
- Build functional tooling for labeling and review workflows, so clinicians, coaches, and designers can author and review evaluation scenarios without an engineer in the loop
- Help close the loop from evaluation results to model and prompt refinement, working toward systems that iterate safely with less human hand-holding

## WHAT WE'RE LOOKING FOR

- Significant experience building and shipping reliable production systems and tooling
- Deep understanding of how to evaluate AI systems - LLM-as-judge, red-teaming and adversarial testing, synthetic scenario generation, and multi-turn and agentic evaluation - and a clear sense of how evals themselves fail. You've deployed evals and automated AI tooling in production, not just prototyped them
- A testing mindset applied to building the measurement system, not running tests against a spec: adversarial instinct, coverage thinking, regression discipline, and documentation others can build on
- You use AI in your daily work and build tools that make the people around you more effective
- Enough fluency in statistics and experimental design to partner with a data scientist on calibration and reliability
- Can design the workflow and build a functional UI for non-engineers like clinicians and labelers
- A genuine interest in improving healthcare alongside an interdisciplinary team, with the judgment to tell a launch-blocking issue from a nice-to-have

Bonus Points

- Experience in healthcare or another regulated, high-trust domain, and familiarity with the regulatory landscape
- Hands-on experience with the eval tooling ecosystem (LangSmith, Braintrust, DeepEval, Ragas, Promptfoo, or similar)
- Red-teaming or AI safety experience - prompt injection, jailbreaks, adversarial and stress testing
- Experience with automated, eval-driven model or prompt optimization
- You've built in an early-stage or fast-moving environment

Nuna is an Equal Employment Opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, age, disability, genetics and/or veteran status.

## #LI-LM1

## About Nuna

## Company Overview
- **One-liner**: Nuna builds data-driven products and services that power value-based payment arrangements and facilitate high-value healthcare delivery, including a patient-focused app for managing chronic conditions.
- **Entity Type**: Private (venture-backed)
- **Headquarters**: San Francisco, California, USA
- **Founded**: 2010
- **Founders**: Jini Kim (CEO) and David Chen (CSO)

## Core Business
- **Primary industry**: Healthcare IT / Healthcare Data Analytics
- **Target customers**: B2B (government health programs such as CMS/Medicaid, health plans, provider networks) and B2C (patients managing chronic conditions)
- **Mission statement**: "Make high quality healthcare affordable and accessible for everyone"

## Products & Services
- **National Medicaid Data Platform**: The country’s first nationwide Medicaid dataset and data pipeline, built for CMS to enable better insights and value-based care.
- **Nuna App**: A patient-focused mobile application that helps manage chronic conditions. Includes an AI coach for personalized wellness guidance, care plan tracking, prescription and lab integration, vital signs monitoring, and secure sharing with doctors. Supported by a Clinical Team that reviews interactions and coordinates care.
- **Value-Based Care Platform**: A financial innovation and operating platform for value-based payment arrangements, originally developed in partnership with leading healthcare organizations.

## Market Standing
- **Valuation**: Not publicly available
- **Key Metric**: Total funding – not disclosed; notable investors include John Doerr (Chairman of Board, Kleiner Perkins), Andy Slavitt (Town Hall Ventures, former Acting Administrator of CMS), and Bing Gordon (venture capitalist).
- **Growth Signals**: Started with a team of ~75 people to build the Medicaid dataset; now employs 278 people (2026). Active hiring across Data, Customer Success, Design, and Sales roles. Partnership with CMS to create the first nationwide Medicaid data pipeline.
- **Notable Partners**: Centers for Medicare & Medicaid Services (CMS); Town Hall Ventures; Kleiner Perkins.

## Competitive Advantages
- **Proven Track Record**: Key team members helped rescue Healthcare.gov in 2013, demonstrating engineering and crisis-response capability.
- **Mission-Driven Culture**: Core values include "Every row of data is a life" and "We are all Kimong’s Nuna" – reflecting deep empathy for patients and caregivers.
- **Data Moat**: Exclusive access to comprehensive Medicaid data gives Nuna unique insights into underserved populations.
- **Clinical Oversight**: The Nuna App includes a licensed Clinical Team that monitors user interactions and ensures medical best practices.
- **Privacy & Security**: Strong focus on data security and patient privacy, with clinical oversight safeguarding accuracy.

## Strategic Focus
- Scaling the Nuna App to help more patients with chronic conditions manage their health between doctor visits.
- Expanding AI-driven coaching and care coordination features to increase patient engagement and adherence.
- Deepening partnerships with providers and health plans to extend value-based care infrastructure.
- Continuing to serve government healthcare programs (Medicaid, Medicare) with data platforms that improve outcomes and reduce costs.

## Why Work Here
- **Culture**: Strong mission-driven environment with values such as "Be the People We Serve" and "Rise by Lifting Others." Emphasis on diversity, equity, and inclusion with employee resource groups.
- **Work Flexibility**: Job-specific – some roles are fully remote (e.g., Chief of Staff, Executive Sales Team; Customer Success Director in Chicago), while others are on-site in San Francisco. Hybrid options may be available.
- **Benefits** (per careers page and Built In): Unlimited PTO; excellent medical, vision, and dental coverage; 401(k) package; home office stipend; educational credit; generous fitness stipend; mental health resources; commuter benefits; pet-friendly office (SF); company equity; family medical leave and generous parental leave; childcare benefits.
- **Engineering Culture**: Emphasis on data engineering, rigorous technical interviews (take-home tests, virtual onsites), and solving complex healthcare data challenges. Innovation is encouraged through lightning talks and coffee chat groups.
- **Interview Process**: Typically 2-3 weeks, including recruiter screen, manager screen, technical screen/take-home, and virtual onsite.

## Sources
1. [nuna.com/about-us](https://www.nuna.com/about-us/)
2. [nuna.com/careers](https://www.nuna.com/careers/)
3. [builtin.com/company/nuna-inc](https://builtin.com/company/nuna-inc)
4. [nuna.com](https://www.nuna.com/)
5. [jobs.ashbyhq.com/nuna](https://jobs.ashbyhq.com/nuna)

## Other roles at Nuna

- [Customer Success Director](https://feeny.ai/job/customer-success-director-nuna-chicago-jx9w06gcm80m) — Chicago, IL
- [Lead Software Engineer](https://feeny.ai/job/lead-software-engineer-nuna-san-francisco-5fqxz9m9xx5d) — San Francisco, CA
- [Software Engineer](https://feeny.ai/job/software-engineer-nuna-san-francisco-zn5h1vxda397) — San Francisco, CA
- [Product Designer](https://feeny.ai/job/product-designer-nuna-san-francisco-aae310hbm4nn) — San Francisco, CA
- [Senior Software Engineer](https://feeny.ai/job/senior-software-engineer-nuna-san-francisco-px0nk91qqtwk) — San Francisco, CA