---
title: 'Senior Backend Engineer, AI Evaluations at Ellipsis Health'
canonical: 'https://feeny.ai/job/senior-backend-engineer-ai-evaluations-ellipsis-health-san-francisco-9y9f1zpn507s'
type: 'job'
last_seen: '2026-11-02'
---

# Senior Backend Engineer, AI Evaluations at Ellipsis Health

- **Company:** Ellipsis Health
- **Location:** San Francisco, CA
- **Employment:** full-time
- **Work type:** hybrid
- **Posted:** 2026-05-13
- **Last confirmed live:** 2026-11-02
- **Apply:** https://jobs.ashbyhq.com/ellipsis-health/60fb0284-8ba7-48a6-b66b-9bf4beb7b133

## Job description

## About the Role

We’re seeking a strong Senior Backend Engineer to join our AI Evaluation team, focused on building the infrastructure and internal tooling that enable reliable, repeatable evaluation of AI systems in production. In this role, you’ll develop the scaffolding around our evaluations platform - API/data integrations, test configuration management, and reporting tools that make evaluations easy to run, extend, and operate at scale. You’ll also build and maintain evaluation frameworks for individual system components such as ASR, LLMs, TTS, knowledge bases, and guardrails, productionalizing these frameworks to support regular analysis, regression detection, and continuous monitoring.

In addition, you’ll create debugging tools that make it easy to inspect end-to-end calls, trace failures, and surface all relevant signals in one place - empowering internal teams to diagnose issues quickly and confidently. The ideal candidate is a pragmatic, infrastructure-minded engineer who enjoys turning ad hoc analysis into durable systems, cares deeply about developer experience, holds a high bar for software-quality and takes pride in building tooling that makes complex AI systems observable, testable, and easier to operate.

Responsibilities:

- Build and maintain infrastructure and tooling for the AI evaluations platform used by internal teams, including automated testing platform for AI voice agents, debugging and observability tools.
- Develop and productionalize evaluation frameworks for individual system components such as ASR, LLMs, TTS, knowledge bases, and guardrails.
- Partner with ML, engineering and QA teams to translate evaluation requirements into robust, maintainable infrastructure and tooling.
- Improve developer experience by making evaluation systems easy to extend, well-documented, and reliable in day-to-day use.
- Ensure evaluation tooling meets production standards for reliability, performance, and maintainability.

Qualifications:

- 5+ years of professional software engineering experience, with a strong focus on building backend systems, platforms, or developer tooling.
- Proven experience designing and maintaining production-grade infrastructure with code, including APIs, services, and data pipelines.
- Strong proficiency in at least one general-purpose programming language (e.g., Python, Typescript/Javascript, Java, or similar).
- Experience using test automation frameworks, evaluation pipelines, or CI/CD-integrated testing systems.
- Familiarity with observability and debugging tools (logging, metrics, tracing) and building internal tools that improve developer and QA workflows.
- Strong debugging skills and a methodical approach to diagnosing production and evaluation issues.
- Ability to collaborate effectively across engineering, QA, and operations teams, translating requirements into reliable, maintainable systems.
- Product-minded approach to infrastructure, with attention to usability, documentation, and long-term maintainability.

Preferred:

- Experience working with complex, multi-component systems (e.g., ASR, LLMs, TTS, or other ML-powered services)
- Experience working in healthcare or other regulated environments, including awareness of HIPAA and PHI handling.
- Familiarity with conversational AI or voice agents, including multi-turn dialogue, latency constraints, and error recovery.
- Familiarity with LLM observability or evaluation tools (e.g., Langfuse, prompt eval frameworks).
- Background in digital health, care coordination, or patient-facing systems.

Salary and Benefits We offer competitive salary and benefits, including 401(k) matching, health, vision, and dental insurance, and very flexible paid time off. The typical salary range for this role is $175,000 to $250,000 USD, depending on skills, qualifications, and relevant experience.

Background Checks As a health technology company, we reserve the right to run background checks on candidates to whom we extend offers, in compliance with applicable laws. We evaluate candidates holistically and comply with all “ban the box” regulations.

Assistance If you have a disability or require accommodations during the application or recruitment process, please contact careers@ellipsishealth.com.

## About Ellipsis Health

## Company Overview
- **One-liner**: Ellipsis Health provides an emotionally intelligent AI care manager, Sage, that automates patient engagement, clinical assessments, and care coordination for health plans, health systems, specialty care, and pharmaceutical companies.
- **Entity Type**: Private (Series A-II; total funding $75.39M)
- **Headquarters**: San Francisco, California, United States
- **Founded**: 2017
- **Founders**: Not publicly available

## Core Business
- Primary industry: Healthcare AI / Healthcare IT
- Target customers: B2B – health plans, health systems, specialty care providers, and pharmaceutical companies
- Mission/purpose: To deliver empathetic, autonomous virtual care management that reduces administrative burden, improves patient outcomes, and enables care teams to work at the top of their licenses.

## Products & Services
- **[Sage – AI Care Manager](https://ellipsishealth.com/product/)**: An agentic AI platform that conducts fully autonomous, natural-language calls with patients. Use cases include program enrollment & engagement, clinical assessments (HRAs, HOS, HCAHPS), care coordination, medication adherence, discharge check-ins, and preventive care outreach. Sage is multilingual, HIPAA and SOC2 Type 2 compliant, and designed with clinical oversight.

## Market Standing
- **Valuation**: Not disclosed
- **Key Metric**: Annual revenue ~$8.1M (LinkedIn estimate) – not independently verified; total disclosed funding $75.39M (CB Insights) / $79.1M (LinkedIn)
- **Notable Investors/Partners**: Khosla Ventures (lead Series A), Salesforce, Mitsui & Co. Global Investment, AME Cloud Ventures, Colliers, and 30+ other investors. Strategic partnerships with Salesforce (Sage integrated into Salesforce Health Cloud) and NVIDIA (to enhance NLP capabilities).
- **Growth Signals**: Headcount grew 60% YoY to 60 employees; LinkedIn followers +80% YoY. Raised $45M Series A-II in July 2025. Filed 14 patents around psychiatric diagnosis, computational linguistics, and mood disorders.

## Competitive Advantages
- **Emotionally intelligent, conversational AI** that adapts to patient tone and responses, driving higher engagement than rigid IVR systems.
- **Healthcare-native security and compliance**: HIPAA, SOC2 Type 2, end-to-end encryption, and continuous third-party audits.
- **Clinical oversight** built into AI operations, preserving trust and safety.
- **Proprietary acoustic and NLP models** for voice-based screening and monitoring of behavioral health conditions.
- **Pre-built integrations** with EHRs and Salesforce Health Cloud, reducing deployment friction.

## Strategic Focus
- Deepen integration with major health IT ecosystems (Salesforce, EHRs).
- Expand use cases into medication adherence and preventive care.
- Scale the platform through partnerships (Clearstep, NVIDIA) to reach more health plans and systems.
- Continue patenting voice-AI innovations for broader clinical applications.

## Why Work Here
- Opportunity to build cutting-edge AI in a regulated, high-impact industry (healthcare).
- Cross-functional team with backgrounds from top tech and healthcare companies.
- Remote-friendly culture with employees across the US, Canada, India, and Brazil.
- Current open roles (via [Ashby](https://jobs.ashbyhq.com/ellipsis-health)): Frontend Engineer, Backend Engineer, Senior Full-Stack Engineer, Technical Recruiter – Healthcare AI.
- Engineering-led environment with strong emphasis on AI safety, ethical AI, and clinical rigor.

## Sources
1. [ellipsishealth.com](https://ellipsishealth.com/)
2. [cbinsights.com](https://www.cbinsights.com/company/ellipsis-health)
3. [linkedin.com](https://www.linkedin.com/company/ellipsis-health)
4. [ellipsishealth.com/product](https://ellipsishealth.com/product/)
5. [ellipsishealth.com/careers](https://ellipsishealth.com/careers/)

## Other roles at Ellipsis Health

- [Recruiting Coordinator](https://feeny.ai/job/recruiting-coordinator-ellipsis-health-san-francisco-sn7c7xqqqkx6) — San Francisco, CA
- [Senior Backend Engineer, Core Services](https://feeny.ai/job/senior-backend-engineer-core-services-ellipsis-health-san-francisco-s29qrpgwbm29) — San Francisco, CA
- [Senior Data Scientist](https://feeny.ai/job/senior-data-scientist-ellipsis-health-san-francisco-nvrrg0sesgcw) — San Francisco, CA
- [Product Manager, Analytics & Reporting](https://feeny.ai/job/product-manager-analytics-reporting-ellipsis-health-san-francisco-v6m6tvdz8wvw) — San Francisco, CA
- [Director of Customer Success](https://feeny.ai/job/director-of-customer-success-ellipsis-health-san-francisco-zrfs215v1wfk) — San Francisco, CA
- [VP Provider Sales](https://feeny.ai/job/vp-provider-sales-ellipsis-health-san-francisco-vf5z5zaxjgnz) — San Francisco, CA
- [VP Payer Sales](https://feeny.ai/job/vp-payer-sales-ellipsis-health-san-francisco-azsrb3xdz1ac) — San Francisco, CA
- [Senior Data Platform Engineer](https://feeny.ai/job/senior-data-platform-engineer-ellipsis-health-san-francisco-40ktf7p6qs16) — San Francisco, CA
- [Senior Backend Engineer, Infrastructure](https://feeny.ai/job/senior-backend-engineer-infrastructure-ellipsis-health-san-francisco-wmz4tk6r9m3p) — San Francisco, CA
- [Forward Deployed Product Manager, AI Assistant](https://feeny.ai/job/forward-deployed-product-manager-ai-assistant-ellipsis-health-united-states-hkxfbzk5dzvk) — United States