---
title: 'ML Engineer: Speech & LLMs at Knowtex'
canonical: 'https://feeny.ai/job/ml-engineer-speech-llms-knowtex-san-francisco-saqmn01f7k8e'
type: 'job'
last_seen: '2026-09-11'
---

# ML Engineer: Speech & LLMs at Knowtex

- **Company:** Knowtex
- **Location:** San Francisco, CA
- **Employment:** full-time
- **Work type:** hybrid
- **Posted:** 2026-08-17
- **Last confirmed live:** 2026-09-11
- **Apply:** https://jobs.ashbyhq.com/knowtex/501ccbb9-72ec-40e6-8c6c-2b4e8b837eab

## Job description

## About Knowtex

Knowtex is building the future of voice AI operating systems for clinicians, transforming how healthcare documentation happens at the point of care. We are experiencing rapid growth across both commercial health systems and federal healthcare, with our ambient documentation platform scaling to thousands of clinicians across hundreds of specialties. We are at an inflection point where advances in speech, language models, and clinical AI can fundamentally change how clinicians interact with technology, giving them more time to focus on what matters most: their patients.

Position Overview We are hiring two ML Engineers / Researchers to help build the next generation of Knowtex's AI stack. We are looking for researchers with deep expertise in one of two areas:

- Speech & Audio: Build state-of-the-art medical speech-to-text systems using our large proprietary dataset of real-world clinical audio, with the goal of bringing more of our speech stack in-house.
- Large Language Models: Develop and optimize models for clinical documentation and structured clinical reasoning, improving quality, cost, latency, and control.

You do not need to be an expert in both areas. We are looking for exceptional depth in either speech/audio modeling or LLMs. These are research-heavy roles with a direct path to production. You will design experiments, build datasets and evaluation systems, train and fine-tune models, and work closely with engineering and clinical teams to deploy successful approaches at scale. This role plays a central part in defining Knowtex's long-term ML strategy.

## Key Responsibilities

Speech & Audio

- Develop and train speech recognition models optimized for medical conversations across hundreds of specialties
- Leverage Knowtex's large proprietary clinical audio dataset to train and fine-tune domain-specific speech models
- Research approaches for improving medical terminology recognition, speaker attribution, punctuation, timestamps, and robustness across accents and clinical environments
- Build rigorous speech evaluation frameworks beyond traditional WER, including medical terminology and clinically significant error measurement
- Explore modern speech architectures, self-supervised learning, speech foundation models, and audio-language models
- Optimize models for low-latency, real-time inference at production scale

Large Language Models

- Develop and optimize models for generating high-quality clinical documentation, including SOAP notes and specialty-specific note formats
- Build models for downstream clinical tasks such as medication extraction, orders, ICD-10 coding, E&M coding, patient visit summaries, and other structured clinical artifacts
- Evaluate open-weight and proprietary model architectures and determine where fine-tuning, distillation, structured generation, or task-specific models can outperform general-purpose API-based approaches
- Fine-tune and post-train models using Knowtex's proprietary clinical datasets
- Develop rigorous evaluation frameworks for clinical accuracy, hallucinations, completeness, formatting, and clinician preferences
- Research approaches for reducing inference cost and latency while maintaining or improving clinical quality

Across Both Tracks

- Move quickly from idea → dataset → experiment → evaluation → production
- Design experiments that clearly measure whether an approach improves real-world clinical outcomes
- Build datasets, benchmarks, and evaluation infrastructure that make model improvements measurable and reproducible
- Collaborate closely with clinicians, applied ML engineers, and platform engineers
- Take successful research beyond prototypes and help deploy models into production
- Balance model quality with latency, inference cost, reliability, and scalability

Required Qualifications

- 2+ years of experience in machine learning research or ML engineering, with deep expertise in speech/audio modeling or large language models
- Strong expertise in Python and PyTorch
- Deep understanding of modern transformer architectures and model training techniques
- Experience training, fine-tuning, or post-training large neural models
- Strong experimental methodology and ability to independently design and execute research projects
- Experience working with large-scale datasets and distributed training environments
- Ability to translate research results into production systems
- Strong understanding of model evaluation and benchmarking
- Bachelor’s, Master’s, or PhD in Computer Science, Machine Learning, or a related technical field, or equivalent research experience

## Preferred Qualifications

For Speech Researchers

- Deep experience with automatic speech recognition (ASR)
- Experience training or fine-tuning Whisper, Conformer, wav2vec, or similar speech architectures
- Experience with large-scale audio datasets and speech data pipelines
- Familiarity with speaker diarization, voice activity detection, streaming ASR, or audio-language models
- Experience optimizing speech models for real-time inference

For LLM Researchers

- Experience fine-tuning or post-training open-weight LLMs
- Experience with supervised fine-tuning, distillation, preference optimization, or reinforcement learning
- Experience building LLM evaluation systems and model benchmarks
- Experience serving and optimizing open-weight models at scale
- Experience with structured generation, tool use, or agentic systems

For Either Track

- Experience in healthcare AI, clinical NLP, or medical speech
- Familiarity with clinical documentation workflows and medical terminology
- Knowledge of coding systems such as ICD-10, CPT, E&M, or SNOMED
- Publications at leading ML, NLP, or speech conferences
- Experience deploying ML systems in HIPAA-compliant or regulated environments
- Experience working in fast-moving startup environments where researchers own projects from experimentation through production

Technical Environment

- AWS
- Python, PyTorch
- Transformer-based LLM and speech architectures
- Open-weight and frontier language models
- Large-scale clinical audio and text datasets
- Distributed model training and inference
- GPU-based model serving and optimization
- Real-time speech and clinical AI pipelines
- Structured clinical evaluation and benchmarking infrastructure

## Compensation & Benefits

- Competitive salary
- Meaningful equity compensation
- Unlimited PTO
- Premium health, dental, and vision coverage
- 401(k) plan
- Work model: Hybrid In-person

## About Knowtex

## Company Overview
- **One-liner**: Knowtex provides an ambient clinical AI platform that automates documentation, coding, and workflow tasks by analyzing doctor-patient conversations in real time.
- **Entity Type**: Private (startup; Y Combinator Summer 2022 batch)
- **Headquarters**: San Francisco, CA, USA
- **Founded**: 2022
- **Founders**: Caroline Zhang (CEO) and Jocelyn Kang (CTO)

## Core Business
- Primary industry: Healthcare AI / Ambient Clinical Intelligence
- Target customers: Enterprise health systems, federal agencies (e.g., U.S. Department of Veterans Affairs), and large medical practices (B2B, enterprise, government)
- Mission: Leverage AI and voice technology to solve inefficiencies and revenue leakage in healthcare, freeing clinicians from documentation burden so they can focus on patient care.

## Products & Services
- **Knowtex Ambient Clinical AI Platform**: A HIPAA-compliant, EHR-agnostic platform that listens to doctor-patient conversations and automatically generates accurate clinical notes, ICD-10/E&M codes, orders, and other documentation. Tailored to over 200 medical specialties. Includes an admin dashboard for operational oversight, performance monitoring, and AI governance.

## Market Standing
- **Valuation/Market Cap**: Not publicly available (private company)
- **Key Metric**: Total funding not disclosed; notable backing includes Y Combinator, Amazon Web Services, UCSF Rosenman Institute, and MedTech Innovators. In 2024, awarded a $15 million contract with the U.S. Department of Veterans Affairs.
- **Notable Investors/Partners**: Y Combinator, Amazon Web Services, UCSF Rosenman Institute, MedTech Innovators, 4D EMR (partnership)
- **Growth Signals**: 
  - Selected by the U.S. Department of Veterans Affairs (VA) for deployment across the nation’s largest integrated health system (170 medical centers, 1,193 outpatient clinics). [knowtex.ai](https://www.knowtex.ai/resources/knowtex-selected-by-u-s-department-of-veterans-affairs-to-deploy-ambient-clinical-ai)
  - Platform has automated over 100 million workflows, supports 1,000+ clinicians, and covers 200+ specialties.
  - Reports 90% decrease in administrative manual work and 10% increase in ROI from improved code capture and billing error prevention. [knowtex.ai](https://www.knowtex.ai/)
  - Achieved SOC 2 Type I, ISO 27001, and HIPAA compliance; features explainable/transparent AI.

## Competitive Advantages
- **Specialty-specific, EHR-agnostic**: Models are tailored to individual medical specialties and integrate with any electronic health record system, avoiding a one-size-fits-all approach.
- **Enterprise & Federal Readiness**: Compliance with HIPAA, SOC 2, ISO 27001, GDPR; built for scale with strong PHI protection and audited security.
- **Transparent, Explainable AI**: Platform provides structured benchmarking, continuous monitoring, and evaluation standards, making AI decisions measurable and trustworthy for clinicians.
- **VA Partnership**: A top-3 winner of the VA’s 2024 AI Tech Sprint for Ambient Scribe and a $15M contract validates the solution at federal scale.

## Strategic Focus
- Scaling the platform across the VA health system (national rollout starting October 2025) and expanding federal partnerships.
- Deepening specialty-specific model performance (200+ specialties) and expanding into new clinical domains.
- Building an integrated AI infrastructure that unifies automation, oversight, and intelligence for health systems.
- Advancing transparent AI evaluation to increase clinician trust and adoption.

## Why Work Here
- **Culture**: According to the careers page, the team values curiosity, ownership, and thoughtful collaboration. Employees work alongside clinicians, engineers, and researchers tackling complex healthcare challenges. [knowtex.ai/careers](https://www.knowtex.ai/careers)
- **Impact**: Directly improve how care is delivered by reducing clinician burnout and administrative burden. Technology is used by the VA and large health systems.
- **Location**: Headquarters in San Francisco, CA. Hybrid/remote policy not explicitly stated, but roles listed on Ashby include San Francisco as location.
- **Team**: Women-founded (Caroline Zhang, Jocelyn Kang) by Stanford AI scientists, with backgrounds in biomedical research, investment banking, and prior startups.
- **Perks**: Not explicitly listed, but as an early-stage YC company, employees likely get significant ownership and growth opportunities.

## Sources
1. [knowtex.ai](https://www.knowtex.ai/)
2. [knowtex.ai/careers](https://www.knowtex.ai/careers)
3. [ycombinator.com/companies/knowtex](https://www.ycombinator.com/companies/knowtex)
4. [knowtex.ai/resources/knowtex-selected-by-u-s-department-of-veterans-affairs-to-deploy-ambient-clinical-ai](https://www.knowtex.ai/resources/knowtex-selected-by-u-s-department-of-veterans-affairs-to-deploy-ambient-clinical-ai)
5. [linkedin.com/company/knowtexai](https://www.linkedin.com/company/knowtexai)

## Other roles at Knowtex

- [Integration Engineer](https://feeny.ai/job/integration-engineer-knowtex-san-francisco-cfh0y7dv8grn) — San Francisco, CA
- [Engineer Manager, Platform](https://feeny.ai/job/engineer-manager-platform-knowtex-san-francisco-rke1tj90p3nb) — San Francisco, CA
- [Customer Support Program Manager](https://feeny.ai/job/customer-support-program-manager-knowtex-united-states-43sx7n1yzx9v) — United States
- [Clinical Implementation Specialist](https://feeny.ai/job/clinical-implementation-specialist-knowtex-united-states-7p59qxzkvrjq) — United States
- [Join Our Talent Community](https://feeny.ai/job/join-our-talent-community-knowtex-united-states-q25ef9gyp2cd) — United States