---
title: 'Multimodal AI Model Optimization Research Engineer at Tavus'
canonical: 'https://feeny.ai/job/multimodal-ai-model-optimization-research-engineer-tavus-00y9w8ra7g66'
type: 'job'
last_seen: '2026-10-15'
---

# Multimodal AI Model Optimization Research Engineer at Tavus

- **Company:** Tavus
- **Employment:** full-time
- **Work type:** hybrid
- **Posted:** 2026-09-28
- **Last confirmed live:** 2026-10-15
- **Apply:** https://jobs.ashbyhq.com/tavus/08ba99c2-3cc2-4d7f-8c4c-b16a1900a88d

## Job description

## About Us

At Tavus, we're building the human layer of AI. Our mission is to make human-AI interaction as natural as face-to-face interaction, enabling the human touch where it has been previously unscalable. We achieve this through pioneering research in multimodal AI for modeling human-to-human communication (language, audio, and video), as well as generating audio-visual avatar behavior. Our models power everything from text-to-video AI avatars to real-time conversational video experiences across industries like healthcare, recruiting, sales, and education.

By enabling AI to see, hear, and communicate with human-like authenticity, we're creating the foundation for the next generation of AI employees, assistants, and companions.

We are a Series B company backed by top investors, including Sequoia, Y Combinator, and Scale VC. Join us in driving the future of human-AI interaction.

## The Role

We’re looking for an experienced Research Scientist/Engineer with a focus on model optimization to join our core AI team. Our ideal partner-in-crime thrives in startup environments, is comfortable prioritizing independently, and is willing to take calculated risks. We’re moving fast and looking for people who can help pave the path.

## Your Mission

- Take cutting-edge research models and make them fast, efficient, and production-ready using sparsification, distillation, and quantization
- Own the optimization lifecycle for key models: define metrics, run experiments, and benchmark trade-offs across latency, cost, and quality
- Partner closely with researchers and engineers to turn new ideas into deployable systems

## Requirements

- Strong experience in deep learning using PyTorch
- Hands-on experience with model optimization and compression, including knowledge distillation, pruning/sparsification, quantization, and mixed precision
- Understanding of efficient architectures such as low-rank adapters
- Strong understanding of inference performance and GPU/accelerator fundamentals
- Strong Python coding skills and reliable research engineering practices
- Experience working with large models and datasets in cloud environments
- Ability to read ML papers, reproduce results, and adapt ideas
- Clear communication and collaboration skills

## Preferred Experience

- Optimization of diffusion models, video/audio generative models, or large language models
- Experience with real-time or streaming systems (low-latency APIs, WebRTC, streaming TTS/video)
- Familiarity with TensorRT, ONNX Runtime, TVM, Triton, or XLA
- Experience writing custom Triton/CUDA kernels or low-level performance tuning
- Experience with experiment tracking, benchmarking, and profiling at scale
- Prior experience in research engineering or applied science roles

Location This position is preferably hybrid in San Francisco, with relocation support offered. Remote candidates are also considered.

## Benefits

When you join Tavus, you’re joining a family. We offer flexible work schedules, unlimited PTO, competitive healthcare and gear stipends, and a collaborative environment focused on learning and impact.

Culture & Diversity We are not looking for cultural fits — we are looking for culture creators. Diversity drives our success, and we combine varied backgrounds, skills, and perspectives to build the best experiences for our clients..

## About Tavus

## Company Overview
- **One-liner**: Tavus is an AI research lab and platform pioneering "Human Computing," building models and APIs that enable AI to see, hear, understand, and respond with lifelike realism in real-time.
- **Entity Type**: Private (Venture-backed)
- **Headquarters**: San Francisco, CA, United States
- **Founded**: 2021
- **Founders**: Hassaan Raza & Quinn Favret

## Core Business
- **Primary industry**: Artificial Intelligence / Human Computing / Multi-modal AI
- **Target customers**: B2B; Product teams and developers (via API and no-code builder), enterprise teams (for onboarding, training, support), and prosumers/individuals (for tutors, companions, assistants).
- **Mission**: "To make human-AI interaction as natural as face-to-face interaction, enabling the human touch where it has been previously unscalable." [ycombinator.com](https://www.ycombinator.com/companies/tavus)

## Products & Services
- **Conversational Video Interface (CVI)**: An API-first platform that allows developers to embed real-time, face-to-face conversational AI into any product. It enables emotionally intelligent, adaptive interactions with sub-second latency.
- **Tavus Platform (No-Code SaaS)**: A no-code builder that lets individuals and enterprises create custom AI humans for use in onboarding, training, customer engagement, or personal use by customizing personality, knowledge, and appearance.
- **Replica API**: Tools for creating personal or stock digital twins (replicas) with studio-grade fidelity, full-face rendering, and identity preservation. Supports both individual and large-scale deployment.
- **Personas**: A system for defining custom personalities and context to tailor the behavior of AI humans.
- **Core AI Models**:
    - **Sparrow (Conversational Flow Model)**: Delivers low-latency, accurate turn-taking by analyzing lexical, semantic, prosodic, and acoustic cues.
    - **Raven (Perception Model)**: A multi-modal perception engine that translates facial expressions, tone, gaze, and emotion into conversational signals.
    - **Phoenix (Rendering Model)**: A gaussian-diffusion based model for high-fidelity, photorealistic facial rendering with contextually accurate emotions and expressions in real time. [tavus.io](https://www.tavus.io/)

## Market Standing
- **Valuation/Market Cap**: Not publicly disclosed.
- **Key Metric**: **Total Funding**: $40M+ (including a $18M raise in 2024 and a subsequent Series B for $40M in 2025). [ycombinator.com](https://www.ycombinator.com/companies/tavus), [techcrunch.com](https://techcrunch.com)
- **Notable Investors/Partners**: Scale Venture Partners, Sequoia Capital, Y Combinator (Summer 2021 batch), HubSpot. [tavus.io](https://www.tavus.io/lp/ai-info-page)
- **Growth Signals**: 
    - Demonstrated product-market fit with case studies for clients like Final Round AI (1.2M practice minutes logged), ACTO (life sciences training), and CareFlick (senior companionship).
    - Raising significant venture capital over multiple rounds, indicating strong investor confidence and a long runway for R&D.

## Competitive Advantages
- **Real-time realism**: Overcomes the "uncanny valley" with sub-second latency and photorealistic rendering, a key differentiator from asynchronous video-generation and static avatar companies.
- **Comprehensive AI Human Stack**: Offers a full operating system (perception, memory, knowledge, action) rather than just a single component like an avatar.
- **Flexibility**: Provides multiple integration paths—a modular API, a bring-your-own-LLM option, and a no-code builder—making it accessible to both technical and non-technical users.
- **Ethical foundation**: Built with a stated commitment to privacy, consent, and empathy, and aims for transparency in disclosure (e.g., SOC 2 and HIPAA compliance).

## Strategic Focus
- **Category Creation**: Actively establishing "Human Computing" as a new industry standard for human-AI interaction, moving beyond simple chatbots and scripted avatars.
- **Developer Ecosystem**: Centered on an API-first strategy to become the platform of choice for developers building the next generation of AI assistants, companions, and employees.
- **Real-time & Multi-modal**: Continuing to push the boundaries of multi-modal AI (video, audio, text, emotion) to create more intuitive and authentic interactions.
- **Enterprise & Prosumer Scale**: Growing the no-code platform to serve both enterprise-scale training/support and personal use cases (tutoring, wellness, companionship).

## Why Work Here
- **Cutting-edge Research & Impact**: A chance to work on frontier multi-modal AI models (perception, rendering, conversation) that directly shape how humans interact with machines.
- **Culture**: Described as a tight-knit, high-output team with semi-annual full-team retreats and a focus on continuous learning alongside AI leaders. [tavus.io](https://www.tavus.io/careers)
- **Compensation & Benefits**: Offers competitive salary, meaningful equity, fully covered medical, dental, and vision for employees and their families, unlimited PTO, and a gear stipend for setting up the workspace. [tavus.io](https://www.tavus.io/careers)
- **Work Environment**: "Our team moves mountains to bring the vision to life, and we’ve got their backs." The company appears to embrace a hybrid/office culture in San Francisco (SOMA) with remote options for some roles (e.g., Senior Software Engineer). [ycombinator.com](https://www.ycombinator.com/companies/tavus)
- **Team Growth**: Team size reported as around 40 members. Currently hiring for roles like AI Researcher (Perception), Customer Engineer, Forward Deployed Engineer, and Solution Engineer. [ycombinator.com](https://www.ycombinator.com/companies/tavus), [jobs.ashbyhq.com](https://jobs.ashbyhq.com/tavus)

## Sources
1. [tavus.io](https://www.tavus.io/)
2. [tavus.io/careers](https://www.tavus.io/careers)
3. [tavus.io/lp/ai-info-page](https://www.tavus.io/lp/ai-info-page)
4. [ycombinator.com](https://www.ycombinator.com/companies/tavus)
5. [jobs.ashbyhq.com](https://jobs.ashbyhq.com/tavus)
6. [techcrunch.com](https://techcrunch.com)

## Other roles at Tavus

- [Talent](https://feeny.ai/job/talent-tavus-san-francisco-g82ww3x86vte) — San Francisco, CA
- [Solutions Engineer](https://feeny.ai/job/solutions-engineer-tavus-san-francisco-373tkvjzm1gx) — San Francisco, CA
- [Head of Marketing](https://feeny.ai/job/head-of-marketing-tavus-san-francisco-6ej7vq5vp0tj) — San Francisco, CA
- [Growth Marketer](https://feeny.ai/job/growth-marketer-tavus-san-francisco-e6prxf9hbt0c) — San Francisco, CA
- [Senior Software Engineer (CVI)](https://feeny.ai/job/senior-software-engineer-cvi-tavus-san-francisco-0xy8c3dnhxe0) — San Francisco, CA
- [Vibe Growth Marketer](https://feeny.ai/job/vibe-growth-marketer-tavus-san-francisco-ynha49h58qg6) — San Francisco, CA
- [Business Development Representative](https://feeny.ai/job/business-development-representative-tavus-san-francisco-vaebfha9r4ex) — San Francisco, CA
- [Product Designer](https://feeny.ai/job/product-designer-tavus-san-francisco-813z9bn38tfr) — San Francisco, CA
- [Brand & Product Designer](https://feeny.ai/job/brand-product-designer-tavus-san-francisco-8ph3kwhaykea) — San Francisco, CA
- [Product Manager](https://feeny.ai/job/product-manager-tavus-san-francisco-w8gfpeca4wqe) — San Francisco, CA