---
title: 'AI Engineer at LiteLLM'
canonical: 'https://feeny.ai/job/ai-engineer-litellm-san-francisco-v2eczyy9ah6c'
type: 'job'
last_seen: '2026-09-09'
---

# AI Engineer at LiteLLM

- **Company:** LiteLLM
- **Location:** San Francisco, CA
- **Compensation:** $200k–$250k
- **Employment:** full-time
- **Work type:** onsite
- **Posted:** 2026-07-30
- **Last confirmed live:** 2026-09-09
- **Apply:** https://jobs.ashbyhq.com/litellm/6e025e39-6f8a-46bd-91f7-8784d1f5076b

## Job description

LiteLLM is the world's most popular AI Gateway, trusted by companies like Adobe, Netflix, and NASA. We're building the next generation of infrastructure for AI applications—and now we're building our second product.

## What you will be working on

Skills: Python, LLM APIs, FastAPI As the AI Engineer, you'll be responsible for ensuring LiteLLM unifies the format for calling LLM APIs in the broader OpenAI + Anthropic spec. This involves writing transformations to convert API requests from OpenAI/Anthropic spec to various LLM provider formats, building provider-agnostic unification functionality (e.g. session management across non-openai models for /v1/responses API, etc.). You'll work directly with the CEO and CTO on critical projects including:

- System design for supporting provider level features at the gateway level (skills, compaction, batches, etc.)
- Thinking about the developer experience - to support millions of users who leverage your work via LiteLLM's Python SDK
- Building across LLM's, MCP's, and Agents by maintaining an excellent LLM interoperability layer, supporting a wide range of MCP Auth flows and enabling various Agent use-cases

What is our tech stack The tech stack includes Python, FastAPI, Redis, Postgres.

## Who we are looking for

- 1-2 years of backend/full-stack experience with production systems
- Passion for open source and user engagement
- Experience working with the OpenAI api (understand the difference between /chat/completions and /responses, and can speak to API-specific nuances)
- Strong work ethic and ability to thrive in small teams
- Eagerness to talk to users and help solve real problems

Why Join LiteLLM?

- Work directly with the founders every day.
- Build products used by thousands of engineering teams around the world.
- Own meaningful product decisions—not just implementation.
- Move incredibly fast and see your work in production immediately.
- Help define the future of AI infrastructure and developer tooling.
- Competitive salary, equity, health, dental, and vision benefits.

## About LiteLLM

LiteLLM is the world's most widely adopted AI Gateway. We provide a unified API for 100+ LLM providers along with routing, authentication, budgets, observability, and governance for production AI systems. Thousands of companies rely on LiteLLM to power AI in production. If you love talking to users, building products, and shipping software that people actually use, we'd love to meet you.

## About LiteLLM

## Company Overview
- **One-liner**: LiteLLM provides an open-source AI gateway and Python SDK that unifies access to 100+ large language models (LLMs) with built-in cost tracking, guardrails, and load balancing.
- **Entity Type**: Private (Seed stage, Y Combinator W23)
- **Headquarters**: San Francisco, California, United States
- **Founded**: 2023
- **Founders**: Krrish Dholakia (CEO) and Ishaan Jaffer (CTO)

## Core Business
- **Primary industry**: AI infrastructure / LLM operations (LLMOps)
- **Target customers**: Platform engineering teams, ML platform teams, and developers building LLM-powered applications (B2B, mid-market to enterprise)
- **Mission or purpose**: To simplify model access, spend tracking, and fallbacks across 100+ LLMs, enabling organizations to manage and scale their entire AI program from a single governed platform.

## Products & Services
- **[AI Gateway (Proxy Server)](https://www.litellm.ai/)**: A centralized service that provides an OpenAI-compatible API, virtual keys, spend tracking, rate limiting, guardrails, load balancing, and an admin dashboard. Deployable on-premises or via Terraform modules on AWS/GCP.
- **[Python SDK](https://github.com/BerriAI/litellm)**: A lightweight library for direct integration, supporting Router with retry/fallback logic, application-level cost tracking, and observability callbacks (Langfuse, MLflow, etc.).
- **[MCP Gateway](https://github.com/BerriAI/litellm)**: Connect MCP (Model Context Protocol) servers to any LLM through the gateway.
- **[A2A Agent Gateway](https://github.com/BerriAI/litellm)**: Invoke A2A (Agent-to-Agent) agents from supported providers such as LangGraph, Vertex AI Agent Engine, and Azure AI Foundry.

## Market Standing
- **Valuation/Market Cap**: Not disclosed
- **Key Metric**: Total funding raised: **$625,000** across multiple seed rounds (last round June 2023); Annual revenue reported as **$1.6M** [linkedin.com](https://www.linkedin.com/company/litellm)
- **Notable Investors/Partners**: Y Combinator (backed, batch W23), with additional undisclosed seed investors [ycombinator.com](https://www.ycombinator.com/companies/litellm/jobs)
- **Growth Signals**:
  - 52,568 GitHub stars, 9,450 forks, 1,005+ contributors [github.com](https://github.com/BerriAI/litellm)
  - 240M+ Docker pulls, 1B+ requests served, 80% uptime (claimed on homepage)
  - 14 employees with **27.3% monthly headcount growth** [linkedin.com](https://www.linkedin.com/company/litellm)
  - Used by companies such as **Netflix, Adobe, Twilio, Siemens, Rocket Money** [github.com](https://github.com/BerriAI/litellm) [ycombinator.com](https://www.ycombinator.com/companies/litellm/jobs)

## Competitive Advantages
- **Unified API**: Single interface for 100+ LLM providers (OpenAI, Anthropic, Gemini, Bedrock, Azure, etc.) using the OpenAI format – drop-in replacement for existing OpenAI code.
- **Production-ready gateway**: Out-of-the-box virtual keys, spend tracking, guardrails, load balancing, and admin dashboard.
- **Performance**: 8ms P95 latency at 1k requests per second (benchmarked) [github.com](https://github.com/BerriAI/litellm)
- **Open source + enterprise**: Self-hosted, air-gapped, and air-gapped deployment options; commercial license for advanced features (SSO, audit logs, custom SLAs).
- **Active community**: 1,387 releases, frequent updates, strong contributor ecosystem.

## Strategic Focus
- **Rust migration**: LiteLLM is actively migrating core components to Rust for improved performance and reliability [litellm.ai](https://www.litellm.ai/).
- **Enterprise expansion**: Building out enterprise features (JWT auth, SSO, audit logs, custom SLAs) and dedicated support for large organizations [github.com](https://github.com/BerriAI/litellm).
- **Agent and MCP support**: Extending the gateway to support A2A agents and MCP servers, positioning LiteLLM as a hub for multi-model, multi-agent workflows.

## Why Work Here
- **Stage & impact**: Early-stage YC startup with high growth (10–14 people, rapidly expanding). Work directly with founders and shape the product from the ground up [ycombinator.com](https://www.ycombinator.com/companies/litellm/jobs).
- **Engineering culture**: Open-source first, with strong emphasis on code quality (Black, Ruff, MyPy, circular import checks). Contributors are active and valued [github.com](https://github.com/BerriAI/litellm).
- **Remote & hybrid options**: Some roles are San Francisco–based (office at 1730 Jones St), others offer remote within the US or Mexico. Equity packages range from 0.05% to 3% depending on role [ycombinator.com](https://www.ycombinator.com/companies/litellm/jobs).
- **Compensation**: Salary ranges vary by role (e.g., Founding Reliability Engineer: $200K–$270K + 0.25%–0.75% equity; Backend Engineer: $150K–$200K + 0.25%–0.75% equity; interns $80K–$100K) [ycombinator.com](https://www.ycombinator.com/companies/litellm/jobs).
- **Team composition**: 36% technical, 21% sales, 14% general management – small, cross-functional team with founders actively coding.

## Sources
1. [litellm.ai](https://www.litellm.ai/)
2. [linkedin.com](https://www.linkedin.com/company/litellm)
3. [ycombinator.com](https://www.ycombinator.com/companies/litellm/jobs)
4. [github.com](https://github.com/BerriAI/litellm)
5. [jobs.ashbyhq.com/litellm](https://jobs.ashbyhq.com/litellm) (careers page, but no additional detail beyond job listings)

## Other roles at LiteLLM

- [Business Manager](https://feeny.ai/job/business-manager-litellm-united-states-rggyetyqw7ka) — United States
- [Account Executive](https://feeny.ai/job/account-executive-litellm-san-francisco-mv5knsvgbbc5) — San Francisco, CA
- [Forward Deployed Engineer (New Grad)](https://feeny.ai/job/forward-deployed-engineer-new-grad-litellm-united-states-e0c74w49szkf) — United States
- [Technical Account Manager](https://feeny.ai/job/technical-account-manager-litellm-united-states-63aqttbmc0f9) — United States
- [Forward Deployed Engineer](https://feeny.ai/job/forward-deployed-engineer-litellm-united-states-dxh3t099p93f) — United States
- [Founding Developer Relations](https://feeny.ai/job/founding-developer-relations-litellm-san-francisco-fe4fwf2gpysb) — San Francisco, CA
- [Senior Support Engineer - India](https://feeny.ai/job/senior-support-engineer-india-litellm-india-wjm52a9f4vmf) — India
- [Product Engineer](https://feeny.ai/job/product-engineer-litellm-san-francisco-33g1q944rcqm) — San Francisco, CA
- [Senior Support Engineer](https://feeny.ai/job/senior-support-engineer-litellm-san-francisco-z0gy525hv7zc) — San Francisco, CA
- [Rust Engineer](https://feeny.ai/job/rust-engineer-litellm-san-francisco-1x6vp82rtvz6) — San Francisco, CA