---
title: 'Product Engineer at LiteLLM'
canonical: 'https://feeny.ai/job/product-engineer-litellm-san-francisco-33g1q944rcqm'
type: 'job'
last_seen: '2026-09-09'
---

# Product Engineer at LiteLLM

- **Company:** LiteLLM
- **Location:** San Francisco, CA
- **Compensation:** $200k–$260k
- **Employment:** full-time
- **Work type:** onsite
- **Posted:** 2026-07-10
- **Last confirmed live:** 2026-09-09
- **Apply:** https://jobs.ashbyhq.com/litellm/8f1fe19b-7f83-4266-a04f-782dedb2c170

## Job description

Product Engineer LiteLLM is the world's most popular AI Gateway, trusted by companies like Adobe, Netflix, and NASA. We're building the next generation of infrastructure for AI applications—and now we're building our second product. We're looking for a Product Engineer who wants to work directly with the founders to figure out what developers need, ship quickly, and build products that thousands of engineering teams use every day.

## About the Role

This isn't a traditional product management role, and it isn't a traditional software engineering role. You'll spend your time talking to customers, understanding their problems, designing solutions, writing production code, and shipping features. One week you might build a new authentication flow, the next week you're redesigning an onboarding experience, integrating with a new provider, or launching an entirely new product. If you enjoy owning a problem end-to-end—from idea to production—you'll thrive here.

## Responsibilities

- Build new product features from concept to production.
- Work directly with customers to understand pain points and validate solutions.
- Ship quickly while maintaining a high quality bar.
- Prototype new ideas and iterate based on user feedback.
- Own major product areas and continuously improve them over time.
- Design backend APIs and services using Python and FastAPI.
- Build intuitive developer experiences across our API, dashboard, and CLI.
- Collaborate directly with the founders on product direction and technical decisions.
- Help define and build LiteLLM's second major product.

## What We're Looking For

- Bachelor's or Master's degree in Computer Science or a related field.
- Strong software engineering fundamentals.
- Experience building production software, preferably with Python.
- Comfortable moving across the stack when needed (backend, frontend, infrastructure, APIs).
- Strong product intuition and curiosity about how customers use software.
- Excellent written and verbal communication.
- High ownership—you naturally identify problems and solve them without waiting to be asked.
- Comfortable operating in ambiguity and making decisions with incomplete information.
- Excited by shipping quickly and learning from real users.

Bonus Points

- Experience at an early-stage startup.
- Experience building developer tools or infrastructure products.
- Open source contributions.
- You've built and launched your own product.
- Familiarity with AI infrastructure, LLMs, or developer platforms.

Why Join LiteLLM?

- Work directly with the founders every day.
- Build products used by thousands of engineering teams around the world.
- Own meaningful product decisions—not just implementation.
- Move incredibly fast and see your work in production immediately.
- Help define the future of AI infrastructure and developer tooling.
- Competitive salary, equity, health, dental, and vision benefits.

## About LiteLLM

LiteLLM is the world's most widely adopted AI Gateway. We provide a unified API for 100+ LLM providers along with routing, authentication, budgets, observability, and governance for production AI systems. Thousands of companies rely on LiteLLM to power AI in production. If you love talking to users, building products, and shipping software that people actually use, we'd love to meet you.

## About LiteLLM

## Company Overview
- **One-liner**: LiteLLM provides an open-source AI gateway and Python SDK that unifies access to 100+ large language models (LLMs) with built-in cost tracking, guardrails, and load balancing.
- **Entity Type**: Private (Seed stage, Y Combinator W23)
- **Headquarters**: San Francisco, California, United States
- **Founded**: 2023
- **Founders**: Krrish Dholakia (CEO) and Ishaan Jaffer (CTO)

## Core Business
- **Primary industry**: AI infrastructure / LLM operations (LLMOps)
- **Target customers**: Platform engineering teams, ML platform teams, and developers building LLM-powered applications (B2B, mid-market to enterprise)
- **Mission or purpose**: To simplify model access, spend tracking, and fallbacks across 100+ LLMs, enabling organizations to manage and scale their entire AI program from a single governed platform.

## Products & Services
- **[AI Gateway (Proxy Server)](https://www.litellm.ai/)**: A centralized service that provides an OpenAI-compatible API, virtual keys, spend tracking, rate limiting, guardrails, load balancing, and an admin dashboard. Deployable on-premises or via Terraform modules on AWS/GCP.
- **[Python SDK](https://github.com/BerriAI/litellm)**: A lightweight library for direct integration, supporting Router with retry/fallback logic, application-level cost tracking, and observability callbacks (Langfuse, MLflow, etc.).
- **[MCP Gateway](https://github.com/BerriAI/litellm)**: Connect MCP (Model Context Protocol) servers to any LLM through the gateway.
- **[A2A Agent Gateway](https://github.com/BerriAI/litellm)**: Invoke A2A (Agent-to-Agent) agents from supported providers such as LangGraph, Vertex AI Agent Engine, and Azure AI Foundry.

## Market Standing
- **Valuation/Market Cap**: Not disclosed
- **Key Metric**: Total funding raised: **$625,000** across multiple seed rounds (last round June 2023); Annual revenue reported as **$1.6M** [linkedin.com](https://www.linkedin.com/company/litellm)
- **Notable Investors/Partners**: Y Combinator (backed, batch W23), with additional undisclosed seed investors [ycombinator.com](https://www.ycombinator.com/companies/litellm/jobs)
- **Growth Signals**:
  - 52,568 GitHub stars, 9,450 forks, 1,005+ contributors [github.com](https://github.com/BerriAI/litellm)
  - 240M+ Docker pulls, 1B+ requests served, 80% uptime (claimed on homepage)
  - 14 employees with **27.3% monthly headcount growth** [linkedin.com](https://www.linkedin.com/company/litellm)
  - Used by companies such as **Netflix, Adobe, Twilio, Siemens, Rocket Money** [github.com](https://github.com/BerriAI/litellm) [ycombinator.com](https://www.ycombinator.com/companies/litellm/jobs)

## Competitive Advantages
- **Unified API**: Single interface for 100+ LLM providers (OpenAI, Anthropic, Gemini, Bedrock, Azure, etc.) using the OpenAI format – drop-in replacement for existing OpenAI code.
- **Production-ready gateway**: Out-of-the-box virtual keys, spend tracking, guardrails, load balancing, and admin dashboard.
- **Performance**: 8ms P95 latency at 1k requests per second (benchmarked) [github.com](https://github.com/BerriAI/litellm)
- **Open source + enterprise**: Self-hosted, air-gapped, and air-gapped deployment options; commercial license for advanced features (SSO, audit logs, custom SLAs).
- **Active community**: 1,387 releases, frequent updates, strong contributor ecosystem.

## Strategic Focus
- **Rust migration**: LiteLLM is actively migrating core components to Rust for improved performance and reliability [litellm.ai](https://www.litellm.ai/).
- **Enterprise expansion**: Building out enterprise features (JWT auth, SSO, audit logs, custom SLAs) and dedicated support for large organizations [github.com](https://github.com/BerriAI/litellm).
- **Agent and MCP support**: Extending the gateway to support A2A agents and MCP servers, positioning LiteLLM as a hub for multi-model, multi-agent workflows.

## Why Work Here
- **Stage & impact**: Early-stage YC startup with high growth (10–14 people, rapidly expanding). Work directly with founders and shape the product from the ground up [ycombinator.com](https://www.ycombinator.com/companies/litellm/jobs).
- **Engineering culture**: Open-source first, with strong emphasis on code quality (Black, Ruff, MyPy, circular import checks). Contributors are active and valued [github.com](https://github.com/BerriAI/litellm).
- **Remote & hybrid options**: Some roles are San Francisco–based (office at 1730 Jones St), others offer remote within the US or Mexico. Equity packages range from 0.05% to 3% depending on role [ycombinator.com](https://www.ycombinator.com/companies/litellm/jobs).
- **Compensation**: Salary ranges vary by role (e.g., Founding Reliability Engineer: $200K–$270K + 0.25%–0.75% equity; Backend Engineer: $150K–$200K + 0.25%–0.75% equity; interns $80K–$100K) [ycombinator.com](https://www.ycombinator.com/companies/litellm/jobs).
- **Team composition**: 36% technical, 21% sales, 14% general management – small, cross-functional team with founders actively coding.

## Sources
1. [litellm.ai](https://www.litellm.ai/)
2. [linkedin.com](https://www.linkedin.com/company/litellm)
3. [ycombinator.com](https://www.ycombinator.com/companies/litellm/jobs)
4. [github.com](https://github.com/BerriAI/litellm)
5. [jobs.ashbyhq.com/litellm](https://jobs.ashbyhq.com/litellm) (careers page, but no additional detail beyond job listings)

## Other roles at LiteLLM

- [Business Manager](https://feeny.ai/job/business-manager-litellm-united-states-rggyetyqw7ka) — United States
- [Account Executive](https://feeny.ai/job/account-executive-litellm-san-francisco-mv5knsvgbbc5) — San Francisco, CA
- [Forward Deployed Engineer (New Grad)](https://feeny.ai/job/forward-deployed-engineer-new-grad-litellm-united-states-e0c74w49szkf) — United States
- [Technical Account Manager](https://feeny.ai/job/technical-account-manager-litellm-united-states-63aqttbmc0f9) — United States
- [Forward Deployed Engineer](https://feeny.ai/job/forward-deployed-engineer-litellm-united-states-dxh3t099p93f) — United States
- [Founding Developer Relations](https://feeny.ai/job/founding-developer-relations-litellm-san-francisco-fe4fwf2gpysb) — San Francisco, CA
- [AI Engineer](https://feeny.ai/job/ai-engineer-litellm-san-francisco-v2eczyy9ah6c) — San Francisco, CA
- [Senior Support Engineer - India](https://feeny.ai/job/senior-support-engineer-india-litellm-india-wjm52a9f4vmf) — India
- [Senior Support Engineer](https://feeny.ai/job/senior-support-engineer-litellm-san-francisco-z0gy525hv7zc) — San Francisco, CA
- [Rust Engineer](https://feeny.ai/job/rust-engineer-litellm-san-francisco-1x6vp82rtvz6) — San Francisco, CA