--- title: 'Forward Deployed Engineer - AI Inference at FriendliAI' canonical: 'https://feeny.ai/job/forward-deployed-engineer-ai-inference-friendliai-san-francisco-qynbjce9sx8b' type: 'job' last_seen: '2026-10-17' --- # Forward Deployed Engineer - AI Inference at FriendliAI - **Company:** FriendliAI - **Location:** San Francisco, CA - **Employment:** full-time - **Work type:** hybrid - **Posted:** 2026-10-07 - **Last confirmed live:** 2026-10-17 - **Apply:** https://jobs.ashbyhq.com/friendliai/cd82b036-83df-4f10-88b7-96d79c0fec6d ## Job description ## About the job FriendliAI is seeking a Forward Deployed Engineer to assist enterprises in deploying, scaling, and operating generative and agentic AI workloads on FriendliAI infrastructure. You will work directly with customers to solve and implement production-grade applications using our products, such as Serverless Endpoints, Dedicated Endpoints, or Container. Friendli Container is our service that allows customers to download our inference engine as Docker images and deploy it in their chosen environment, such as private clouds or on-premises. Our Friendli Container can be adopted directly to AWS EKS clusters using our EKS add-on product. You will work directly on our customers’ projects, collaborating with their engineering teams to solve AI inference challenges like scaling, orchestration, and monitoring. This is a hands-on, customer-embedded role. If you have worked in DevOps, platform engineering, or SRE for AI applications, this is your ideal position. ## Key Responsibilities - Design and implement large-scale deployment architectures for LLM and multimodal inference - Deploy and manage containerized workloads across Kubernetes clusters - Diagnose production issues, such as performance bottlenecks, and implement temporary fixes as needed - Collaborate with customers’ DevOps teams to integrate FriendliAI’s infrastructure into their CI/CD workflows - Develop scripts, Helm charts, and Terraform modules that simplify repeated deployments - Contribute field insights to shape our platform reliability, observability, and scaling strategies - Lead workshops, technical sessions, or webinars to help customers master infrastructure best practices. ## Qualifications - 3+ years of experience in cloud infrastructure, DevOps, or reliability engineering - Bachelor’s or Master's degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent - Proficiency with Kubernetes, Docker, Terraform, and Helm - Strong foundation in distributed systems, networking, and performance tuning - Experience with GPU-based computing and generative AI model serving workloads - Strong technical background in backend systems or AI tooling - Experience operating workloads on AWS, GCP, or OCI - Excellent problem-solving and debugging skills in real-world environments ## Preferred Experience - Experience deploying large models (LLMs, diffusion models) on GPUs or clusters - Familiarity with inference frameworks (Triton, vLLM, TensorRT, DeepSpeed-Inference) - Familiarity with observability stacks (Prometheus, Grafana, Loki, ELK, OTEL) - Understanding of networking security and compliance frameworks (e.g., SOC 2) - Experience supporting on-prem or hybrid-cloud deployments ## Benefits - A front-row seat to the generative AI infrastructure revolution - Competitive compensation and benefits package - Daily lunch and dinner provided; unlimited snacks and beverages - Health check-up and top-tier hardware support - Flexible working hours and a highly collaborative environment ## About us FriendliAI is the fastest inference cloud for agents, built to run frontier open-weight models in production at scale. It delivers up to 7x faster output token speed, up to 90% lower inference costs, and 99.99% uptime across the most demanding agent workloads — long-context inference, real-time streaming, and accurate tool calling. We are a small, fast-moving team doing work that matters at one of the most exciting moments in the history of technology. With our world-class inference stack, we are building the platform teams can actually rely on. ## About FriendliAI ## Company Overview - **One-liner**: FriendliAI is The Frontier AI Inference Cloud, providing a highly optimized platform for deploying, scaling, and monitoring large language and multimodal models with unmatched speed and cost efficiency. - **Entity Type**: Private (Seed stage; $26.7M total funding) - **Headquarters**: San Francisco, California, United States (with offices in Seoul, South Korea and Redwood City, CA) - **Founded**: 2021 - **Founders**: Byung-Gon Chun (CEO) and Gyeong-In Yu (CTO) – the researchers who invented the continuous batching technique now standard in AI inference. ## Core Business - **Primary industry**: AI Infrastructure / Cloud Inference Platform (Infrastructure as a Service) - **Target customers**: B2B – enterprises and AI teams deploying generative AI and agent workloads at production scale; focuses on open-weight and custom models. - **Mission**: Democratize access to production-grade AI so teams can focus on building great products. ## Products & Services - **[FriendliAI Inference Cloud](https://friendli.ai/)**: A fully managed platform that instantly deploys 580,000+ Hugging Face models (language, audio, vision) with one click. Supports bring-your-own fine-tuned or proprietary models. Key features: - Custom GPU kernels, smart caching, continuous batching, speculative decoding, and parallel inference. - 2×+ faster inference than standard solutions, up to 3× faster than vLLM. - 50% to 90% cost savings relative to closed model APIs. - 99.99% uptime SLAs, geo-distributed infrastructure, enterprise-grade fault tolerance. - SOC 2 Type II and HIPAA compliant (March 2026). ## Market Standing - **Valuation/Market Cap**: Not disclosed (private company) - **Key Metric**: Total funding $26.7M (two seed rounds: $6.7M in 2021 and $20M led by Capstone Partners in September 2025) - **Notable Investors/Partners**: Capstone Partners (lead investor in latest round), plus two other investors in the first seed round. - **Growth Signals**: - Headcount grew 90% YoY to ~45 employees (as of mid-2026). - Active job postings: 19 positions (up 1800% YoY) – strong hiring momentum. - Appointed Brian Yoo (former Moloco COO) as Chief Business Officer in April 2026 to drive hypergrowth. - Achieved SOC 2 Type II and HIPAA compliance in March 2026, opening regulated markets. - LinkedIn followers grew 257% yearly (8,221 followers). ## Competitive Advantages - **Inventor of continuous batching** – the technique is now industry standard, giving FriendliAI deep technical expertise. - **Purpose-built inference engine** that constantly evolves for state-of-the-art models, delivering 2–3× speed improvements over alternatives like vLLM. - **Massive model library** – 580,000+ Hugging Face models deployable instantly with zero manual optimization. - **Enterprise-grade reliability** – 99.99% SLA, SOC 2 Type II, HIPAA, multi-cloud scaling. - **Cost leadership** – 50–90% savings vs. closed model APIs, maximizing tokens per dollar. ## Strategic Focus - **Hypergrowth** – scaling sales, engineering, and solutions architecture teams globally (US and Korea). - **Compliance and enterprise readiness** – recent SOC 2/HIPAA certification to serve healthcare and regulated industries. - **Multi-cloud and geo-distribution** – expanding infrastructure footprint for global low-latency inference. - **Open-weight and custom model support** – enabling model ownership and flexibility for enterprises. ## Why Work Here - **Culture**: “Boldest innovations come from great teams” – passionate, humble, curious builders. Mission-driven to democratize AI. - **Work policy**: Hybrid (San Francisco and Seoul offices); some roles are in-office (Seoul) or hybrid (SF). Employees engage in a mix of remote and on-site work. - **Engineering focus**: Heavy emphasis on AI inference engine, GPU kernels, Python developer tools, AI agents, backend, and platform security. High technical bar. - **Perks**: Not explicitly listed, but fast-growing startup with significant impact in the AI infrastructure space; opportunity to work on cutting-edge inference optimization. - **Locations**: San Francisco (HQ), Redwood City, CA, and Gangnam-gu, Seoul – global team with cross-cultural collaboration. ## Sources 1. [friendli.ai](https://friendli.ai/) 2. [friendli.ai/careers](https://friendli.ai/careers) 3. [jobs.ashbyhq.com/friendliai](https://jobs.ashbyhq.com/friendliai) 4. [builtin.com/company/friendliai](https://builtin.com/company/friendliai) 5. [linkedin.com/company/friendliai](https://www.linkedin.com/company/friendliai) ## Other roles at FriendliAI - [Strategic Account Executive](https://feeny.ai/job/strategic-account-executive-friendliai-san-francisco-agpd8wfwpcdv) — San Francisco, CA - [Director of Product](https://feeny.ai/job/director-of-product-friendliai-san-francisco-vckbhh439mws) — San Francisco, CA - [Software Engineer - Full Stack](https://feeny.ai/job/software-engineer-full-stack-friendliai-seoul-2kvnfm2hejx8) — Seoul, South Korea - [Software Engineer – Cloud Infrastructure](https://feeny.ai/job/software-engineer-cloud-infrastructure-friendliai-san-francisco-z8fjtpn12kpn) — San Francisco, CA - [Software Engineer - Cloud Infrastructure](https://feeny.ai/job/software-engineer-cloud-infrastructure-friendliai-seoul-rf473fpa9850) — Seoul, South Korea - [Software Engineer – Python Developer Tools](https://feeny.ai/job/software-engineer-python-developer-tools-friendliai-seoul-z74wd9kp8fex) — Seoul, South Korea - [Software Engineer – GPU Kernel](https://feeny.ai/job/software-engineer-gpu-kernel-friendliai-seoul-yg1eq3ks7m5a) — Seoul, South Korea - [Software Engineer – AI Inference Engine](https://feeny.ai/job/software-engineer-ai-inference-engine-friendliai-seoul-8pdpe18z4ejk) — Seoul, South Korea - [Customer Success Engineer (contract based)](https://feeny.ai/job/customer-success-engineer-contract-based-friendliai-seoul-9ang4dybr9c9) — Seoul, South Korea - [Software Engineer - Senior Backend](https://feeny.ai/job/software-engineer-senior-backend-friendliai-san-francisco-h5x2ghcnfrg7) — San Francisco, CA