---
title: 'Staff Software Engineer, Inference API at Cerebras Systems'
canonical: 'https://feeny.ai/job/staff-software-engineer-inference-api-cerebras-systems-sunnyvale-77y4130jrpyh'
type: 'job'
last_seen: '2026-10-13'
---

# Staff Software Engineer, Inference API at Cerebras Systems

- **Company:** [Cerebras Systems](https://feeny.ai/companies/cerebras-systems)
- **Location:** Sunnyvale, CA
- **Employment:** full-time
- **Work type:** hybrid
- **Posted:** 2026-09-23
- **Last confirmed live:** 2026-10-13
- **Apply:** https://jobs.ashbyhq.com/cerebras/78f1098a-6022-40ba-b7a0-948711921de2

## Job description

Cerebras Systems builds the world's largest AI chip, 56 times larger than GPUs. This architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud inference services.

This order of magnitude increase in speed is transforming the user experience of AI applications, unlocking real-time iteration and increasing intelligence via additional agentic computation.

Cerebras works with the leading model labs, global enterprises, and cutting-edge AI-native startups. [OpenAI recently announced a multi-year partnership](https://openai.com/index/cerebras-partnership/) with Cerebras, to deploy 750 megawatts of scale, transforming key workloads with ultra high-speed inference.

## About the Role

Cerebras is building a new generation of disaggregated AI inference systems that combine GPU-accelerated prefill with ultra-fast decode on the Cerebras Wafer-Scale Engine. We are hiring a Software Engineer to build and evolve the ML API layer that makes this heterogeneous serving system accessible, reliable, and easy to use. You will work across our inference APIs, model integration layer, request-routing services and Cerebras inference platform to deliver a consistent experience across models and accelerator backends. This role sits at the intersection of machine learning systems, API design, model serving, and distributed systems. You will enable new model architectures and inference capabilities, define stable user-facing behavior, and ensure that features such as streaming, sampling, tool use, structured outputs, multimodal inputs, and model configuration behave correctly and consistently in production. You will work closely with model enablement, compiler, runtime, cloud infrastructure, product, customer-facing teams, and customers directly. This is a hands-on software engineering role for someone who enjoys turning rapidly evolving ML capabilities into durable, production-quality APIs.

## Responsibilities

- Build production ML inference APIs. Design, implement, and maintain APIs for chat completions, text generation, streaming, model configuration, tool calling, structured outputs, multimodal inputs, and other emerging inference capabilities.
- Deliver a unified serving experience. Create consistent request and response semantics across GPU prefill, Cerebras decode, and other heterogeneous inference backends.
- Enable new models and capabilities. Integrate emerging foundation models, tokenizers, prompt formats, sampling methods, attention variants, multimodal inputs, and model-specific features into the serving platform.
- Own API compatibility and evolution. Maintain compatibility with widely adopted inference interfaces while designing Cerebras-specific extensions. Establish clear versioning, deprecation, validation, and backward compatibility practices.
- Integrate with model-serving runtimes. Extend and integrate custom inference services with vLLM, PyTorch, Hugging Face libraries, the AMD ROCm stack, and Cerebras runtime components.
- Support disaggregated inference. Build the control and data paths required to coordinate GPU prefill with Cerebras decode, including request routing, state transfer, error handling, retries, and lifecycle management.
- Improve serving performance. Optimize streaming behavior, time to first token, request latency, throughput, batching, serialization, tokenization, scheduling, and communication between serving components.
- Ensure functional and numerical correctness. Build validation systems for tokenization, sampling, logits, generated outputs, precision changes, model upgrades, determinism, and compatibility across serving backends.
- Strengthen reliability and observability. Define end-to-end service indicators and build structured logging, tracing, metrics, dashboards, health checks, and diagnostic tooling for production inference traffic.
- Develop testing and qualification infrastructure. Create conformance tests, workload-replay tools, model-validation suites, performance benchmarks, integration tests, and release gates.
- Improve developer experience. Build intuitive configuration, SDKs, documentation, examples, debugging tools, and self-service workflows for internal developers, customers, and partners.
- Collaborate across the stack. Partner with compiler, runtime, kernel, cloud, product, and solutions teams to translate model and customer requirements into scalable serving capabilities.

## Minimum Qualifications

- 5+ years of software engineering experience, including substantial individual-contributor ownership of production software or distributed systems.
- Strong programming ability in Python and Go plus experience developing performance-sensitive or highly concurrent services in C++, Rust, or a similar systems language.
- Experience building stable APIs with clear validation, error handling, observability, compatibility, and versioning practices.
- Experience integrating software across service, framework, runtime, and infrastructure boundaries.
- Experience designing or maintaining OpenAI-compatible, gRPC, REST, or streaming inference APIs.
- Experience with Linux, containers, Kubernetes or comparable orchestration systems, CI/CD, and operating latency-sensitive services in production.
- Ability to diagnose correctness, reliability, and performance issues across multiple components of a distributed serving system.
- Strong communication and cross-functional execution skills, with the ability to turn ambiguous model or product requirements into production-quality software.
- Bachelor's degree in computer science, Computer Engineering, Electrical Engineering, or a related discipline, or equivalent practical experience.

## Preferred Qualifications

- Experience modifying or contributing to vLLM, SGLang, PyTorch, Hugging Face Transformers, Triton, TensorRT-LLM, or another open-source ML systems project.
- Experience creating API conformance, model-quality, numerical-comparison, determinism, or performance-regression test systems
- Experience building SDKs, developer tools, model registries, configuration systems, or self-service ML platforms.
- Experience with multi-model or multi-tenant inference platforms, including routing, admission control, fairness, quotas, rate limiting, and capacity-aware scheduling
- Understanding of model-specific tokenization, chat templates, generation configuration, logits processing, stopping criteria, tool calling, structured generation, and constrained decoding.
- Experience with disaggregated prefill/decode architectures, KV-cache transfer, prefix caching, chunked prefill, memory-aware admission control, or request scheduling.
- Experience designing, building, or operating production APIs and services for machine learning, large language models, or other data-intensive applications.
- Familiarity with reduced-precision inference and quantization formats such as BF16, FP8, FP4, INT8, or INT4.

## Why Join Cerebras

People who are serious about software make their own hardware. At Cerebras, we have built a breakthrough architecture that is unlocking new opportunities for the AI industry. With dozens of model releases and rapid growth, we’ve reached an inflection point in our business. Members of our team tell us there are five main reasons they joined Cerebras:

- Build a breakthrough AI platform beyond the constraints of the GPU.
- Publish and open source their cutting-edge AI research.
- Work on one of the fastest AI supercomputers in the world.
- Enjoy job stability with startup vitality.
- Our simple, non-corporate work culture that respects individual beliefs.

Find out more about what it's like to work at Cerebras [here](https://www.cerebras.ai/join-us)! Apply today and become part of the forefront of groundbreaking advancements in AI! Cerebras Systems is committed to creating an equal and diverse environment and is proud to be an equal opportunity employer. We celebrate different backgrounds, perspectives, and skills. We believe inclusive teams build better products and companies. We try every day to build a work environment that empowers people to do their best work through continuous learning, growth and support of those around them. This website or its third-party tools process personal data. For more details, click [here](https://www.cerebras.net/privacy/) to review our CCPA disclosure notice.

## About Cerebras Systems

## Company Overview
- **One-liner**: Cerebras builds wafer-scale AI chips (the largest semiconductors ever made) and supercomputers that deliver up to 15x faster inference than GPUs, offered both on-premises and via cloud APIs.
- **Entity Type**: Public (NASDAQ: CBRS) — filed for IPO and now listed on the Nasdaq.
- **Headquarters**: Sunnyvale, California, United States
- **Founded**: 2015
- **Founders**: Andrew Feldman, Gary Lauterbach, Michael James, Sean Lie, Jean-Philippe Fricker

## Core Business
- **Primary industries**: Semiconductor manufacturing, AI computing hardware and software, cloud AI services.
- **Target customers**: B2B – enterprises, government research labs, cloud service providers, and AI developers.
- **Mission**: “Building impossible technologies so you can too” – accelerating AI through wafer-scale computing.

## Products & Services
- **Wafer-Scale Engine (WSE-3)**: The world’s largest AI chip (215 mm²), 58x larger than a typical GPU, built for ultra‑fast AI training and inference. Manufactured by TSMC.
- **CS-3 Supercomputer**: A single system powered by the WSE-3, delivering record-breaking AI performance for on-premises deployment.
- **Cerebras AI Inference Cloud & Training Cloud**: Pay-as-you‑go APIs that provide drop‑in OpenAI‑compatible access to Cerebras’ hardware for serving, fine‑tuning, and pre‑training models.
- **Condor Galaxy Network**: A series of supercomputers (e.g., CG‑1 with 4 exaFLOPs) built in partnership with G42 for high‑performance AI.

## Market Standing
- **Valuation/Market Cap**: Not disclosed (recently public; market cap data not available in provided sources).
- **Key Metric**: Annual revenue of $78.7 M (latest); total funding of $2.7 B across 15 rounds.
- **Notable Investors/Partners**: G42, OpenAI (signed in 2026), Amazon Web Services (signed in 2026), Meta (Llama API), Perplexity, Mistral, Hugging Face, OpenRouter.
- **Growth Signals**: 
  - 34.3% YoY headcount growth (701 employees).
  - Rapid data center expansion across North America and Europe.
  - Won the HPCwire Readers’ & Editors’ Choice 2025 for Best AI Product/Technology.
  - Named to Forbes America’s Best Startup Employers 2026 and Fast Company Most Innovative Companies 2026 (AI).

## Competitive Advantages
- **Wafer‑scale architecture**: Eliminates interconnect bottlenecks, reducing latency and delivering up to 15x faster inference than GPU clusters.
- **Largest AI chip ever built**: 58x larger than any GPU, enabling massive on‑chip memory (SRAM) and compute density.
- **Full‑stack offering**: Hardware + cloud APIs + on‑prem systems, giving customers deployment flexibility.
- **Strong customer relationships**: Key contracts with OpenAI, AWS, G42, and Mayo Clinic (2024 Gordon Bell Prize work).

## Strategic Focus
- **Scale inference capacity**: Building out data centers to become the world’s #1 provider of high‑speed AI inference.
- **Deepen cloud partnerships**: Expanding pay‑as‑you‑go cloud access and enterprise deployments.
- **Advance wafer‑scale technology**: Continuous R&D on WSE‑3 successors and software stack (compiler, ML workflows).
- **Broaden model support**: Already serving Llama, Gemma, Qwen, Mistral, and more; integrating with Hugging Face and OpenRouter.

## Why Work Here
- **Culture**: “Extraordinary people, breakthrough innovation, global impact” – the company highlights collaborative, low‑overhead teams with little bureaucracy.
- **Work environment**: Hybrid/office with locations in Sunnyvale, San Diego, Toronto, and Bangalore. Inclusive and flexible policy.
- **Perks**: Premium medical/dental/vision, life insurance, generous vacation, 401(k) and Group RRSP retirement plans, daily catered meals, healthy snacks, family‑friendly events (including CEO’s famous BBQ).
- **Engineering focus**: Tackling fundamental challenges in chip design, system software, compiler technology, and ML workflows. Positions range from hardware (Design Verification, Manufacturing) to software (SRE, Compiler, Cloud).
- **Philanthrophy**: Supports local communities and hosts students from around the world for Q&A.

## Sources
1. [cerebras.ai/company](https://www.cerebras.ai/company)
2. [cerebras.ai](https://www.cerebras.ai/)
3. [cerebras.ai/join-us](https://www.cerebras.ai/join-us)
4. [linkedin.com/company/cerebras-systems](https://www.linkedin.com/company/cerebras-systems)

## Other roles at Cerebras Systems

- [SEC Reporting & Technical Accounting Manager](https://feeny.ai/job/sec-reporting-technical-accounting-manager-cerebras-systems-sunnyvale-g6g9t52gcjn6) — Sunnyvale, CA
- [Senior Manager, Production & Fleet Operations](https://feeny.ai/job/senior-manager-production-fleet-operations-cerebras-systems-sunnyvale-rftjnccf36my) — Sunnyvale, CA
- [Senior Manager, Production & Fleet Operations](https://feeny.ai/job/senior-manager-production-fleet-operations-cerebras-systems-sunnyvale-tnzs6bz9xfma) — Sunnyvale, CA
- [Full Stack Machine Learning Engineer](https://feeny.ai/job/full-stack-machine-learning-engineer-cerebras-systems-bengaluru-fm2j4tvccywm) — Bengaluru, India
- [DevX Technical Writer](https://feeny.ai/job/devx-technical-writer-cerebras-systems-sunnyvale-rvrgasgk4wnx) — Sunnyvale, CA
- [Hardware Technical Analyst](https://feeny.ai/job/hardware-technical-analyst-cerebras-systems-sunnyvale-0cd9ayk6xwx5) — Sunnyvale, CA
- [Head of DevRel](https://feeny.ai/job/head-of-devrel-cerebras-systems-sunnyvale-2ahvwfsv5ame) — Sunnyvale, CA
- [Cluster Datacenter Manager](https://feeny.ai/job/cluster-datacenter-manager-cerebras-systems-alabama-xx25gw65zsh2) — Alabama, United States
- [Senior ASIC RTL Design Engineer, AI Hardware](https://feeny.ai/job/senior-asic-rtl-design-engineer-ai-hardware-cerebras-systems-sunnyvale-yajfd7m9cbmv) — Sunnyvale, CA
- [ML Research Scientist - Inference ML](https://feeny.ai/job/ml-research-scientist-inference-ml-cerebras-systems-united-states-and-c4r79zpxmzkj) — United States and, Canada