---
title: 'GPU Engineer at Kog'
canonical: 'https://feeny.ai/job/gpu-engineer-kog-paris-mmp993ct247g'
type: 'job'
last_seen: '2026-11-10'
---

# GPU Engineer at Kog

- **Company:** Kog
- **Location:** Paris, France
- **Employment:** full-time
- **Work type:** remote
- **Posted:** 2026-11-09
- **Last confirmed live:** 2026-11-10
- **Apply:** https://jobs.ashbyhq.com/kog/e02be7e1-eb8a-4d96-89bc-2d83841a6654

## Job description

## About Kog

At Kog, we're building real-time LLM inference for AI agents. Coding agents execute sequences of tasks where each step depends on the model's previous response. The faster the model generates tokens, the faster these agents can reason, code, and act. Our approach is to co-design model architecture, the inference engine, and low-level GPU kernels to minimize generation latency on standard NVIDIA and AMD datacenter GPUs. We've developed several technologies to make this possible:

- Monokernel: a persistent GPU program that keeps the entire token generation loop on the GPU, avoiding CPU handoffs.
- KCCL: our custom inter-GPU communication layer, designed to minimize latency.
- Delayed Tensor Parallelism (DTP): a model architecture that overlaps communication with computation and weight loading.
- Laneformer 2B: our code-generation model demonstrating this co-design approach.

Our engineers work directly with GPU hardware, using CUDA, PTX, HIP, and CDNA ISA to optimize execution. Implementing and optimizing a new AI model for a specific GPU architecture can take months of manual engineering. Meanwhile, new models are constantly being released. Our goal is to use AI agents to implement and optimize the models we choose on the GPUs we choose, in a fraction of the time required today. These agents will build on the GPU optimizations and hardware insights our team has developed. They'll explore optimization strategies, generate and modify kernels, verify correctness, benchmark performance on real hardware, and iterate based on the results. We want to make model porting and GPU optimization much faster and repeatable, allowing Kog to keep pace with the latest AI models rather than spending months adapting each one manually. This is a major engineering direction we're now developing, and you'll help shape it. Learn more at https://blog.kog.ai.

## What you'll work on

You'll join our GPU engineering team to tackle challenging performance problems and help build the internal agents that will accelerate this work.

- Develop and optimize GPU kernels using C++, CUDA, HIP, and low-level instructions.
- Investigate GPU architecture, memory access, synchronization, and execution bottlenecks.
- Profile and benchmark workloads to identify and validate performance improvements.
- Optimize inference operations and communication across multiple GPUs.
- Implement and optimize new AI models on NVIDIA and AMD GPUs.
- Help build AI agents for GPU engineering, enabling automated code optimization, correctness testing, benchmarking, and iterative improvements.
- Take ownership of technical decisions and their implementation.

## What we're looking for

We're looking for an experienced GPU Engineer who can contribute quickly and independently to complex performance problems.

- Strong hands-on experience developing and optimizing GPU kernels, ideally using CUDA C++ or HIP/ROCm.
- Deep understanding of GPU architecture, memory hierarchy, parallel execution, and performance bottlenecks.
- Strong C++ skills and experience profiling and benchmarking GPU workloads.
- Proven ownership of challenging technical projects, from implementation to measured results.
- Interest in developing AI agents and tools to accelerate GPU engineering.

Experience with PTX, CDNA ISA, multi-GPU systems, low-latency inference, or GPU performance portability is particularly valuable. A PhD in HPC, computer science, or a related field is a plus when backed by substantial engineering work. Previous experience building AI agents is not required.

Location and availability Candidates must already be based in Paris or elsewhere in Île-de-France. We are not considering relocation from other regions or countries for this position. We're looking for someone who can join the team quickly.

## About Kog

## Company Overview
- **One-liner**: Kog builds a realtime AI inference engine that achieves up to 3,000 tokens per second per request on standard datacenter GPUs through low‑level GPU engineering and model‑architecture co‑design.
- **Entity Type**: Private (seed‑stage startup)
- **Headquarters**: Paris, France
- **Founded**: 2023
- **Founders**: Gaël Delalleau

## Core Business
- **Primary industry**: AI infrastructure / GPU inference
- **Target customers**: B2B – GPU providers, AI labs, enterprises deploying AI agents and agentic workflows
- **Mission statement**: "Power every digital experience by providing a vertically integrated, realtime AI inference engine."

## Products & Services
- **Kog Inference Engine (KIE)**: A drop‑in replacement for vLLM and TensorRT‑LLM. Uses a monokernel runtime (single persistent GPU program), custom collective‑communication library (KCCL), and hand‑crafted assembly‑level GPU code (CUDA/PTX, HIP/CDNA) – no dependency on PyTorch, Triton, or vendor libraries on the hot path.
- **Kog LaneFormer**: A model architecture that implements Delayed Tensor Parallelism (DTP), overlapping cross‑device communication with computation to hide latency in multi‑GPU decoding.
- **Kog Communication Library (KCCL)**: A latency‑optimized (<3 μs) inter‑GPU communication layer tuned at the assembly level for each GPU architecture, replacing RCCL/NCCL.

## Market Standing
- **Valuation / Market Cap**: Not disclosed
- **Key metric**: Total funding – $5M raised (Seed round led by Varsity VC with BPI France Deep Tech Program)
- **Notable investors**: Varsity VC, BPI France; awarded the French Tech 2030 label (October 2025)
- **Growth signals**: Launched public tech preview on AMD MI300X in May 2026 reaching 3,000 tok/s per request; 2,100 tok/s on NVIDIA H200; up to 3.5× faster than vLLM on AMD Instinct; team of 14 with 10 engineers/researchers (5 PhDs); press coverage from French Tech Journal.

## Competitive Advantages
- **Hardware‑software co‑design**: Every layer is optimised for uninterrupted compute – monokernel eliminates kernel‑launch overhead and host‑side scheduling.
- **Custom low‑level stack**: Avoids general‑purpose frameworks (PyTorch, Triton, CUTLASS, etc.) on the critical path; uses hand‑tuned assembly for each GPU.
- **Delayed Tensor Parallelism (DTP)**: Novel architecture that hides all‑reduce communication behind computation, crucial for single‑request latency.
- **Speed niche**: Matches dedicated inference‑silicon speeds on standard datacenter GPUs, without lock‑in to proprietary hardware.

## Strategic Focus
- **Current priorities**: Optimising single‑request decode latency for AI agents; expanding support for larger models and batching; targeting sovereign‑AI buyers and enterprise GPU fleets.
- **Direction**: Prove that software optimisations can unlock the full speed of existing GPU hardware, reducing reliance on custom chips.

## Why Work Here
- **Culture**: High‑density team of PhDs and GPU research engineers who “treat hardware limits as a starting point.” Emphasis on deep technical challenges at the intersection of architecture, runtime, and kernel design.
- **Team size**: ~14 people (as of mid‑2026), offering outsized impact per person.
- **Locations**: Headquarters in Paris, with presence in Hungary and Portugal – likely hybrid/remote‑friendly.
- **Perks**: Work on cutting‑edge inference technology with top investors (Varsity VC) and government recognition (French Tech 2030). The company is small enough that engineers shape the entire stack.

## Sources
1. [Kog.ai](https://www.kog.ai/)
2. [LinkedIn](https://fr.linkedin.com/company/kogai)
3. [Kog Blog – Real‑time LLM Inference on Standard GPUs (3,000 tok/s)](https://blog.kog.ai/real-time-llm-inference-on-standard-gpus-3-000-tokens-s-per-request/)
4. [French Tech Journal – AI Industry Spent Billions Chasing Faster Chips](https://www.frenchtechjournal.com/the-ai-industry-spent-billions-chasing-faster-chips-inference-startup-kog-says-they-were-solving-the-wrong-problem/)
5. [GitHub](https://github.com/kog-ai)

## Other roles at Kog

- [GPU Engineer](https://feeny.ai/job/gpu-engineer-bot-auto-houston-56xpxvrhb6c9) — Houston, TX / San Francisco Bay Area