---
title: 'GPU Engineer at Kog'
canonical: 'https://feeny.ai/job/gpu-engineer-kog-paris-c71xtrnkrb6r'
type: 'job'
last_seen: '2026-09-15'
---

# GPU Engineer at Kog

- **Company:** Kog
- **Location:** Paris, France
- **Employment:** full-time
- **Work type:** hybrid
- **Posted:** 2026-06-09
- **Last confirmed live:** 2026-09-15
- **Apply:** https://jobs.ashbyhq.com/kog/e3950334-a2a6-43cc-a744-df6c38683166

## Job description

## About Kog

Kog builds the Kog Inference Engine, a real-time inference engine for AI agents running on standard datacenter GPUs. We co-design three layers: model architecture, inference engine, and low-level GPU kernels. We design the full inference stack around standard AMD and NVIDIA datacenter GPUs, from model architecture down to low-level kernels. Kog generates 3,500 tokens/s per request on 8 AMD MI300X GPUs and 2,100 tokens/s per request on 8 NVIDIA H200 GPUs, in FP16 at batch size 1, with quantization and speculative decoding disabled. Our hot path removes framework and host overhead. On NVIDIA, we write CUDA and PTX by hand. On AMD, we use HIP and CDNA ISA. The team has 11 people, including 10 engineers and researchers and 5 PhDs. Test it at [playground.kog.ai](http://playground.kog.ai). Read the technical details on the [Kog Labs blog](https://blog.kog.ai).

## What you will work on

The GPU Engineer role focuses on low-level execution and GPU performance. Memory behavior, synchronization, latency, and hardware constraints shape the work. You will work on:

- Our monokernel pipeline, where the decode loop runs as one persistent GPU program from the first token to the last while the GPU keeps control of the hot path.
- Low-level kernel optimization across AMD and NVIDIA, with both platforms treated as first-class targets.
- Memory-bound execution paths, including the batch-size-1 GEMV regime as our primary target.
- Profiling infrastructure that isolates bottlenecks inside a persistent GPU program and connects measurements to implementation decisions.
- Inter-GPU communication through KCCL, our latency-focused communication layer.
- Scaling the engine to third-party MoE models, with DeepSeek V4 as the current porting target.
- Internal agents for GPU engineering and kernel optimization, built on the expertise and execution infrastructure developed by the GPU team.

## What we look for

We look for engineers who have worked below the framework layer on problems where hardware behavior or performance was central. Relevant evidence includes:

- CUDA, HIP, PTX, CDNA ISA, SASS, shaders, drivers, or comparable low-level systems work.
- GPU kernels where you measured performance and can explain why a change moved the result.
- Latency-sensitive, memory-bound, or synchronization-sensitive execution paths.
- Profiling work that identified a real bottleneck and led to an implementation change.
- Upstream contributions to inference engines, compilers, drivers, graphics systems, or other performance-critical projects.
- Original technical work in graphics, game engines, video, Vulkan, drivers, HPC, or scientific computing at the hardware and performance layer.

PyTorch custom operations and Triton are relevant when the work shows hardware-level reasoning below the API layer. We review a technical artifact during the process. This can be public code, a merged upstream contribution, a thesis, or a detailed technical write-up based on work you can share.

## What we offer

You will join a small team at a stage where individual engineers can still shape the core technology, while the engine is advanced enough to start being tested against real customer workloads.

- High individual impact, with direct ownership over technical decisions and systems that sit on the critical path of inference performance.
- AMD and NVIDIA as first-class targets, giving you exposure to different GPU architectures, programming models, and hardware behaviors within the same inference engine.
- A broad technical surface, where you can follow a performance problem from profiling and kernel execution through synchronization and inter-GPU communication.
- A role at the foundation of our internal GPU engineering agents, where the expertise and systems built by the GPU team become the substrate for automated kernel optimization.
- A Paris-based team with support for candidates outside the region: at least one week per month in Paris, with travel and accommodation covered by Kog.

## About Kog

## Company Overview
- **One-liner**: Kog builds a realtime AI inference engine that achieves up to 3,000 tokens per second per request on standard datacenter GPUs through low‑level GPU engineering and model‑architecture co‑design.
- **Entity Type**: Private (seed‑stage startup)
- **Headquarters**: Paris, France
- **Founded**: 2023
- **Founders**: Gaël Delalleau

## Core Business
- **Primary industry**: AI infrastructure / GPU inference
- **Target customers**: B2B – GPU providers, AI labs, enterprises deploying AI agents and agentic workflows
- **Mission statement**: "Power every digital experience by providing a vertically integrated, realtime AI inference engine."

## Products & Services
- **Kog Inference Engine (KIE)**: A drop‑in replacement for vLLM and TensorRT‑LLM. Uses a monokernel runtime (single persistent GPU program), custom collective‑communication library (KCCL), and hand‑crafted assembly‑level GPU code (CUDA/PTX, HIP/CDNA) – no dependency on PyTorch, Triton, or vendor libraries on the hot path.
- **Kog LaneFormer**: A model architecture that implements Delayed Tensor Parallelism (DTP), overlapping cross‑device communication with computation to hide latency in multi‑GPU decoding.
- **Kog Communication Library (KCCL)**: A latency‑optimized (<3 μs) inter‑GPU communication layer tuned at the assembly level for each GPU architecture, replacing RCCL/NCCL.

## Market Standing
- **Valuation / Market Cap**: Not disclosed
- **Key metric**: Total funding – $5M raised (Seed round led by Varsity VC with BPI France Deep Tech Program)
- **Notable investors**: Varsity VC, BPI France; awarded the French Tech 2030 label (October 2025)
- **Growth signals**: Launched public tech preview on AMD MI300X in May 2026 reaching 3,000 tok/s per request; 2,100 tok/s on NVIDIA H200; up to 3.5× faster than vLLM on AMD Instinct; team of 14 with 10 engineers/researchers (5 PhDs); press coverage from French Tech Journal.

## Competitive Advantages
- **Hardware‑software co‑design**: Every layer is optimised for uninterrupted compute – monokernel eliminates kernel‑launch overhead and host‑side scheduling.
- **Custom low‑level stack**: Avoids general‑purpose frameworks (PyTorch, Triton, CUTLASS, etc.) on the critical path; uses hand‑tuned assembly for each GPU.
- **Delayed Tensor Parallelism (DTP)**: Novel architecture that hides all‑reduce communication behind computation, crucial for single‑request latency.
- **Speed niche**: Matches dedicated inference‑silicon speeds on standard datacenter GPUs, without lock‑in to proprietary hardware.

## Strategic Focus
- **Current priorities**: Optimising single‑request decode latency for AI agents; expanding support for larger models and batching; targeting sovereign‑AI buyers and enterprise GPU fleets.
- **Direction**: Prove that software optimisations can unlock the full speed of existing GPU hardware, reducing reliance on custom chips.

## Why Work Here
- **Culture**: High‑density team of PhDs and GPU research engineers who “treat hardware limits as a starting point.” Emphasis on deep technical challenges at the intersection of architecture, runtime, and kernel design.
- **Team size**: ~14 people (as of mid‑2026), offering outsized impact per person.
- **Locations**: Headquarters in Paris, with presence in Hungary and Portugal – likely hybrid/remote‑friendly.
- **Perks**: Work on cutting‑edge inference technology with top investors (Varsity VC) and government recognition (French Tech 2030). The company is small enough that engineers shape the entire stack.

## Sources
1. [Kog.ai](https://www.kog.ai/)
2. [LinkedIn](https://fr.linkedin.com/company/kogai)
3. [Kog Blog – Real‑time LLM Inference on Standard GPUs (3,000 tok/s)](https://blog.kog.ai/real-time-llm-inference-on-standard-gpus-3-000-tokens-s-per-request/)
4. [French Tech Journal – AI Industry Spent Billions Chasing Faster Chips](https://www.frenchtechjournal.com/the-ai-industry-spent-billions-chasing-faster-chips-inference-startup-kog-says-they-were-solving-the-wrong-problem/)
5. [GitHub](https://github.com/kog-ai)

## Other roles at Kog

- [Research Engineer](https://feeny.ai/job/research-engineer-kog-paris-5ffy8ryxsz1a) — Paris, France
- [GPU Engineer](https://feeny.ai/job/gpu-engineer-bot-auto-houston-56xpxvrhb6c9) — Houston, TX / San Francisco, CA