---
title: 'Software Development Engineer in Test at Clockwork.io'
canonical: 'https://feeny.ai/job/software-development-engineer-in-test-clockwork-io-palo-alto-vh0c96bkfx1e'
type: 'job'
last_seen: '2026-09-07'
---

# Software Development Engineer in Test at Clockwork.io

- **Company:** Clockwork.io
- **Location:** Palo Alto, CA
- **Work type:** onsite
- **Posted:** 2026-07-31
- **Last confirmed live:** 2026-09-07
- **Apply:** https://job-boards.greenhouse.io/clockworksystems/jobs/6131381004

## Job description

## About Clockwork Systems

Clockwork.io – Software Driven Fabrics to increase GPU cluster utilization

Clockwork Systems was founded by Stanford researchers and veteran systems engineers who share a vision for redefining the foundations of distributed computing. As AI workloads grow increasingly complex, traditional infrastructure struggles to meet the demands of performance, reliability, and precise coordination. Clockwork is pioneering a software-driven approach to AI fabrics by delivering cross-stack observability to catch and quickly resolve problems, workload fault tolerance to keep jobs running through failures, and performance acceleration that dynamically routes and paces traffic to avoid congestion.

To learn more, visit[www.clockwork.io](https://www.clockwork.io).

Summary

We are looking for a Software Development Engineer in Test who thrives in fast-paced startup environments and is excited to be a co-owner and major contributor to our testing and CI/CD infrastructure. You will play a critical role in ensuring the quality, performance, and reliability of every release before it reaches our customers.

Our software runs on production GPU and CPU clusters across Ethernet, RDMA/RoCE, and InfiniBand networks. Testing here means standing up real multi-node clusters, running real workloads against them, and measuring what happens. This is a hands-on engineering role, not a manual QA position. You will be deeply embedded in our engineering team, responsible for building robust automation, maintaining pipelines, and contributing directly to the product during quieter times. A meaningful part of this role is DevOps work, not just test authoring.

## Responsibilities

- Design, build, and maintain integration, regression, and end-to-end tests for distributed systems running on Kubernetes, Slurm, and bare-metal GPU/CPU clusters
- Extend our in-house end-to-end test automation framework and share ownership of the CI/CD testing infrastructure and pipeline
- Build and run performance and scale benchmarks, and catch regressions before they ship
- Automate cluster lifecycle and environment management across cloud and bare-metal environments
- Ensure all code releases meet high quality standards before shipping to customers
- Collaborate closely with engineers to identify gaps in test coverage and build tools or frameworks to fill them
- Investigate and diagnose build failures, flaky tests, and regressions
- Contribute to engineering efforts beyond QA when appropriate (feature development, tooling, etc.)

## Qualifications

- 3+ years of experience in a Test Engineering, DevOps, or QA role with a strong technical background
- Strong Python; comfortable reading and debugging Go
- Demonstrated expertise in building automated test scripts and frameworks from scratch
- Proven experience with CI/CD tools such as GitHub Actions, Jenkins, or GitLab CI, and testing frameworks such as pytest and Playwright
- Experience with public cloud infrastructure (GCP, AWS, or Azure)
- Familiarity with release management and deployment processes, including versioning, release candidates, and rollout verification
- Working knowledge of Linux, containerization, and orchestration (Docker, Kubernetes, Helm)
- Strong debugging, problem-solving, and communication skills
- A startup mindset, self-motivated, adaptable, and eager to take ownership

## Nice to have

- Experience with AI/ML infrastructure: GPUs, RDMA/RoCE or InfiniBand, NCCL, DCGM, PyTorch training jobs
- Workload managers and schedulers (Slurm, Kubeflow/PyTorchJob)
- Build systems at scale (Bazel), infrastructure-as-code (Terraform)
- Chaos or fault-injection testing of distributed systems

## Why Join Us

- Be part of a foundational team shaping the future of infrastructure software
- Work onsite in a collaborative environment in Palo Alto
- Opportunity to grow into broader engineering roles

Enjoy

- Challenging projects.
- A friendly and inclusive workplace culture.
- Competitive compensation.
- A great benefits package.
- Catered lunch.

Compensation for this position will vary based on the skills and experience you bring, as well as internal equity considerations. For candidates hired at the posted level, the expected base salary range is $130,000 - $175,000. The offered compensation package may also include stock options or other equity awards, subject to Clockwork’s equity program and applicable approvals. In addition to cash compensation, this role is eligible to participate in the company’s equity program, which may include stock options granted in accordance with the company’s equity plan and subject to approval and applicable vesting schedules. Clockwork Systems is an equal opportunity employer. We are committed to building world-class teams by welcoming bright, passionate individuals from all backgrounds. All qualified applicants will receive consideration for employment without regard to race, color, ancestry, religion, age, sex, sexual orientation, gender identity or expression, national origin, disability, or protected veteran status. We believe diversity drives innovation, and we grow stronger together.

## About Clockwork.io

## Company Overview

- **One-liner**: Clockwork builds software-driven AI fabrics that optimize GPU clusters for fault tolerance, deterministic performance, and higher utilization, working across any network (InfiniBand, RoCE, Ethernet) and any GPU vendor.
- **Entity Type**: Private (Series B)
- **Headquarters**: Palo Alto, California, United States
- **Founded**: 2018 (originally founded as Tick Tock Networks, renamed Clockwork Systems in 2021)
- **Founders**: Balaji Prabhakar, Yilong Geng, Deepak Merugu, Suresh Vasudevan (CEO), Mendel Rosenblum (Chief Scientist)

## Core Business

- **Primary industry**: AI infrastructure / data center networking / GPU cluster optimization
- **Target customers**: B2B, targeting hyperscalers, cloud providers, enterprises, and AI labs running large-scale distributed training and inference workloads
- **Mission/purpose**: To eliminate communication bottlenecks in AI fabrics, making AI workloads resilient, performant, and efficient — all through software alone.

## Products & Services

- **Clockwork FleetIQ**: A software-driven AI fabric that improves GPU cluster utilization and job completion times by 1.1–1.5x, reduces disruptive failures by over 90%, and provides observability, fault tolerance, and dynamic traffic control. Works across any network and GPU vendor.
- **Clockwork TorchPass**: A fault-tolerance framework for AI training that, according to a SemiAnalysis report, is the only approach that does not degrade training performance, recovering millions in wasted compute annually.
- **Live GPU Migration**: Introduced in June 2026, enables AI workloads to continue running through failures by migrating GPU state live, minimizing downtime.
- **Global Clock Sync**: Software-based near-nanosecond clock synchronization across distributed components, enabling precise telemetry and one-way delay measurement.
- **Distributed State Transfer**: Captures and transfers distributed network/communication state (model parameters, NCCL communicators, etc.) to support path failover and live GPU migration.
- **Dynamic Traffic Control**: Programmable traffic steering and pacing at the packet level to relieve congestion and bound tail latencies.

## Market Standing

- **Total Funding**: $41.5M raised (Series B, with $20.5M raised approximately 10 months prior to June 2026)
- **Valuation**: Not publicly disclosed
- **Notable Investors**: Lead Series A investor New Enterprise Associates (NEA). Individual investors include Lip Bu Tan (CEO of Intel), Diane Greene (former CEO of VMware), John Chambers (former CEO/Executive Chairman of Cisco), John Hennessy (former President of Stanford, Chairman of Alphabet), Ram Shriram (Trustee of Stanford and Alphabet), Jerry Yang (AME Cloud Ventures, co-founder of Yahoo!).
- **Growth Signals**: Launched "You Only Compute Once" (YOCO) contractual commitment to end GPU waste; TorchPass validated by SemiAnalysis; released Live GPU Migration capability; active hiring with 9 open roles across engineering, sales, and solutions engineering; CEO interview in June 2026.

## Competitive Advantages

- **Vendor-neutral software-only approach**: Operates across any network fabric (InfiniBand, RoCE, Ethernet) and any GPU vendor — no proprietary hardware required.
- **Foundational research from Stanford**: The Huygens clock synchronization system, developed by co-founder Yilong Geng during his PhD, provides near-nanosecond precision that underpins the entire platform.
- **Unique fault-tolerance without performance loss**: TorchPass is claimed to be the only fault-tolerance framework that does not cost training performance, according to third-party analysis.
- **Live GPU migration**: Allows workloads to survive failures by moving GPU state in real time, a capability not commonly available in competing solutions.
- **Strong advisory network**: Backed by luminaries like Diane Greene, John Chambers, and John Hennessy, providing deep industry credibility.

## Strategic Focus

- **Scale product adoption**: Expanding from early hyperscaler and enterprise deployments to broader market, with a focus on large-scale AI clusters.
- **Deepen observability and automation**: Continuing to enhance the observe-decide-act control loop for autonomic collective communications.
- **Expand global sales and partnerships**: Hiring senior account executives in Europe and solutions engineers in New York, indicating geographic expansion.
- **Build finance and marketing infrastructure**: Recent hires include VP of Finance Alison Qu and VP of Growth Marketing Anita Pandey, signaling preparation for further growth.

## Why Work Here

- **High-impact problem**: Solving fundamental communication bottlenecks in AI infrastructure, directly affecting GPU utilization and job completion times for the largest AI workloads.
- **Strong leadership**: CEO Suresh Vasudevan previously led Nimble Storage from startup to IPO and acquisition by HPE, and was CEO of Sysdig. Co-founders include Stanford professors and VMware co-founder Mendel Rosenblum.
- **Culture**: Emphasizes that "Behind every packet, there’s a person." Perks include free lunch, free gym, TGIFs, and comprehensive benefits.
- **Work policy**: Mix of onsite (Palo Alto, San Francisco) and remote (e.g., New York) roles, with most engineering roles onsite.
- **Recent funding and growth**: Series B stage with $41.5M raised and active hiring indicates strong runway and momentum.

## Sources

1. [clockwork.io/about-us](https://clockwork.io/about-us/)
2. [clockwork.io](https://clockwork.io/)
3. [clockwork.io/careers](https://clockwork.io/careers/)
4. [greenhouse.io - job board](https://job-boards.greenhouse.io/clockworksystems/jobs/5870086004)
5. [cbinsights.com - Clockwork profile](https://www.cbinsights.com/company/clockwork-3)

## Other roles at Clockwork.io

- [Account Executive](https://feeny.ai/job/account-executive-clockwork-io-palo-alto-r2h4t102yfpq) — Palo Alto, CA
- [Senior Software Engineer](https://feeny.ai/job/senior-software-engineer-clockwork-io-palo-alto-4p3r9k73dzd5) — Palo Alto, CA
- [Senior Software Engineer – Network Observability](https://feeny.ai/job/senior-software-engineer-network-observability-clockwork-io-palo-alto-r2gff7gbgpjx) — Palo Alto, CA
- [Senior Software Engineer – Time Synchronization](https://feeny.ai/job/senior-software-engineer-time-synchronization-clockwork-io-palo-alto-b3j0fz4jpytv) — Palo Alto, CA
- [Senior Systems Engineer - AI Infrastructure](https://feeny.ai/job/senior-systems-engineer-ai-infrastructure-clockwork-io-palo-alto-yk8myb4rsjpj) — Palo Alto, CA
- [Senior Frontend Engineer](https://feeny.ai/job/senior-frontend-engineer-clockwork-io-palo-alto-hrcgyvhb9ac0) — Palo Alto, CA
- [Senior Account Executive - UK](https://feeny.ai/job/senior-account-executive-uk-clockwork-io-london-england-jf58eyb29gq7) — London England, United Kingdom
- [Tech Lead - Full Stack](https://feeny.ai/job/tech-lead-full-stack-clockwork-io-palo-alto-qhytynb881v8) — Palo Alto, CA
- [Software Development Engineer in Test](https://feeny.ai/job/software-development-engineer-in-test-archipelago-analytics-inc-india-mgr25dws7znb) — India
- [Software Development Engineer in Test](https://feeny.ai/job/software-development-engineer-in-test-stockx-bengaluru-z3ffmr9e8svf) — Bengaluru, India