---
title: 'Senior Software Engineer, TorchPass Product Engineering at Clockwork.io'
canonical: 'https://feeny.ai/job/senior-software-engineer-torchpass-product-engineering-clockwork-io-palo-alto-8c6j4p73f2n5'
type: 'job'
last_seen: '2026-10-12'
---

# Senior Software Engineer, TorchPass Product Engineering at Clockwork.io

- **Company:** Clockwork.io
- **Location:** Palo Alto, CA
- **Work type:** onsite
- **Posted:** 2026-10-09
- **Last confirmed live:** 2026-10-12
- **Apply:** https://job-boards.greenhouse.io/clockworksystems/jobs/6221661004

## Job description

## About Clockwork Systems

Clockwork.io – Software Driven Fabrics to increase GPU cluster utilization

Clockwork Systems was founded by Stanford researchers and veteran systems engineers who share a vision for redefining the foundations of distributed computing. As AI workloads grow increasingly complex, traditional infrastructure struggles to meet the demands of performance, reliability, and precise coordination. Clockwork is pioneering a software-driven approach to AI fabrics by delivering cross-stack observability to catch and quickly resolve problems, workload fault tolerance to keep jobs running through failures, and performance acceleration that dynamically routes and paces traffic to avoid congestion.

To learn more, visit[www.clockwork.io](https://www.clockwork.io).

## About the Role

TorchPass keeps distributed PyTorch training running through hardware and network failures. The Product Engineering team makes it something customers can install, trust, and run on their own clusters.

We are looking for an experienced engineer who has turned working infrastructure software into a product that other people can deploy and operate. You will own the parts of TorchPass that customers touch, and you will be one of the engineers they hear from when something does not work.

This is a product and deployment role. It does not require GPU runtime, NCCL, or kernel experience.

## What You Will Do

- Build and own the customer-facing surfaces of TorchPass: installation, configuration, and the views that show customers what the product is doing
- Keep product documentation accurate with every release
- Work directly with customers and our solutions engineers during evaluations: read logs, separate product defects from environment issues, and say clearly what happened
- Turn what we learn in the field into fixes, tests, and documentation
- Define, together with the core engineering team, what TorchPass needs from a customer's cluster and what it guarantees in return
- Extend TorchPass to new schedulers and job launchers as customers need them

## What We're Looking For

- 4+ years building or operating infrastructure software in production
- You have taken a system that worked for its authors and made it installable and supportable by people who did not build it
- Strong debugging skills on production systems you did not write
- Deep experience with Kubernetes or Slurm
- Strong Python, and the ability to read and change Go
- You use AI coding tools well, and you understand and stand behind the code you ship
- You write documentation that people follow successfully
- You are comfortable working with customers on technical problems
- A degree in computer science, electrical engineering, or a related field, or equivalent experience

## Preferred

- Experience with GPU clusters or machine learning training infrastructure
- Experience with release engineering: build pipelines, packaging, artifact delivery
- Experience bringing up clusters across more than one cloud provider

Enjoy

- Challenging projects.
- A friendly and inclusive workplace culture.
- Competitive compensation.
- A great benefits package.
- Catered lunch.

Compensation for this position will vary based on the skills and experience you bring, as well as internal equity considerations. For candidates hired at the posted level, the expected base salary range is $150,000 - $230,000. In addition to cash compensation, this role is eligible to participate in the company’s equity program, which may include stock options granted in accordance with the company’s equity plan and subject to approval and applicable vesting schedules. Clockwork Systems is an equal opportunity employer. We are committed to building world-class teams by welcoming bright, passionate individuals from all backgrounds. All qualified applicants will receive consideration for employment without regard to race, color, ancestry, religion, age, sex, sexual orientation, gender identity or expression, national origin, disability, or protected veteran status. We believe diversity drives innovation, and we grow stronger together.

## About Clockwork.io

## Company Overview

- **One-liner**: Clockwork builds software-driven AI fabrics that optimize GPU clusters for fault tolerance, deterministic performance, and higher utilization, working across any network (InfiniBand, RoCE, Ethernet) and any GPU vendor.
- **Entity Type**: Private (Series B)
- **Headquarters**: Palo Alto, California, United States
- **Founded**: 2018 (originally founded as Tick Tock Networks, renamed Clockwork Systems in 2021)
- **Founders**: Balaji Prabhakar, Yilong Geng, Deepak Merugu, Suresh Vasudevan (CEO), Mendel Rosenblum (Chief Scientist)

## Core Business

- **Primary industry**: AI infrastructure / data center networking / GPU cluster optimization
- **Target customers**: B2B, targeting hyperscalers, cloud providers, enterprises, and AI labs running large-scale distributed training and inference workloads
- **Mission/purpose**: To eliminate communication bottlenecks in AI fabrics, making AI workloads resilient, performant, and efficient — all through software alone.

## Products & Services

- **Clockwork FleetIQ**: A software-driven AI fabric that improves GPU cluster utilization and job completion times by 1.1–1.5x, reduces disruptive failures by over 90%, and provides observability, fault tolerance, and dynamic traffic control. Works across any network and GPU vendor.
- **Clockwork TorchPass**: A fault-tolerance framework for AI training that, according to a SemiAnalysis report, is the only approach that does not degrade training performance, recovering millions in wasted compute annually.
- **Live GPU Migration**: Introduced in June 2026, enables AI workloads to continue running through failures by migrating GPU state live, minimizing downtime.
- **Global Clock Sync**: Software-based near-nanosecond clock synchronization across distributed components, enabling precise telemetry and one-way delay measurement.
- **Distributed State Transfer**: Captures and transfers distributed network/communication state (model parameters, NCCL communicators, etc.) to support path failover and live GPU migration.
- **Dynamic Traffic Control**: Programmable traffic steering and pacing at the packet level to relieve congestion and bound tail latencies.

## Market Standing

- **Total Funding**: $41.5M raised (Series B, with $20.5M raised approximately 10 months prior to June 2026)
- **Valuation**: Not publicly disclosed
- **Notable Investors**: Lead Series A investor New Enterprise Associates (NEA). Individual investors include Lip Bu Tan (CEO of Intel), Diane Greene (former CEO of VMware), John Chambers (former CEO/Executive Chairman of Cisco), John Hennessy (former President of Stanford, Chairman of Alphabet), Ram Shriram (Trustee of Stanford and Alphabet), Jerry Yang (AME Cloud Ventures, co-founder of Yahoo!).
- **Growth Signals**: Launched "You Only Compute Once" (YOCO) contractual commitment to end GPU waste; TorchPass validated by SemiAnalysis; released Live GPU Migration capability; active hiring with 9 open roles across engineering, sales, and solutions engineering; CEO interview in June 2026.

## Competitive Advantages

- **Vendor-neutral software-only approach**: Operates across any network fabric (InfiniBand, RoCE, Ethernet) and any GPU vendor — no proprietary hardware required.
- **Foundational research from Stanford**: The Huygens clock synchronization system, developed by co-founder Yilong Geng during his PhD, provides near-nanosecond precision that underpins the entire platform.
- **Unique fault-tolerance without performance loss**: TorchPass is claimed to be the only fault-tolerance framework that does not cost training performance, according to third-party analysis.
- **Live GPU migration**: Allows workloads to survive failures by moving GPU state in real time, a capability not commonly available in competing solutions.
- **Strong advisory network**: Backed by luminaries like Diane Greene, John Chambers, and John Hennessy, providing deep industry credibility.

## Strategic Focus

- **Scale product adoption**: Expanding from early hyperscaler and enterprise deployments to broader market, with a focus on large-scale AI clusters.
- **Deepen observability and automation**: Continuing to enhance the observe-decide-act control loop for autonomic collective communications.
- **Expand global sales and partnerships**: Hiring senior account executives in Europe and solutions engineers in New York, indicating geographic expansion.
- **Build finance and marketing infrastructure**: Recent hires include VP of Finance Alison Qu and VP of Growth Marketing Anita Pandey, signaling preparation for further growth.

## Why Work Here

- **High-impact problem**: Solving fundamental communication bottlenecks in AI infrastructure, directly affecting GPU utilization and job completion times for the largest AI workloads.
- **Strong leadership**: CEO Suresh Vasudevan previously led Nimble Storage from startup to IPO and acquisition by HPE, and was CEO of Sysdig. Co-founders include Stanford professors and VMware co-founder Mendel Rosenblum.
- **Culture**: Emphasizes that "Behind every packet, there’s a person." Perks include free lunch, free gym, TGIFs, and comprehensive benefits.
- **Work policy**: Mix of onsite (Palo Alto, San Francisco) and remote (e.g., New York) roles, with most engineering roles onsite.
- **Recent funding and growth**: Series B stage with $41.5M raised and active hiring indicates strong runway and momentum.

## Sources

1. [clockwork.io/about-us](https://clockwork.io/about-us/)
2. [clockwork.io](https://clockwork.io/)
3. [clockwork.io/careers](https://clockwork.io/careers/)
4. [greenhouse.io - job board](https://job-boards.greenhouse.io/clockworksystems/jobs/5870086004)
5. [cbinsights.com - Clockwork profile](https://www.cbinsights.com/company/clockwork-3)

## Other roles at Clockwork.io

- [Senior HPC Developer - RDMA Networking](https://feeny.ai/job/senior-hpc-developer-rdma-networking-clockwork-io-palo-alto-eqz2f7jqd9n0) — Palo Alto, CA
- [Senior Human Resources Manager](https://feeny.ai/job/senior-human-resources-manager-clockwork-io-palo-alto-r3tgcbpk71tc) — Palo Alto, CA
- [Software Engineer](https://feeny.ai/job/software-engineer-clockwork-io-palo-alto-4nv80gmv4jzd) — Palo Alto, CA
- [Software Engineer Intern](https://feeny.ai/job/software-engineer-intern-clockwork-io-palo-alto-ww7d3e4tvcba) — Palo Alto, CA
- [Software Development Engineer in Test](https://feeny.ai/job/software-development-engineer-in-test-clockwork-io-palo-alto-vh0c96bkfx1e) — Palo Alto, CA
- [Senior Software Engineer – Network Observability](https://feeny.ai/job/senior-software-engineer-network-observability-clockwork-io-palo-alto-r2gff7gbgpjx) — Palo Alto, CA
- [Senior Software Engineer – Time Synchronization](https://feeny.ai/job/senior-software-engineer-time-synchronization-clockwork-io-palo-alto-b3j0fz4jpytv) — Palo Alto, CA
- [Senior Systems Engineer - AI Infrastructure](https://feeny.ai/job/senior-systems-engineer-ai-infrastructure-clockwork-io-palo-alto-yk8myb4rsjpj) — Palo Alto, CA
- [Tech Lead - Full Stack](https://feeny.ai/job/tech-lead-full-stack-clockwork-io-palo-alto-qhytynb881v8) — Palo Alto, CA