---
title: 'Senior Software Engineer – Network Observability at Clockwork.io'
canonical: 'https://feeny.ai/job/senior-software-engineer-network-observability-clockwork-io-palo-alto-r2gff7gbgpjx'
type: 'job'
last_seen: '2026-09-07'
---

# Senior Software Engineer – Network Observability at Clockwork.io

- **Company:** Clockwork.io
- **Location:** Palo Alto, CA
- **Work type:** onsite
- **Posted:** 2026-05-02
- **Last confirmed live:** 2026-09-07
- **Apply:** https://job-boards.greenhouse.io/clockworksystems/jobs/5986912004

## Job description

## About Clockwork Systems

Clockwork.io – Software Driven Fabrics to increase GPU cluster utilization

Clockwork Systems was founded by Stanford researchers and veteran systems engineers who share a vision for redefining the foundations of distributed computing. As AI workloads grow increasingly complex, traditional infrastructure struggles to meet the demands of performance, reliability, and precise coordination. Clockwork is pioneering a software-driven approach to AI fabrics by delivering cross-stack observability to catch and quickly resolve problems, workload fault tolerance to keep jobs running through failures, and performance acceleration that dynamically routes and paces traffic to avoid congestion.

To learn more, visit[www.clockwork.io](https://www.clockwork.io).

## About the Role

We are seeking an experienced Senior Software Engineer to contribute to the architecture, development, and scaling of a high-performance network monitoring and observability platform. This role will focus on building systems that provide deep visibility into RDMA, RoCE, InfiniBand, and TCP/IP networks. The ideal candidate has strong experience in distributed systems, Linux networking, and modern observability stacks (e.g., Grafana/Prometheus).

## What You Will Do

- Design, develop, and scale high-performance network monitoring platforms for RDMA, RoCE, InfiniBand, and TCP/IP infrastructure.
- Build backend telemetry services, observability dashboards, alerts, diagnostics, anomaly detection, SLA monitoring, and traffic analysis workflows.
- Troubleshoot complex production issues across application, OS, server, RDMA, and network layers while optimizing low-latency collection, aggregation, and alerting.
- Collaborate with cross-functional teams to deliver scalable solutions, improve engineering practices, automate operational workflows, and contribute to the technical direction of the platform.

## What We’re Looking For

- Bachelor's or Master's degree in Computer Science, Computer Engineering, Electrical Engineering, or a related technical field.
- Strong hands-on programming experience in C++, Go, Python, Rust, or similar systems programming languages.
- Proven experience delivering complex infrastructure projects and contributing to the design and implementation of distributed systems.
- Experience building distributed systems, backend services, telemetry pipelines, or observability platforms.
- Hands-on experience with RDMA, RoCE, InfiniBand, or other high-performance network fabrics.
- Familiarity with libibverbs, RDMA verbs, RDMA CM, queue pairs, completion queues, memory registration, and related RDMA concepts.
- Strong knowledge of Linux networking, TCP/IP, DNS, HTTP, routing, MTU, congestion control, packet loss, latency, and performance tuning.
- Experience with traceroute-style diagnostics, path discovery, network reachability checks, synthetic probes, or active network measurements.
- Experience with monitoring and visualization platforms such as Prometheus, Grafana, Datadog, Splunk, OpenTelemetry, or similar tools.
- Strong debugging skills across software, operating system, server, and network layers.
- Experience operating production systems in Linux-based environments.
- Strong technical judgment and ability to build reliable, scalable, and maintainable systems.

## Nice to Have

- Experience supporting AI/ML, HPC, storage, or GPU cluster infrastructure workloads.
- Experience with large-scale RoCE or InfiniBand deployments.
- Experience with NCCL, distributed training infrastructure, or AI cluster diagnostics.
- Experience with eBPF, XDP, DPDK, perf, tcpdump, Wireshark, ethtool, iproute2, rdma-core, or Linux kernel networking tools.
- Experience with cloud infrastructure on AWS, GCP, or Azure.
- Experience with Kubernetes, service discovery, configuration management, and infrastructure automation.
- Knowledge of security, compliance, and infrastructure best practices.
- Experience designing time-series data systems, alerting pipelines, or high-cardinality telemetry platforms.

Enjoy

- Challenging projects.
- A friendly and inclusive workplace culture.
- Competitive compensation.
- A great benefits package.
- Catered lunch.

Compensation for this position will vary based on the skills and experience you bring, as well as internal equity considerations. For candidates hired at the posted level, the expected base salary range is $140,000 - $210,000. The offered compensation package may also include stock options or other equity awards, subject to Clockwork’s equity program and applicable approvals. In addition to cash compensation, this role is eligible to participate in the company’s equity program, which may include stock options granted in accordance with the company’s equity plan and subject to approval and applicable vesting schedules. Clockwork Systems is an equal opportunity employer. We are committed to building world-class teams by welcoming bright, passionate individuals from all backgrounds. All qualified applicants will receive consideration for employment without regard to race, color, ancestry, religion, age, sex, sexual orientation, gender identity or expression, national origin, disability, or protected veteran status. We believe diversity drives innovation, and we grow stronger together.

## About Clockwork.io

## Company Overview

- **One-liner**: Clockwork builds software-driven AI fabrics that optimize GPU clusters for fault tolerance, deterministic performance, and higher utilization, working across any network (InfiniBand, RoCE, Ethernet) and any GPU vendor.
- **Entity Type**: Private (Series B)
- **Headquarters**: Palo Alto, California, United States
- **Founded**: 2018 (originally founded as Tick Tock Networks, renamed Clockwork Systems in 2021)
- **Founders**: Balaji Prabhakar, Yilong Geng, Deepak Merugu, Suresh Vasudevan (CEO), Mendel Rosenblum (Chief Scientist)

## Core Business

- **Primary industry**: AI infrastructure / data center networking / GPU cluster optimization
- **Target customers**: B2B, targeting hyperscalers, cloud providers, enterprises, and AI labs running large-scale distributed training and inference workloads
- **Mission/purpose**: To eliminate communication bottlenecks in AI fabrics, making AI workloads resilient, performant, and efficient — all through software alone.

## Products & Services

- **Clockwork FleetIQ**: A software-driven AI fabric that improves GPU cluster utilization and job completion times by 1.1–1.5x, reduces disruptive failures by over 90%, and provides observability, fault tolerance, and dynamic traffic control. Works across any network and GPU vendor.
- **Clockwork TorchPass**: A fault-tolerance framework for AI training that, according to a SemiAnalysis report, is the only approach that does not degrade training performance, recovering millions in wasted compute annually.
- **Live GPU Migration**: Introduced in June 2026, enables AI workloads to continue running through failures by migrating GPU state live, minimizing downtime.
- **Global Clock Sync**: Software-based near-nanosecond clock synchronization across distributed components, enabling precise telemetry and one-way delay measurement.
- **Distributed State Transfer**: Captures and transfers distributed network/communication state (model parameters, NCCL communicators, etc.) to support path failover and live GPU migration.
- **Dynamic Traffic Control**: Programmable traffic steering and pacing at the packet level to relieve congestion and bound tail latencies.

## Market Standing

- **Total Funding**: $41.5M raised (Series B, with $20.5M raised approximately 10 months prior to June 2026)
- **Valuation**: Not publicly disclosed
- **Notable Investors**: Lead Series A investor New Enterprise Associates (NEA). Individual investors include Lip Bu Tan (CEO of Intel), Diane Greene (former CEO of VMware), John Chambers (former CEO/Executive Chairman of Cisco), John Hennessy (former President of Stanford, Chairman of Alphabet), Ram Shriram (Trustee of Stanford and Alphabet), Jerry Yang (AME Cloud Ventures, co-founder of Yahoo!).
- **Growth Signals**: Launched "You Only Compute Once" (YOCO) contractual commitment to end GPU waste; TorchPass validated by SemiAnalysis; released Live GPU Migration capability; active hiring with 9 open roles across engineering, sales, and solutions engineering; CEO interview in June 2026.

## Competitive Advantages

- **Vendor-neutral software-only approach**: Operates across any network fabric (InfiniBand, RoCE, Ethernet) and any GPU vendor — no proprietary hardware required.
- **Foundational research from Stanford**: The Huygens clock synchronization system, developed by co-founder Yilong Geng during his PhD, provides near-nanosecond precision that underpins the entire platform.
- **Unique fault-tolerance without performance loss**: TorchPass is claimed to be the only fault-tolerance framework that does not cost training performance, according to third-party analysis.
- **Live GPU migration**: Allows workloads to survive failures by moving GPU state in real time, a capability not commonly available in competing solutions.
- **Strong advisory network**: Backed by luminaries like Diane Greene, John Chambers, and John Hennessy, providing deep industry credibility.

## Strategic Focus

- **Scale product adoption**: Expanding from early hyperscaler and enterprise deployments to broader market, with a focus on large-scale AI clusters.
- **Deepen observability and automation**: Continuing to enhance the observe-decide-act control loop for autonomic collective communications.
- **Expand global sales and partnerships**: Hiring senior account executives in Europe and solutions engineers in New York, indicating geographic expansion.
- **Build finance and marketing infrastructure**: Recent hires include VP of Finance Alison Qu and VP of Growth Marketing Anita Pandey, signaling preparation for further growth.

## Why Work Here

- **High-impact problem**: Solving fundamental communication bottlenecks in AI infrastructure, directly affecting GPU utilization and job completion times for the largest AI workloads.
- **Strong leadership**: CEO Suresh Vasudevan previously led Nimble Storage from startup to IPO and acquisition by HPE, and was CEO of Sysdig. Co-founders include Stanford professors and VMware co-founder Mendel Rosenblum.
- **Culture**: Emphasizes that "Behind every packet, there’s a person." Perks include free lunch, free gym, TGIFs, and comprehensive benefits.
- **Work policy**: Mix of onsite (Palo Alto, San Francisco) and remote (e.g., New York) roles, with most engineering roles onsite.
- **Recent funding and growth**: Series B stage with $41.5M raised and active hiring indicates strong runway and momentum.

## Sources

1. [clockwork.io/about-us](https://clockwork.io/about-us/)
2. [clockwork.io](https://clockwork.io/)
3. [clockwork.io/careers](https://clockwork.io/careers/)
4. [greenhouse.io - job board](https://job-boards.greenhouse.io/clockworksystems/jobs/5870086004)
5. [cbinsights.com - Clockwork profile](https://www.cbinsights.com/company/clockwork-3)

## Other roles at Clockwork.io

- [Account Executive](https://feeny.ai/job/account-executive-clockwork-io-palo-alto-r2h4t102yfpq) — Palo Alto, CA
- [Software Development Engineer in Test](https://feeny.ai/job/software-development-engineer-in-test-clockwork-io-palo-alto-vh0c96bkfx1e) — Palo Alto, CA
- [Senior Software Engineer](https://feeny.ai/job/senior-software-engineer-clockwork-io-palo-alto-4p3r9k73dzd5) — Palo Alto, CA
- [Senior Software Engineer – Time Synchronization](https://feeny.ai/job/senior-software-engineer-time-synchronization-clockwork-io-palo-alto-b3j0fz4jpytv) — Palo Alto, CA
- [Senior Systems Engineer - AI Infrastructure](https://feeny.ai/job/senior-systems-engineer-ai-infrastructure-clockwork-io-palo-alto-yk8myb4rsjpj) — Palo Alto, CA
- [Senior Frontend Engineer](https://feeny.ai/job/senior-frontend-engineer-clockwork-io-palo-alto-hrcgyvhb9ac0) — Palo Alto, CA
- [Senior Account Executive - UK](https://feeny.ai/job/senior-account-executive-uk-clockwork-io-london-england-jf58eyb29gq7) — London England, United Kingdom
- [Tech Lead - Full Stack](https://feeny.ai/job/tech-lead-full-stack-clockwork-io-palo-alto-qhytynb881v8) — Palo Alto, CA