---
title: 'Software Engineer - Web Crawling at Tavily'
canonical: 'https://feeny.ai/job/software-engineer-web-crawling-tavily-israel-p0kt8q3jaw03'
type: 'job'
last_seen: '2026-10-16'
---

# Software Engineer - Web Crawling at Tavily

- **Company:** Tavily
- **Location:** Israel
- **Employment:** full-time
- **Work type:** onsite
- **Posted:** 2026-03-06
- **Last confirmed live:** 2026-10-16
- **Apply:** https://jobs.ashbyhq.com/tavily/2bf1e995-6aa1-49a6-9929-e72f6d787274

## Job description

## About Tavily

Tavily is building the search engine for AI agents. We believe the future of work will be led by AI agents, and that requires restructuring how the web is accessed. Our Search API provides web access for AI agents, enabling real-time internet search optimized for LLMs and Retrieval-Augmented Generation (RAG). We are backed by leading investors and serve developers and enterprises worldwide. Our team is fast-moving and ambitious. We ship quickly, iterate constantly, and care deeply about impact.

## The Role

- You'll build the systems that power how AI interacts with the live web.
- You'll design and implement large-scale, intelligent web acquisition pipelines, making sense of an ever-changing online ecosystem and transforming it into structured, high-quality data.
- This role blends technical understanding of browsers, networks, and automation with hands-on engineering.
- You'll work on adaptive systems that navigate modern websites, understand evolving web interaction ,detection systems, and gather meaningful information safely, efficiently, and at scale.
- Your work will directly shape Tavily's ability to provide AI agents with accurate, real-time knowledge of the world , bridging the gap between the open web and intelligent reasoning.
- This position is based in Israel and requires existing legal authorization to work in Israel. Tavily is not able to provide visa sponsorship for this role.

## What You'll Do

- Build distributed data acquisition systems that capture and structure the live web
- Investigate and analyze browser internals, fingerprinting, and anti-automation systems to develop stealthy, adaptive orchestration layers
- Prototype and deploy intelligent automation frameworks using Playwright, Puppeteer, and low-level browser control protocols (CDP)
- Conduct hands-on research into network flows, JavaScript-based protections, and emerging web standards affecting automation
- Collaborate with AI and infrastructure teams to integrate real-time web data into retrieval pipelines and LLM-powered agents
- Translate deep technical insights into production-grade components, balancing research freedom with engineering rigor
- Continuously evolve Tavily's capabilities in resilience, speed, and authenticity of web interaction

## What We're Looking For

- 3–5 years of experience as a backend or systems engineer, ideally working with large-scale, distributed, or web-facing infrastructure
- Strong programming skills in Python or Node.js, with experience in Go or C++ considered a strong plus
- Proven experience building and maintaining browser automation systems (Playwright, Puppeteer, or CDP) in production environments
- Solid understanding of browser internals, network protocols, and web interaction mechanisms
- Experience designing high-performance, resilient systems that handle scale, concurrency, and complex orchestration
- Strong debugging and analytical skills - able to investigate edge cases, performance bottlenecks, and behavior under dynamic web conditions
- Comfortable working in a fast-moving environment, collaborating closely with product, AI, and infrastructure teams to ship reliable systems quickly
- Excellent documentation and communication skills to ensure smooth integration and operational visibility

## Nice to Have

- Familiarity with cloud infrastructure, containerization (Docker), Kubernetes, and CI/CD
- Experience integrating AI or retrieval systems into production pipelines

## About Tavily

## Company Overview
- **One-liner**: Tavily provides a purpose-built search API that delivers real-time, structured web context for AI agents, enabling large language models to stay grounded and factual.
- **Entity Type**: Private (Series A)
- **Headquarters**: New York, New York, United States
- **Founded**: 2024
- **Founders**: Not publicly available

## Core Business
- **Primary industry**: AI infrastructure / Agentic web search
- **Target customers**: B2B – developers, AI startups, and enterprises building agentic applications (e.g., enterprise data analysis, AI assistant development, academic research)
- **Mission**: “To help AI agents navigate the web quickly and efficiently by shaping the context they see” [tavily.com](https://www.tavily.com/about)

## Products & Services
- **Tavily Search API**: A production-grade retrieval stack that takes a natural language query, searches the live web, extracts relevant content, and returns it structured and chunked for LLMs. Features real-time search, intelligent caching, and indexing. Supports drop-in integration with OpenAI, Anthropic, Groq, and other LLM providers. [tavily.com](https://tavily.com/)
- **/research endpoint**: A state-of-the-art research endpoint that achieves high accuracy on factual benchmarks like SimpleQA, designed for deep, multi-document synthesis. [tavily.com](https://tavily.com/)

## Market Standing
- **Valuation/Market Cap**: Not disclosed
- **Key Metric**: Total funding raised – **$25M** (latest round: $20M Series A, raised ~1 year ago) [cbinsights.com](https://www.cbinsights.com/company/tavily)
- **Notable Investors/Partners**:
  - Investors: Nebius Group, Insight Partners, Alpha Wave Global, N-Up Program, a16z Bio Health, and others [cbinsights.com](https://www.cbinsights.com/company/tavily)
  - Partners: JetBrains, Databricks (MCP Marketplace), IBM (WatsonX platform) [tavily.com](https://tavily.com/)
- **Growth Signals**:
  - **2M+ developers** using Tavily
  - **300M+ monthly requests** handled
  - **99.99% uptime** SLA
  - **180 ms p50 latency** on /search – fastest on the market
  - Billions of pages crawled and extracted without downtime
  - Press mentions: “Tavily is Joining Nebius” (partnership/acquisition signal) [tavily.com](https://tavily.com/)

## Competitive Advantages
- **Real-time performance**: Sub-200ms p50 latency for search queries, with intelligent caching and indexing to keep latency predictable as traffic scales.
- **Built-in safety**: Security, privacy, and content validation layers that block PII leakage, prompt injection, and malicious sources.
- **Enterprise-grade reliability**: 99.99% uptime SLA and proven at scale (300M+ monthly requests).
- **Developer experience**: Drop-in integration with all major LLM providers; a single API for real-time web access.
- **Freshness & relevance**: Prioritizes recent, high-quality web context to reduce hallucination in agentic workflows.

## Strategic Focus
- Scale the agentic web infrastructure to onboard “the next billion AI agents” [tavily.com](https://www.tavily.com/about)
- Deepen enterprise partnerships (Databricks, IBM, JetBrains) and expand into new verticals
- Continue improving retrieval accuracy and efficiency (e.g., the /research endpoint achieving state-of-the-art on SimpleQA)
- Grow the in-person engineering team across offices in New York, Tel Aviv, and Abu Dhabi

## Why Work Here
- **Culture**: In-office collaboration is core – teams work together in New York City, Tel Aviv, or Abu Dhabi. The company believes spontaneous problem-solving and strong relationships happen best in person. [tavily.com/careers](https://www.tavily.com/careers)
- **Benefits**: Comprehensive medical, dental, and vision coverage; monthly meal allowance; competitive salary and meaningful equity; annual learning budget for conferences and workshops.
- **Work environment**: Daily team lunches and regular dinners; a focus on curiosity, collaboration, and solving hard problems. The interview process is transparent and typically takes 2–4 weeks.
- **Engineering culture**: Emphasis on shipping quickly while building for the long arc (“play the long game”, “stay a student”). The team values technical excellence and clear communication.
- **Growth opportunities**: Internships offered; roles span engineering, product, sales, marketing, and customer experience. Current open positions include Forward Deployed Engineering Team Lead, Senior Site Reliability Engineer, Technical Support Engineer, and Product Marketing Manager. [tavily.com/careers/jobs](https://www.tavily.com/careers/jobs)

## Sources
1. [tavily.com – Company overview and product](https://tavily.com/)
2. [tavily.com/about – Mission and team](https://www.tavily.com/about)
3. [tavily.com/careers – Culture and benefits](https://www.tavily.com/careers)
4. [tavily.com/careers/jobs – Open positions](https://www.tavily.com/careers/jobs)
5. [cbinsights.com – Funding, investors, competitors](https://www.cbinsights.com/company/tavily)

## Other roles at Tavily

- [Sr. Manager, GTM Enablement](https://feeny.ai/job/sr-manager-gtm-enablement-tavily-austin-wevfzqr96fd2) — Austin, TX
- [Senior Product Manager](https://feeny.ai/job/senior-product-manager-tavily-tel-aviv-ht0rqj3j1yza) — Tel Aviv
- [EMEA SDR](https://feeny.ai/job/emea-sdr-tavily-london-7j9fcvrn4hpm) — London, United Kingdom
- [Amer SDR](https://feeny.ai/job/amer-sdr-tavily-austin-ekhevn6c6khq) — Austin, TX
- [Data Analyst](https://feeny.ai/job/data-analyst-tavily-israel-xx8kpcw01m34) — Israel
- [Software Engineer](https://feeny.ai/job/software-engineer-tavily-israel-k1s9fvsyjtbz) — Israel
- [IT and Security Analyst](https://feeny.ai/job/it-and-security-analyst-tavily-tel-aviv-nzqhzaxdheta) — Tel Aviv
- [DevOps Engineer](https://feeny.ai/job/devops-engineer-tavily-tel-aviv-d7wcrmxfwcfp) — Tel Aviv
- [DevOps Engineer](https://feeny.ai/job/devops-engineer-tavily-new-york-241zkwwmhv91) — New York, NY
- [Data Scientist](https://feeny.ai/job/data-scientist-tavily-tel-aviv-0kyww4n65gc0) — Tel Aviv