---
title: 'Data Scientist - Evaluations, Chanakya at Sarvam'
canonical: 'https://feeny.ai/job/data-scientist-evaluations-chanakya-sarvam-bengaluru-yavhq6hq5jq7'
type: 'job'
last_seen: '2026-10-17'
---

# Data Scientist - Evaluations, Chanakya at Sarvam

- **Company:** [Sarvam](https://feeny.ai/companies/sarvam)
- **Location:** Bengaluru, India
- **Employment:** full-time
- **Work type:** onsite
- **Posted:** 2026-04-17
- **Last confirmed live:** 2026-10-17
- **Apply:** https://jobs.ashbyhq.com/sarvam/7c42207a-bb56-413c-839f-9ad692df840e/application

**Skills:** Python, pandas, NumPy, HuggingFace datasets, RAGAS, EleutherAI Eval Harness, LangSmith, Prompt engineering, Model fine-tuning, RLHF, Statistics, Probability, LLMs, Data analysis, Healthcare domain knowledge, Legal domain knowledge, Defense domain knowledge, Finance domain knowledge, Red-teaming, Adversarial evaluation

> Design and build domain-specific evaluation frameworks and quality metrics for Sarvam's AI outputs. The role involves running structured evaluation cycles, defining quality signals with domain experts, and collaborating with MLOps and deployment teams to ensure model reliability and performance in production.

## Job description

## About Sarvam

Sarvam is building the bedrock of Sovereign AI for India. The company is developing India's full-stack sovereign AI platform, building across research, models, infrastructure and applications with a singular focus on making AI genuinely work for India. Sarvam works with leading enterprises and public institutions and is backed by Lightspeed, Peak XV, and Khosla Ventures. Sarvam partners with India's leading brands, including Tata Capital, SBI Life, CRED, IDFC, and LIC.

## About the Role

This is an intellectually demanding role. The Data Scientist anchors the evaluations function for this vertical — designing, building, and maintaining evaluation frameworks that measure model and system quality in operational context. You are not running standard benchmarks. You are building domain-specific eval harnesses for high-stakes use cases where a wrong answer carries real consequences. You will work closely with the MLOps Engineer, the PM, and the deployment team. The evaluations you design are the mechanism by which the team determines whether what we've built is good enough to deploy — and whether it stays good after deployment.

## What You'll Do

- Design and build evaluation frameworks for Sarvam's AI outputs across domain-specific requirements: document comprehension, command summarisation, geospatial reasoning, enterprise workflow automation, and others as they emerge
- Define quality metrics in collaboration with domain experts and clients; translate operational requirements into measurable, defensible signals
- Run structured evaluation cycles pre- and post-deployment; build dashboards that surface model quality in production
- Identify failure modes, edge cases, and distribution shifts — with the bias of someone looking for what's wrong, not confirming what's right
- Collaborate with the MLOps Engineer to operationalise eval pipelines — automated, triggered by deployment events, versioned, and reproducible
- Build and manage domain-specific datasets for fine-tuning, evaluation, and benchmarking — including human annotation workflows where needed
- Publish internal findings and quality reports that feed the product and engineering roadmap

## What We're Looking For

- 3–6 years in data science, ML research, or applied AI; at least 2 years working with LLMs in production contexts
- Strong statistics and probability fundamentals — you understand what makes an evaluation valid and what makes it misleading
- Experience designing evaluation frameworks from scratch: custom metrics, inter-rater reliability, red-teaming methodologies
- Python proficiency; comfort with pandas, NumPy, HuggingFace datasets, RAGAS, EleutherAI Eval Harness, LangSmith, or equivalent
- Experience with prompt engineering, model fine-tuning, or RLHF in applied settings
- Ability to work with unstructured domain data: PDFs, doctrine documents, transcripts, and field reports

Bonus Points

- Prior work in high-stakes domains (healthcare, legal, defence, finance) where output quality carries real-world consequences
- Red-teaming or adversarial evaluation experience

Note: We are looking for people who can own the outcomes described here, not people who match every line of this specification. If this problem excites you and you believe you can do this work, we want to hear from you.

Why Sarvam? Sarvam is a fast-moving, high talent-density team building full-stack AI for India, working on problems that push the frontiers of AI with real population-scale impact.

- Work alongside researchers, engineers, builders, and business leaders who move fast and hold each other to a very high bar
- High ownership and high impact, from day one
- Everything we do is AI-first, from the way we build and ship to the way we think about problems
- You can work on problems that could change how an entire country learns, works, and communicates

If you want to work on problems at the frontier of AI in India, Sarvam is the place to be.

## About Sarvam

## Company Overview
- **One-liner**: Sarvam is building India's full-stack sovereign AI platform, delivering state-of-the-art models and enterprise-grade applications for Indic languages.
- **Entity Type**: Private (Series A – $41M raised)
- **Headquarters**: Bengaluru, India
- **Founded**: August 2023
- **Founders**: Dr. Vivek Raghavan, Dr. Pratyush Kumar

## Core Business
- Primary industry: Artificial Intelligence / Generative AI
- Target customers: Enterprises, Government, Developers (B2B, B2G)
- Mission: Make AI widely accessible to everyone in India by building foundational components that reflect how the country thinks, speaks, reasons, and solves problems.

## Products & Services
- **Sarvam Cloud**: Fully managed, automatically scaling API platform for Indic AI capabilities.
- **Speech to Text (Saaras)**: High‑accuracy ASR across 12 Indic languages.
- **Text to Speech (Bulbul)**: Natural voices across 11 Indic languages.
- **Translation**: State‑of‑the‑art translation across 23 languages.
- **Document Digitisation (Akshar)**: Convert PDFs and images into structured data.
- **Vision Models**: Image understanding and processing.
- **Samvaad**: Conversational AI studio for building voice agents.
- **Arya**: AI‑for‑work platform with forward‑deployed engineers for custom agent design and integration.
- **Deployment Options**: Private cloud, on‑premise, hybrid, or air‑gapped environments.

## Market Standing
- **Valuation/Market Cap**: Not disclosed (private)
- **Key Metric**: Total funding $41M (from Khosla Ventures, Lightspeed, Peak XV Partners)
- **Notable Investors/Partners**: Khosla Ventures, Lightspeed, Peak XV Partners
- **Growth Signals**: 62 open positions across engineering, models, product, sales, and more; case studies with large‑scale deployments (50,000+ farmer feedback calls, millions of policy calls); SOC 2 Type II, ISO 27001, DPDP compliant.

## Competitive Advantages
- **Sovereign AI**: All technology developed and operated entirely in India, with full data control and compliance with Indian data protection laws.
- **Full‑stack platform**: From foundational models to enterprise agents, everything built in‑house.
- **Forward‑deployed engineering**: Dedicated engineers embedded with customer teams to ensure successful delivery.
- **Population‑scale impact**: Proven ability to serve millions of users in Indic languages across government and BFSI.
- **Deployment flexibility**: Can run on any infrastructure, including air‑gapped for regulated industries.

## Strategic Focus
- Continue building state‑of‑the‑art models and tools for Indic languages.
- Expand enterprise and government adoption of conversational AI agents.
- Nurture the Indian AI ecosystem by open‑sourcing models and tools.
- Scale team rapidly (62 open roles) to support growing customer demand.

## Why Work Here
- **Mission‑driven opportunity**: Shape AI that serves India’s linguistic and cultural diversity.
- **On‑site first**: All roles are based in Bengaluru (Delhi also available), fostering deep collaboration.
- **Cutting‑edge work**: Foundational model research, ML infrastructure, full‑stack product engineering.
- **High‑impact culture**: Teams work directly with customers via forward‑deployed engineers, seeing immediate real‑world use.
- **Backed by top investors**: Financial stability and growth runway from Lightspeed, Peak XV, and Khosla Ventures.

## Sources
1. [sarvam.ai](https://www.sarvam.ai/)
2. [sarvam.ai/about-us](https://www.sarvam.ai/about-us)
3. [sarvam.ai/careers](https://www.sarvam.ai/careers)
4. [jobs.ashbyhq.com/sarvam](https://jobs.ashbyhq.com/sarvam)
5. [linkedin.com/company/sarvam-ai](https://www.linkedin.com/company/sarvam-ai)

## Other roles at Sarvam

- [Software Engineer - Model API Platform](https://feeny.ai/job/software-engineer-model-api-platform-sarvam-bengaluru-6252xtzyzy9f) — Bengaluru, India
- [ML Engineer (Training Infra), Foundational Models](https://feeny.ai/job/ml-engineer-training-infra-foundational-models-sarvam-bengaluru-byw6f3s2ty06) — Bengaluru, India
- [Staff Engineer - Telephony](https://feeny.ai/job/staff-engineer-telephony-sarvam-bengaluru-4mt7q90ph60d) — Bengaluru, India
- [Staff Engineer - Systems Engineering](https://feeny.ai/job/staff-engineer-systems-engineering-sarvam-bengaluru-z6az3vchvm99) — Bengaluru, India
- [Growth Marketing Intern](https://feeny.ai/job/growth-marketing-intern-sarvam-bengaluru-nd6d1ycn199k) — Bengaluru, India
- [Marketing Intern](https://feeny.ai/job/marketing-intern-sarvam-bengaluru-d560v4tmah8c) — Bengaluru, India
- [Staff Software Engineer - Frontend](https://feeny.ai/job/staff-software-engineer-frontend-sarvam-bengaluru-tae371pf8h7r) — Bengaluru, India
- [GTM Strategy - Chanakya](https://feeny.ai/job/gtm-strategy-chanakya-sarvam-delhi-zp2xxvmq6s2z) — Delhi, India
- [DevOps Engineer](https://feeny.ai/job/devops-engineer-sarvam-bengaluru-5ss75qfs1rn0) — Bengaluru, India
- [Product Manager, Model APIs](https://feeny.ai/job/product-manager-model-apis-sarvam-bengaluru-jrkanrnc39wr) — Bengaluru, India