--- title: 'Research Scientist - VLM Pretraining at Epsilon Labs, Inc.' canonical: 'https://feeny.ai/job/research-scientist-vlm-pretraining-epsilon-labs-inc-san-francisco-e9sv1335xj0a' type: 'job' last_seen: '2026-09-09' --- # Research Scientist - VLM Pretraining at Epsilon Labs, Inc. - **Company:** Epsilon Labs, Inc. - **Location:** San Francisco, CA - **Employment:** full-time - **Work type:** onsite - **Posted:** 2026-08-31 - **Last confirmed live:** 2026-09-09 - **Apply:** https://jobs.ashbyhq.com/epsilon-health/d6258ef8-7cd3-47d5-b48d-44c177b7b068 ## Job description ## ABOUT US We're tackling one of healthcare's most critical challenges in medical imaging and diagnostics. Our company operates at the intersection of cutting-edge AI and clinical practice, building technology that directly impacts patient outcomes. We've assembled one of the industry's most comprehensive and diverse medical imaging datasets and have a proven product-market fit with a substantial customer pipeline already in place. ## ROLE OVERVIEW We're seeking a Research Scientist with deep expertise in large-scale vision-language pretraining to join our ML Research team. You'll be at the forefront of developing state-of-the-art multimodal models for clinical use in radiology settings. This role owns the pretraining stage of our radiology report generation model: VLM architecture design, multimodal data and task mixtures, and the large-scale training runs that build grounded visual understanding across X-rays, CT scans, and MRI. You'll work with one of the largest and most diverse medical imaging datasets in the industry, paired with the reports that make multimodal pretraining at this scale possible, while maintaining the clinical rigor required for healthcare deployment. Post-training and RL are owned by a partner role you'll collaborate with closely. ## KEY RESPONSIBILITIES - Design, train, and scale vision-language foundation models for radiology applications, owning the pretraining stage end to end. - Develop VLM architectures suited to medical imaging, including native and variable resolution handling, high-resolution tiling, connector design, and token budgets for volumetric studies. - Build and tune multimodal pretraining mixtures across captioning, VQA, grounding, and retrieval tasks, balancing data sources to avoid regressions in language capability. - Develop fine-grained visual grounding during pretraining, enabling models to localize findings within medical images using bounding boxes or segmentation masks. - Own pretraining evaluation (zero- and few-shot transfer, probing, and downstream fine-tunability) — as the signal for base model quality. - Train joint vision-language embedding spaces using contrastive and generative objectives, including region- and sentence-level alignment between images and reports. - Contribute hands-on to all stages of pretraining including dataset curation, architecture design, distributed training, and handoff of base checkpoints to post-training. - Stay current with cutting-edge research in vision-language modeling and large-scale multimodal pretraining. - Drive research and technical excellence through conference publications and technical blog posts, establishing best practices for pretraining medical VLMs at scale. ## QUALIFICATIONS - 6+ years of academia/industry experience in vision-language modeling, multimodal learning, or related fields - Deep expertise in pretraining large vision-language models (e.g., LLaVA, Flamingo, CogVLM, Qwen-VL, InternVL, or similar architectures) - Strong foundation in modern VLM pretraining techniques including: - Vision-language connector and fusion architectures (projection, cross-attention, resampler-based) - Variable and high-resolution image handling (native resolution, dynamic tiling, token compression) - Contrastive and generative objectives for learning joint vision-language embedding spaces - Data and task mixture design, including curriculum and mixture-ratio ablations - Experience with fine-grained visual grounding (referring expression comprehension, phrase grounding, box or mask prediction) - Track record of implementing complex models from research papers and adapting them to new domains - Proficiency in PyTorch or JAX, with experience training large models on multi-GPU/distributed systems - Experience with autoregressive language modeling and long-context training - Hands-on experience with medical imaging applications, particularly radiology report generation - Strong software engineering skills and ability to write production-quality code ## PREFERRED QUALIFICATIONS - Publications at top-tier conferences (NeurIPS, ICML, ICLR, CVPR, ACL, EMNLP, MICCAI) - Experience training vision encoders from scratch, or co-designing them with a downstream VLM - Experience with interleaved image-text pretraining and synthetic recaptioning pipelines - Experience with 3D medical image processing and temporal modeling - Familiarity with clinical NLP and medical knowledge representation - Knowledge of evaluation methodologies for long-form generation, including factuality assessment and hallucination detection - Experience with model interpretability, explainability, and uncertainty quantification in safety-critical applications ## About Epsilon Labs, Inc. ## Company Overview - **One-liner**: Epsilon Health is a tech-enabled teleradiology practice that combines board-certified radiologists with proprietary AI to deliver accurate, rapid diagnostic reports. - **Entity Type**: Private (venture-backed startup, pre-Series A) - **Headquarters**: San Francisco, CA, USA - **Founded**: 2024 - **Founders**: Not publicly disclosed (key executives include Dr. Roi Bittane – Chief Medical Officer, and Rustin Rassoli – Founder/Executive; team also includes leaders from Google DeepMind, Meta, and Twitch) ## Core Business - **Primary industry**: Healthcare – Teleradiology / Medical Imaging / Health Tech - **Target customers**: B2B – Imaging centers, hospitals, health systems, and payers in the United States - **Mission**: “To make sure no diagnosis is missed, delayed, or wrong.” ## Products & Services - **Epsilon Platform (Proprietary AI + Workflow)**: A SaaS-enabled interpretation service that integrates AI models for triage, flagging critical findings, and automating busywork. Radiologists use the platform to increase speed and accuracy while reducing burnout. - **Teleradiology Services**: 24/7 remote radiology interpretation with a turnaround time of 24 hours or less; critical findings flagged immediately. Combines human expertise with AI assistance. ## Market Standing - **Valuation/Market Cap**: Not publicly available (early-stage startup) - **Key Metric**: Total funding not disclosed; backed by venture studio Atomic (based on team background). Headcount ~6 employees (as of mid-2025). - **Notable Investors/Partners**: Implied backing from Atomic; no formal funding announcement found. - **Growth Signals**: Founded in 2024, actively hiring for Research Scientist, Research Engineer, and Senior Backend Engineer roles. Addresses a massive market gap: 700 million scans/year in the US, with a radiologist shortage projected to reach 15,000 by 2030. ## Competitive Advantages - **Integrated AI + Human Workflow**: Unlike point solutions that fail in production, Epsilon builds AI directly into its practice, allowing continuous iteration across hospitals and imaging centers. - **Speed & Accuracy**: 75% of critical findings flagged immediately; reports delivered in 24 hours or less. - **Radiologist-Centric Design**: Removes administrative busywork to let radiologists focus on interpretation, reducing burnout. - **Team Depth**: Combines clinical leadership (former CMO of Envision Radiology) with top-tier ML engineering (Google DeepMind, Meta, Twitch alumni). ## Strategic Focus - **Scale the practice** to meet growing imaging demand by onboarding more radiologists and imaging center partners. - **Deepen AI capabilities** for automated triage, detection, and workflow optimization. - **Expand partnerships** with health systems and payers to improve patient outcomes and reduce costs. ## Why Work Here - **Culture**: “Radiology rebuilt from the ground up” – a mission-driven environment focused on solving a critical healthcare crisis. Emphasis on collaboration between radiologists, engineers, and technologists. - **Work Policy**: Hybrid – in-office presence in San Francisco (SOMA area) with remote flexibility for certain roles (Built In lists both “In-Office” and “Remote Workspace” options). - **Engineering Culture**: Small, high-impact team with autonomy; roles span ML infrastructure, computer vision, and backend systems. Opportunity to shape the product from an early stage. - **Perks**: Not detailed, but typical for early-stage health tech (likely equity, health benefits, and the chance to work on life-saving technology). ## Sources 1. [epsilon.health](https://www.epsilon.health/) – Company homepage and product description 2. [epsilon.health/about](https://www.epsilon.health/about) – Mission, background, and statistics 3. [epsilon.health/team](https://www.epsilon.health/team) – Leadership and engineering team 4. [builtin.com/company/epsilon-health](https://builtin.com/company/epsilon-health) – Office location, headcount, and work policy 5. [jobs.ashbyhq.com/epsilon-health](https://jobs.ashbyhq.com/epsilon-health) – Active job openings ## Other roles at Epsilon Labs, Inc. - [Research Scientist - Vision Foundation Models](https://feeny.ai/job/research-scientist-vision-foundation-models-epsilon-labs-inc-san-francisco-mzemvmd5nd0b) — San Francisco, CA - [Research Scientist - Post-training / RL](https://feeny.ai/job/research-scientist-post-training-rl-epsilon-labs-inc-san-francisco-88p5ss3dcaga) — San Francisco, CA - [Research Engineer - Data Quality & Evals](https://feeny.ai/job/research-engineer-data-quality-evals-epsilon-labs-inc-san-francisco-bvtb488px6jv) — San Francisco, CA - [Software Engineer - Product](https://feeny.ai/job/software-engineer-product-epsilon-labs-inc-san-francisco-y7ttr13zgn08) — San Francisco, CA - [Software Engineer - ML Infrastructure](https://feeny.ai/job/software-engineer-ml-infrastructure-epsilon-labs-inc-san-francisco-fbhbgrb0sk47) — San Francisco, CA - [Software Engineer - Core Systems](https://feeny.ai/job/software-engineer-core-systems-epsilon-labs-inc-san-francisco-55xn9qbzrags) — San Francisco, CA - [Reading Radiologists](https://feeny.ai/job/reading-radiologists-epsilon-labs-inc-remote-jr85tcf6zpp3)