---
title: 'Data Engineer, Platform at Basis Research Institute'
canonical: 'https://feeny.ai/job/data-engineer-platform-basis-research-institute-new-york-z7jf6pnk7yr3'
type: 'job'
last_seen: '2026-09-10'
---

# Data Engineer, Platform at Basis Research Institute

- **Company:** Basis Research Institute
- **Location:** New York, NY
- **Employment:** full-time
- **Posted:** 2025-11-23
- **Last confirmed live:** 2026-09-10
- **Apply:** https://jobs.ashbyhq.com/basis-research/97ff8c58-b594-44b8-886c-21cf631dad1c

## Job description

## About Basis

[Basis](https://www.basis.ai) is a nonprofit applied AI research organization with two mutually reinforcing goals. The first is to understand and build intelligence. This means to establish the mathematical principles of what it means to reason, to learn, to make decisions, to understand, and to explain; and to construct software that implements these principles. The second is to advance society’s ability to solve intractable problems. This means expanding the scale, complexity, and breadth of problems that we can solve today, and even more importantly, accelerating our ability to solve problems in the future. To achieve these goals, we’re building both a new technological foundation that draws inspiration from how humans reason, and a new kind of collaborative organization that puts human values first.

## About the Role

Data Engineers on the Platform team at Basis build trustworthy data pipelines with comprehensive provenance and quality gates, curate documented datasets for training and evaluation, and ensure data infrastructure scales reliably. You will work on both platform-specific data needs and cross-project data coordination, preventing duplicate work and facilitating shared datasets. We are looking for people who are technically excellent and treat data quality as a first-class concern. The ideal Data Engineer has experience with ML data pipelines, understands the full lifecycle from raw data through model training and evaluation, and brings rigor to data provenance, lineage tracking, and quality assurance. You combine software engineering discipline with deep understanding of data systems and ML requirements. This role is embedded across Platform and Research teams, working on infrastructure that supports both commercial offerings and internal research. You will help Basis scale data operations to support medium-scale models, ensure data governance as we serve external customers, and build systems that researchers can trust for reproducible experiments. We seek individuals who aspire to do rigorous, high-quality, robust data engineering, but are not afraid to iterate, learn from real usage, and explore different approaches to achieve excellence. Basis is a collaborative effort, both internally and with our external partners; we are looking for people who enjoy building data foundations for problems larger than ones they can tackle alone.

We expect you to:

- Have demonstrated significant achievements in data engineering for ML/AI systems. Examples include:
- Building data pipelines for model training or evaluation at scale
- Developing feature stores or data platforms serving multiple teams
- Creating data quality frameworks and implementing governance systems
- Designing data architectures that enabled new ML capabilities
- Possess strong proficiency in data technologies including SQL (expert level), Python for data processing, distributed computing frameworks (Spark, Dask), and workflow orchestration tools (Airflow, Dagster, Prefect).
- Have experience with cloud data platforms including data warehouses (Snowflake, BigQuery, Redshift), data lakes, object storage (S3), and streaming systems (Kafka, Kinesis, Flink) for both batch and real-time processing.
- Understand ML data requirements including feature engineering, training/validation/test splits, data versioning, experiment reproducibility, and the specific data needs of different model types and training procedures.
- Be skilled at data quality and governance including implementing validation frameworks, anomaly detection, data lineage tracking, metadata management, and ensuring compliance with privacy and security policies.
- Have knowledge of data modeling principles for both relational and NoSQL systems, understanding of schema design, normalization/denormalization tradeoffs, and performance optimization.
- Value data provenance and documentation. You ensure data pipelines are transparent, decisions are documented, and others can understand and trust the data you deliver.
- Progress with autonomy on complex data challenges. You can scope data projects, make sound architectural decisions, and deliver complete solutions from ingestion through consumption.
- Be excited about enabling rigorous research through trustworthy data infrastructure that advances our ability to solve intractable problems.

In addition, the following would be an advantage:

- Experience with feature stores (Tecton, Feast) or building feature platforms.
- Background in ML research or research engineering providing understanding of data needs across experiment lifecycle.
- Experience with data lineage tools (Apache Atlas, DataHub, Monte Carlo) and metadata management.
- Knowledge of vector databases and embedding pipelines for modern AI applications.
- Contributions to data engineering open-source projects (Airflow, dbt, Great Expectations).
- Understanding of responsible AI and data governance practices.

Responsibilities:

- Design and build data pipelines for training and evaluation across Basis research projects and platform offerings, ensuring reliability, performance, and scalability.
- Implement data quality frameworks including validation rules, quality gates, anomaly detection, and monitoring that catch data issues before they impact research or production systems.
- Develop and maintain feature stores or equivalent systems that enable consistent feature access across training and serving environments, preventing train-serve skew.
- Ensure data provenance and lineage tracking so researchers and engineers can understand data origins, transformations applied, and dependencies, enabling reproducible experiments and debugging.
- Curate documented datasets for model training and evaluation, including dataset versioning, comprehensive documentation, quality metrics, and metadata that enables appropriate usage.
- Coordinate cross-project data initiatives to prevent duplicate data work, facilitate shared datasets, and ensure consistent data practices across Basis as the organization scales.
- Optimize data infrastructure for scale as compute grows, including cost optimization, performance tuning, caching strategies, and efficient data access patterns.
- Collaborate with research and engineering teams to understand data needs, translate requirements into technical solutions, and provide consultation on data architecture and best practices.
- Implement data governance policies ensuring compliance with privacy regulations, security requirements, and responsible AI practices as Basis serves external customers.
- Contribute to the culture and direction of Basis by modeling data quality rigor, documentation excellence, and focus on trustworthy data infrastructure.

## Role Details

Exceptional candidates who may not meet all of the following criteria are still encouraged to apply.

- FT/PT: Full-time.
- In-person Policy: We are in the office four days a week. Be prepared to attend multi-day Basis-wide in-person events.
- Location: New York City.
- Salary range: Competitive salary.

Non-Discrimination Notice Basis Research Institute provides equal employment opportunities without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, or genetics and prohibits discrimination based on all protected characteristics.

Privacy Notice By submitting your application, you grant Basis permission to use your materials for both hiring evaluation and recruitment-related research and development purposes. Your information may be processed in different countries, including the US. You retain copyright while providing Basis a license to use these materials for the stated purposes. Read our full Global Data Privacy Notice [here](https://www.basis.ai/join-us/privacy-notice/).

## About Basis Research Institute

## Company Overview
- **One-liner**: Basis Research Institute is a nonprofit applied research organization advancing artificial intelligence through probabilistic modeling and causal inference to solve society's most intractable problems, building open-source software that draws inspiration from human reasoning.
- **Entity Type**: Private (Nonprofit 501(c)(3) Research Institute)
- **Headquarters**: New York City, New York, USA (with additional office in Cambridge, Massachusetts, USA)
- **Founded**: 2022
- **Founders**: Zenna Tavares, Emily Mackevicius, and Eli Bingham

## Core Business
- **Primary industry**: Artificial Intelligence Research & Development
- **Target customers**: Municipal governments (via consulting), academic collaborators, for-profit companies (via contract work), and the broader open-source AI community
- **Mission**: Two mutually reinforcing goals: (1) advance basic scientific research in artificial intelligence to understand and build intelligence, and (2) help solve society’s intractable problems by expanding the scale, complexity, and breadth of problems that can be solved

## Products & Services
- **Core Technology Development**: Advanced probabilistic modeling and causal inference frameworks implemented as open-source software, including causal probabilistic programming languages
- **Applied Challenge Projects**:
  - **AI-Driven Robot Design**: Using AI to automate and accelerate robot design and control
  - **Dynamical Modeling for Biological Systems**: Modeling complex biological processes (e.g., neuroscience, cellular dynamics)
  - **Participatory City Modeling**: Consulting with municipal governments to model urban systems and support data-driven policy decisions
  - **Modeling, Abstraction, and Reasoning Agent (MARA)**: A research project focused on building AI agents capable of abstract reasoning and world modeling
- **Postdoctoral Fellowship Program**: A fellowship for early-career scientists to direct their own research within Basis’s mission
- **Open-Source Software Releases**: Publishing research tools and libraries for the broader AI community

## Market Standing
- **Valuation/Market Cap**: Not applicable (nonprofit research organization)
- **Key Metric**: Funded through a combination of research grants, private philanthropy, and contract work with partners; specific funding amounts are not publicly disclosed
- **Notable Investors/Partners**: Supported by an advisory board including Armando Solar-Lezama (MIT), Joshua Tenenbaum (MIT), Kevin Ellis (Cornell), Anthony Philippakis (Broad Institute), and Rui Costa (Columbia University). Collaborates with academic institutions (MIT, Columbia, Cornell, NYU) and municipal governments
- **Growth Signals**: Headcount has grown 41.7% year-over-year (LinkedIn data); active job postings increased 107.1% year-over-year; expanding into London with a GTM Lead role; 29 active job openings as of mid-2026

## Competitive Advantages
- **Nonprofit Structure**: Unlike for-profit AI labs, Basis is a 501(c)(3) nonprofit, enabling it to prioritize open-source, public-good research over commercial outcomes
- **Unique Research Cycle**: A structured cycle between core technology development and applied challenge projects, where insights from real-world problems feed back into foundational research
- **Interdisciplinary Approach**: Combines insights from cognitive science (human reasoning), computer science (probabilistic programming), and domain expertise (biology, urban planning, robotics)
- **High-Profile Advisory Board**: Advisors from leading institutions (MIT, Cornell, Broad Institute, Columbia) provide deep academic and scientific credibility
- **Open-Source Commitment**: All software is released as open-source, building community trust and enabling broad adoption

## Strategic Focus
- **Core AI Foundations**: Advancing probabilistic modeling, causal inference, program synthesis, neuro-symbolic methods, reinforcement learning, and world models
- **Challenge-Driven Research**: Using applied projects (robotics, biology, city modeling) to identify shared bottlenecks that guide further methodological research
- **Collaborative Expansion**: Actively seeking partnerships with academia, government, social sector, and industry to tackle high-impact problems
- **Geographic Growth**: Opening presence in London (UK/EU GTM) and expanding roles in Ithaca, NY, while maintaining NYC and Cambridge hubs
- **Open-Source Ecosystem**: Building durable foundations that enable other researchers and practitioners to build upon Basis’s work

## Why Work Here
- **Mission-Driven Culture**: Employees work on problems that matter beyond publications, with impact on AI science and real-world societal challenges
- **Autonomy and Growth**: Organizational structure allows employees to see their autonomy and responsibilities grow with their projects, distinct from traditional career ladders
- **Collaborative Environment**: Work alongside researchers from top institutions (MIT, Columbia, Cornell) and domain experts from academia, government, and industry
- **Open-Source Impact**: Ship high-quality open-source software used by thousands, contributing to the broader AI community
- **Locations**: Offices in NYC (primary) and Cambridge, MA, with remote flexibility for some roles; London expansion underway
- **Notable Perks**: Postdoctoral fellowships available for early-career scientists; focus on human values and collaborative organization
- **Engineering Culture**: Emphasis on building durable foundations, shipping production-quality code, and working at the intersection of research and engineering

## Sources
1. [basis.ai/about](https://www.basis.ai/about/)
2. [basis.ai](https://www.basis.ai/)
3. [jobs.ashbyhq.com/basis-research](https://jobs.ashbyhq.com/basis-research)
4. [basis.ai/join-us](https://www.basis.ai/join-us/)
5. [LinkedIn - Basis Research Institute](https://www.linkedin.com/company/basis-ri)

## Other roles at Basis Research Institute

- [Research Operations Manager](https://feeny.ai/job/research-operations-manager-basis-research-institute-new-york-95tarn9pj8tz) — New York, NY
- [GTM Lead, UK/EU & US](https://feeny.ai/job/gtm-lead-uk-eu-us-basis-research-institute-london-b9sq0m016y20) — London, United Kingdom
- [Hardware Systems & Lab Engineer, Robotics & Operations](https://feeny.ai/job/hardware-systems-lab-engineer-robotics-operations-basis-research-institute-new-g6a4cmtj6jzd) — New York, NY
- [ML Systems Engineer, Infrastructure & Cloud](https://feeny.ai/job/ml-systems-engineer-infrastructure-cloud-basis-research-institute-new-york-fmkf7rgb6w6r) — New York, NY
- [Research Engineer, Operations](https://feeny.ai/job/research-engineer-operations-basis-research-institute-new-york-55vvr38p29sn) — New York, NY
- [Research Engineer, Platform](https://feeny.ai/job/research-engineer-platform-basis-research-institute-new-york-3sep3r6cksy6) — New York, NY
- [Research Scientist, Program Synthesis & Neuro-symbolic Methods](https://feeny.ai/job/research-scientist-program-synthesis-neuro-symbolic-methods-basis-research-yw82mzv4975d) — New York, NY
- [Research Scientist, Reinforcement Learning](https://feeny.ai/job/research-scientist-reinforcement-learning-basis-research-institute-new-york-gprjb8ja06q7) — New York, NY
- [Research Scientist, World Models](https://feeny.ai/job/research-scientist-world-models-basis-research-institute-new-york-xwd9d07xy13d) — New York, NY
- [Postdoctoral Fellow, Open Call](https://feeny.ai/job/postdoctoral-fellow-open-call-basis-research-institute-new-york-gpaxsvg2kqhc) — New York, NY