---
title: 'Model Test and Measurement Engineer at DEFCON AI'
canonical: 'https://feeny.ai/job/model-test-and-measurement-engineer-defcon-ai-united-states-2re9pe19frwr'
type: 'job'
last_seen: '2026-11-06'
---

# Model Test and Measurement Engineer at DEFCON AI

- **Company:** DEFCON AI
- **Location:** United States
- **Work type:** remote
- **Posted:** 2026-10-01
- **Last confirmed live:** 2026-11-06
- **Apply:** https://job-boards.greenhouse.io/defcon/jobs/5253367007

## Job description

## ABOUT DEFCON AI

RESILIENCE IN THE FACE OF DISRUPTION. DEFCON AI is an insights company that leverages artificial intelligence, mathematical optimization, data analytics, and software engineering for resilient optimization of complex systems. In today’s dynamically changing world, DEFCON AI’s technology aligns outcomes with operational goals, better decision making, and empowers customers to anticipate assess, and mitigate the impacts of disruptions.

Be the independent voice that keeps the whole program honest — your evaluation is the standard everyone else is held to.

## About the Role

You'll join the analytics and AI engineering team behind a system that genuinely matters: an AI-assisted platform that brings together records from dozens of disparate data sources, resolves them to the correct individual, highlights what analysts should review first, and provides transparent, explainable recommendations that users can trust. Operating within a secure government cloud environment, the platform tackles complex challenges in AI, data integration, and decision support where quality, trust, and accountability are mission-critical.

As a Model Test & Measurement Engineer, you'll own the evaluation framework that helps ensure those systems perform as intended. You'll build and maintain labeled ground truth datasets, design statistically sound audit and sampling methodologies, measure model performance across releases, and create the evidence packages that support deployment decisions. You'll independently validate both scoring and generative AI capabilities, helping the team understand not only whether a model works, but how confidently its outputs can be trusted.

This is a role with genuine influence. Your assessments will inform release decisions, drive improvement efforts, and provide the objective evidence customers rely on when evaluating system performance. You'll work closely with data scientists, AI engineers, and technical leadership while maintaining the independence needed to provide clear, defensible evaluations. You do not build the models you validate, and you do not approve thresholds or release authorization against your own evidence.

If you're energized by measurement, validation, and making complex AI systems more trustworthy, this is an opportunity to have an outsized impact on both the technology and the mission it supports.

This is a fully remote role with occasional travel to DEFCON AI headquarters, customer sites, and partner facilities as needed.

## Key Responsibilities

- Construct labeled ground truth for model evaluation
- Design and run sampled audits
- Gate releases against model version; maintain version inventory, evaluation records, and rollback triggers
- Run drift and override review
- Produce human-oversight and fairness / disparate-effect evidence
- Maintain independence from the build roles: produce evaluation evidence, but do not build the models, approve thresholds, or authorize releases against that evidence
- Measure workflow improvement: review time, throughput, backlog movement, and override and rework rates
- Design evaluation-phase QC: sample selection that does not mix populations, the unit of review, and fair comparison when methods or searched sources differ
- Define the measurement events other teams must emit, and confirm the review workspace emits workflow events from first use

Required Qualifications

- 5+ years with model validation as a named responsibility, not a side task
- Direct experience with ground-truth construction and sampling design
- Strong Python and SQL; comfort working independently from the teams whose models you evaluate
- US Citizenship Required
- Active US Secret clearance
- Elevated personnel security requirements apply to portions of this work and are discussed during screening

## Preferred Qualifications

- Regulated-industry or government model-risk background
- Experience with fairness / disparate-effect testing and human-oversight documentation
- NIST AI RMF or comparable practice
- Active Top Secret clearance

What Success Looks Like

- Evaluation evidence a customer can rely on, independent of the teams that build the models
- Releases gated against a clear version and evaluation record
- Workflow metrics (review time, throughput, backlog, override/rework rates) that give the program an honest read on whether it's working

## What We Offer

- A fully remote, results-based environment
- Competitive salary, bonus, and equity package
- 100% employer paid, comprehensive health insurance including medical, dental, and vision for you and your family
- Unlimited PTO, with your manager's approval
- Flexible work environment where you manage your work day
- 14 weeks of fully-paid parental leave

Salary Range: $150,000–$190,000. This represents the typical salary range for this position based on experience, skills, and other factors. We’re an Equal Opportunity Employer: You’ll receive consideration for employment without regard to race, sex, color, religion, sexual orientation, gender identity, national origin, protected veteran status, or on the basis of disability.

Applicant Data Disclosure

By submitting an application, you acknowledge that Defcon AI uses third-party service providers to facilitate its recruitment and hiring processes. These providers include applicant tracking systems, candidate verification platforms, and fraud detection tools (collectively, "Hiring Platforms"). Your application materials, including your résumé, cover letter, work samples, responses to application questions, and any other information you submit, may be transmitted to and processed by these Hiring Platforms for the following purposes:

- Managing and administering your application throughout the hiring process;
- Verifying the accuracy and authenticity of application materials, including by cross-referencing information you provide against publicly available sources and proprietary databases;
- Identifying indicators of potentially fraudulent, fabricated, or materially misleading application content, including but not limited to discrepancies between submitted materials and publicly available professional profiles, geographic anomalies, and fabricated work histories.

Applications that are flagged through this process as containing indicators of fraud or material misrepresentation may be declined from further consideration. If you have questions about the status of your application or the evaluation process, please contact recruiting@defconai.com.

Defcon AI requires its Hiring Platform providers to process your information solely for the purposes described above and in accordance with applicable law. Your information will be retained only for as long as necessary to fulfill these purposes and any applicable legal obligations, after which it will be deleted in accordance with Defcon AI's data retention policies.

For more information about how your data is used, please refer to our Privacy Policy and [Applicant Privacy Notice](https://nam09.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redcellpartners.com%2Fapplicant-privacy-policy%2F&data=05%7C02%7Ckat.creamer%40redcellpartners.com%7Cc0f94f3daed94dc7503108de8b61955e%7Cf861de501a2a42359dbf28afd57d1d97%7C0%7C0%7C639101448841578570%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=WoPiBMZqQyPPK5N4E1sCagkbMY2S8P3aSRSoy7T4los%3D&reserved=0).

## About DEFCON AI

## Company Overview
- **One-liner**: DEFCON AI builds AI-powered decision-augmentation platforms that optimize complex operations under uncertainty for national security, manufacturing, logistics, and healthcare organizations.
- **Entity Type**: Private (Seed stage; raised $44M total funding)
- **Headquarters**: McLean, Virginia, United States
- **Founded**: 2022
- **Founders**: Yisroel Brumer (Co-Founder & CEO) and Grant Verstandig (Co-Founder & Executive Chairman)

## Core Business
- **Primary industries**: National Security/Defense, Manufacturing, Logistics & Transportation, Healthcare
- **Target customers**: B2B — primarily U.S. Department of Defense and other government agencies, as well as large commercial enterprises in manufacturing, logistics, and healthcare.
- **Mission or purpose**: To deliver decision-augmentation capabilities that help organizations anticipate disruption, evaluate alternatives in real time, and make confident decisions before disruptions cascade.

## Products & Services
- **ARTIV Air**: Cloud-native optimization platform for airlift planners to rapidly generate and evaluate multiple mission scenarios, balancing effectiveness, efficiency, and resiliency for routing mission-critical cargo and personnel.
- **R-ALIGN**: Enables logistics planners to model and evaluate global multimodal operations (air, sea, rail, road) using time-sensitive optimization in contested and dynamic environments.
- **FlightForge**: AI-driven tool that delivers mission-ready flight schedules in seconds, balancing key factors for accuracy and efficiency with real-time crew-to-sortie visibility.
- **DockForge**: Optimizes ship maintenance scheduling by assigning jobs across dry docks and shipyards, maximizing throughput and allowing planners to explore trade-offs.
- **R-IMS (Resilient Infrastructure Management System)**: Provides real-time view of capacity, dependencies, and schedule risk through simulation and scenario analysis for infrastructure and program teams.
- **HelioNet**: Helps enterprises design resilient, cost-efficient global logistics networks through advanced modeling and long-horizon optimization (warehouse/hub location, tariff/disruption scenarios).
- **SimSource**: Advanced optimization, simulation, and forecasting platform for managing complex, nested scheduling demands under uncertainty in manufacturing.
- **Synapse Scheduler**: Uses optimization, ML, and LLMs to generate workforce schedules that meet staffing needs and individual preferences (targeting healthcare).

## Market Standing
- **Valuation/Market Cap**: Not publicly disclosed.
- **Key Metric**: Total funding of **$44.0M** (Seed round led by Bessemer Venture Partners in August 2024; earlier venture round in October 2022).
- **Notable Investors/Partners**: Bessemer Venture Partners (lead), Red Cell Partners, and other undisclosed investors. The company was founded in response to a direct request from the U.S. Department of War.
- **Growth Signals**: Rapid transition from concept to mission-ready platform since founding in 2022; growing network of partners; expansion into commercial sectors (manufacturing, healthcare, logistics) beyond initial defense focus. LinkedIn shows 3,361 followers (+33.4% yearly).

## Competitive Advantages
- **Deep domain expertise**: The team combines senior defense leaders, algorithm developers, and operators, bridging theory and real-world execution.
- **Proven government pedigree**: Founded to meet a direct need from the Department of War, giving it strong credibility and an existing customer base in national security.
- **Cross-industry applicability**: Core optimization technology is platform-agnostic and has been adapted to manufacturing, healthcare, and logistics — not just defense.
- **Focus on uncertainty**: Unlike traditional static planning tools, the entire product suite is built to handle contested, dynamic, and uncertain environments.

## Strategic Focus
- **Current priorities**: Scaling the platform across multiple industries (defense, manufacturing, logistics, healthcare) and deepening capabilities in resilient optimization.
- **Direction for growth**: Expanding the partner network, continuing to win government contracts, and building out commercial applications for complex scheduling and supply chain resilience.

## Why Work Here
- **Culture highlights**: Mission-focused team of world-class AI engineers, data scientists, and mathematics experts working on pressing national challenges. Emphasis on collaboration, transparent communication, and operational impact.
- **Remote/hybrid/office policy**: **Remote-first** — ability to work fully remote within the United States. Certain roles may have in-office requirements.
- **Notable perks**: 100% employer-paid health insurance (medical, dental, vision) for employee and family; Unlimited PTO with management approval (all federal holidays observed); 14 weeks paid parental leave (maternity and paternity) at normal pay; 401K and FSA available; professional development and continued learning opportunities.
- **Engineering culture**: Building with bleeding-edge AI, mathematical optimization, and mission-grade software engineering. The team values rapid iteration and real-world deployment over theoretical work.

## Sources
1. [defconai.com](https://www.defconai.com/)
2. [defconai.com/careers](https://www.defconai.com/careers)
3. [defconai.com/about](https://www.defconai.com/about)
4. [boards.greenhouse.io/defcon](https://boards.greenhouse.io/defcon)
5. [linkedin.com/company/defcon-ai](https://www.linkedin.com/company/defcon-ai)

## Other roles at DEFCON AI

- [Controller](https://feeny.ai/job/controller-defcon-ai-mclean-3f2y567mv0qb) — Mclean, VA
- [IT Help Desk Specialist](https://feeny.ai/job/it-help-desk-specialist-defcon-ai-mclean-yc3pkxnmypjv) — Mclean, VA
- [Senior Entity Resolution Engineer](https://feeny.ai/job/senior-entity-resolution-engineer-defcon-ai-united-states-1vhe7q9xv1a3) — United States
- [ML Engineer, Retrieval & Grounded Generation](https://feeny.ai/job/ml-engineer-retrieval-grounded-generation-defcon-ai-united-states-0hc6tabj65br) — United States
- [Information Extraction Engineer](https://feeny.ai/job/information-extraction-engineer-defcon-ai-united-states-agn4q9kc11ka) — United States
- [Senior Data Platform Engineer](https://feeny.ai/job/senior-data-platform-engineer-defcon-ai-united-states-rjjmm1y0kf5r) — United States
- [Triad Modernization Advisor (1099)](https://feeny.ai/job/triad-modernization-advisor-1099-defcon-ai-offutt-afb-61ek5pfz3hts) — Offutt Afb, NE