---
title: 'Research Scientist, Human Data & Evaluation at Contra'
canonical: 'https://feeny.ai/job/research-scientist-human-data-evaluation-contra-san-francisco-8bxfhw0jrjdc'
type: 'job'
last_seen: '2026-09-10'
---

# Research Scientist, Human Data & Evaluation at Contra

- **Company:** Contra
- **Location:** San Francisco, CA
- **Employment:** full-time
- **Work type:** remote
- **Posted:** 2026-08-17
- **Last confirmed live:** 2026-09-10
- **Apply:** https://jobs.ashbyhq.com/contra/e16cd3ca-082f-4608-abe7-8dc86b595fc7

## Job description

## Who we are and what we do

Contra Labs is a human-centered AI lab focused on creative and multimodal outputs, where human taste defines the next generation of AI capabilities. We build industry-leading creative preference datasets that power benchmarking, evaluation, and post-training for the world's leading AI models and applications. Our work helps define what "good" looks like across design, video, imagery, and beyond. Built on top of Contra, the professional network for independent creatives, Contra Labs connects frontier AI labs with a global network of top creative talent. Together, we turn human judgment into the training and evaluation layer that enables AI models and tools to power the next generation of human creativity.

## Why this role exists

Contra Labs is expanding its research work with frontier AI labs across evaluation, post-training data, benchmarking, and human-centered research. We need a researcher who can take an ambiguous client or research question and independently turn it into a rigorous project: identify the right research direction, develop the methodology, write the proposal, run the work, analyze the results, and produce insights that shape what we or the client should do next. This person should complement our existing project execution capacity by bringing deeper research ownership and the ability to generate net-new ideas and approaches.

## What you'll own

- Research direction & proposals
- Translate ambiguous client questions into clear research questions, hypotheses, and study designs.
- Develop research proposals and methodologies from scratch rather than relying on a predefined approach.
- Identify promising new research directions and push the team toward ideas we have not already considered.
- Help define what Contra Labs should measure, study, and build next.
- Evaluation & data methodology
- Design human-data collection strategies, evaluation protocols, rubrics, annotation schemes, and experiments.
- Develop strong prompt strategies for eliciting, evaluating, and comparing model behavior.
- Determine appropriate sampling, evaluator selection, calibration, quality-control, and reliability methods.
- Design methodologies that work across creative and multimodal domains such as design, image, video, and agentic workflows.
- Analysis & insight generation
- Analyze qualitative and quantitative evaluation data.
- Move beyond reporting results to explain why patterns occur and what they imply.
- Synthesize findings into clear recommendations, research narratives, and new hypotheses.
- Produce client-ready reports, presentations, benchmarks, and research publications.
- End-to-end project ownership
- Independently own research engagements from initial scoping through final delivery.
- Work closely with project leads, creative experts, research engineers, and data/engineering teams.
- Maintain a high bar for research rigor and data quality while operating on fast client timelines.
- Potentially support multiple concurrent projects at different stages.
- Client work
- Be comfortable working on research driven by real client problems and constraints.
- Participate in scoping and proposal development and help establish credibility with sophisticated AI/ML research teams.
- Direct client-facing experience is valuable, but this is not primarily an account-management role. You will collaborate with Strategic Project Leads on client delivery.

## What we're looking for

- 3–5+ years of relevant research experience, with a Master’s, PhD, or equivalent industry research depth in HCI, Human-Centered AI, ML, design research, or a related field.
- Strong qualitative and quantitative research skills, with experience in human evaluation, experimentation, data collection, UX/HCI, annotation, or model behavior research.
- Proven ability to independently design and execute rigorous research, including deciding what data to collect, why, and how to analyze it.
- Ability to turn data and observations into new insights, research directions, and actionable recommendations.
- Comfortable analyzing data with Python and/or SQL.
- Strong written communication and ability to clearly explain methodological tradeoffs and research decisions.
- Comfortable operating independently through ambiguity and balancing research rigor with startup speed and pragmatism.

Bonus

- Familiarity with modern ML and generative AI systems.
- Understanding of RLHF / post-training / preference data / model evaluation.
- Experience studying image, video, multimodal, creative, or agentic systems.
- Experience designing prompts or systematically studying prompting/model behavior.
- Experience working with large-scale human-data or annotation programs.
- Experience working directly with AI research or model teams.
- Strong publication record or other evidence of consistently generating original research ideas.
- Startup / 0-to-1 experience.

## Requirements

- Based in San Francisco, California

Total Comp

- Salary: $150,000 - $200,000 USD + Equity
- Medical, Dental, Vision Benefits
- 401k Matching
- We will provide you with a company laptop on your start date

Interview Process

- Interview with Recruiting Team (20 minutes)
- Interview with Head of Research (30 minutes)
- Technical interview with Data Lead (30 minutes)
- Interview with Head of Contra Labs (30 minutes)
- Paid Case Study and Presentation (60 minutes)

Note: Contra communicates with applicants through @[contra.com](http://contra.com) domains only. We never ask for money from potential employees. For the latest job postings, visit [Contra Careers](https://www.contra.com/careers).

## About Contra

## Company Overview
- **One-liner**: Contra is a commission-free professional network and freelance platform that connects independent creatives and digital professionals with clients and companies for project-based work.
- **Entity Type**: Private (Series B)
- **Headquarters**: San Francisco, California, United States
- **Founded**: 2019
- **Founders**: Not publicly listed on primary sources; the company was started by a team of independents.

## Core Business
- **Primary industry/industries**: Freelance marketplace, professional networking, HR tech / future of work
- **Target customers**: B2B (companies and agencies hiring freelancers) and B2C (independent freelancers, creatives, and digital professionals)
- **Mission or purpose statement**: "A professional network for the jobs and skills of the future." The company is "proudly an independent-first community empowering a new way to work" and believes "remote-first culture, freedom from the 9-5, and a global team is the best way to work." [contra.com](https://contra.com/mission)

## Products & Services
- **Contra Platform (Core Marketplace)**: A commission-free freelance marketplace where independents create profiles, get discovered by clients, and manage projects. Clients pay a flat $29/month per contractor or $29 per project contract fee, while freelancers keep 100% of their earnings. [contra.com](https://contra.com/features/find-freelance-jobs)
- **Contra Pro**: A premium subscription ($29/month) for freelancers offering portfolio customizations, branding tools, and advanced analytics. [contra.com](https://contra.com/features/find-freelance-jobs)
- **Contra for Agencies**: A dedicated offering for agencies to manage freelance talent and projects on the platform. [contra.com](https://contra.com/)
- **Payments & Contracts**: Built-in tools for signing contracts, invoicing, and processing commission-free payments, providing an all-in-one workflow for freelance projects. [contra.com](https://contra.com/features/find-freelance-jobs)

## Market Standing
- **Valuation/Market Cap**: Not publicly disclosed.
- **Key Metric**: Total Funding of **$45.24 million** across 4 rounds (Seed, Series A, Series B, and a 2025 Venture Round). [linkedin.com](https://www.linkedin.com/company/contrahq)
- **Notable Investors/Partners**: New Enterprise Associates (led Series B), Unusual Ventures (led Series A). [linkedin.com](https://www.linkedin.com/company/contrahq)
- **Growth Signals**: 
    - Over **$250 million** in total projects facilitated on the platform. [contra.com](https://contra.com/features/find-freelance-jobs)
    - **1.5 million+** independent experts on the platform. [contra.com](https://contra.com/features/find-freelance-jobs)
    - **50,000+** teams trust the platform. [contra.com](https://contra.com/features/find-freelance-jobs)
    - Headcount grew **72% year-over-year** to 421 employees. [linkedin.com](https://www.linkedin.com/company/contrahq)
    - Active job postings grew **133% year-over-year**. [linkedin.com](https://www.linkedin.com/company/contrahq)
    - LinkedIn followers grew **45.3% year-over-year** to over 61,000. [linkedin.com](https://www.linkedin.com/company/contrahq)

## Competitive Advantages
- **Commission-Free Model**: Unlike competitors (e.g., Upwork, Fiverr) that take a percentage of freelancer earnings, Contra allows independents to keep 100% of what they earn, charging clients a flat fee instead. This is a major differentiator and value proposition for talent. [contra.com](https://contra.com/)
- **All-in-One Workflow**: The platform integrates discovery, contracts, payments, and project management into a single workspace, reducing friction for both freelancers and clients. [contra.com](https://contra.com/features/find-freelance-jobs)
- **Vetted Client Base**: Clients are vetted by Contra, ensuring freelancers work with high-quality, legitimate companies and agencies. [contra.com](https://contra.com/features/find-freelance-jobs)
- **Built by Independents**: The company was founded by freelancers who understood the pain points of existing networks, giving them deep empathy for their core user base. [contra.com](https://contra.com/)

## Strategic Focus
- **Global Talent Expansion**: With a workforce spread across 53 countries and a 72% headcount growth rate, Contra is aggressively building a global, remote-first team. [linkedin.com](https://www.linkedin.com/company/contrahq)
- **Platform Growth**: Scaling the marketplace by attracting more clients and freelancers, evidenced by a 133% year-over-year increase in job postings. [linkedin.com](https://www.linkedin.com/company/contrahq)
- **Product Development**: Continuing to build out the all-in-one workflow for contract work, including payments, contracts, and project management features. [contra.com](https://contra.com/features/find-freelance-jobs)
- **Community Building**: Fostering a global community of independents, with a strong emphasis on design and creative talent (172 employees in Design roles). [linkedin.com](https://www.linkedin.com/company/contrahq)

## Why Work Here
- **Flexible-first culture**: The company explicitly champions remote-first work and freedom from the traditional 9-5, which is core to its product and internal culture. [contra.com](https://contra.com/careers)
- **Global & Distributed Team**: Employees are located across 53 countries, with significant talent hubs in the US, Pakistan, India, Nigeria, and Canada. This offers a truly global work environment. [linkedin.com](https://www.linkedin.com/company/contrahq)
- **Competitive Compensation**: Offers competitive base salaries, pre-IPO stock options, and 100% health coverage for US and Canada full-time employees, plus monthly health reimbursements for international contractors. [contra.com](https://contra.com/careers)
- **Generous Time Off**: 15 paid vacation days, 10 company holidays, 7 flex days, and a full week off at the end of the year. [contra.com](https://contra.com/careers)
- **No Meeting Wednesdays**: A dedicated day each week with no internal meetings to allow for deep, focused work. [contra.com](https://contra.com/careers)
- **Wellness & Equipment**: Reimbursements for home office equipment and other needs to do your best work. [contra.com](https://contra.com/careers)
- **High-Growth Environment**: With 72% annual headcount growth and a global team spanning 53 countries, the company offers a dynamic, fast-paced work environment with significant career growth potential. [linkedin.com](https://www.linkedin.com/company/contrahq)

## Sources
1. [contra.com - Careers Page](https://contra.com/careers)
2. [contra.com - Mission Page](https://contra.com/mission)
3. [contra.com - Homepage](https://contra.com/)
4. [contra.com - Features / Find Freelance Jobs](https://contra.com/features/find-freelance-jobs)
5. [linkedin.com - Company Profile](https://www.linkedin.com/company/contrahq)

## Other roles at Contra

- [Research Scientist, Human Data & Evaluation](https://feeny.ai/job/research-scientist-human-data-evaluation-contra-new-york-cywz7s0yajc0) — New York, NY
- [Special Project Lead](https://feeny.ai/job/special-project-lead-contra-new-york-pfhjeb1s0fn6) — New York, NY
- [Strategic Project Lead](https://feeny.ai/job/strategic-project-lead-contra-san-francisco-8c987bhpvnms) — San Francisco, CA
- [Strategic Project Lead](https://feeny.ai/job/strategic-project-lead-contra-new-york-et98ynas0909) — New York, NY