LLM Red Team Intern (Evaluation Systems) at Elloe AI (Austin, TX)LLM Red Team Intern (Evaluation Systems)

Internship | Remote | LLM Evaluation | Reports to CTO or Safety Lead

Elloe is the immune system for AI. We don’t train models — we protect their outputs. We trace every hallucination, enforce every policy boundary, and create an audit trail for every critical LLM interaction.

Our modules (TruthChecker™, AutoRAG™, Autopsy™) are embedded in hospitals, banks, and regulatory sandboxes. Our job is to make sure these systems are safe before anything hits production. This role will help us break, stress-test, and harden the models used by governments and enterprises alike.

You’ll red team real-world LLM deployments, design eval harnesses, and help scale Elloe’s output-level safety layer. This isn’t just prompt tuning — it’s forensic risk mapping. You’ll work directly with product and safety leads to uncover failure patterns and codify guardrails for GenAI systems under real-world scrutiny.

1. Red Teaming & Risk Testing

• Create prompts to trigger hallucinations, policy violations, or failure scenarios
• Stress test Elloe-protected deployments using open and proprietary models
• Document behavioral exploits across use cases (healthcare, compliance, gov)

1. Evaluation Design

• Build truthsets and scoring rubrics tied to factuality, policy, or ethical standards
• Benchmark Elloe’s modules across model types (Claude, GPT-4, Gemini, open models)
• Collaborate with product to refine and expand our eval harnesses

1. Safety Intelligence

• Identify blind spots in current detection logic
• Recommend scoring methods or red flag thresholds for deployment
• Support internal model comparison reports or customer safety audits

• ML/AI researcher or engineer (undergrad, grad, or early career)
• Experience working with LLMs, eval sets, and prompt design
• Strong attention to detail, grounded in safety and adversarial thinking
• Bonus: exposure to safety benchmarks like TruthfulQA, MMLU, or red teaming tools

This is real-world alignment, not research theater. You’ll be helping define how AI gets deployed responsibly — with traceability, transparency, and real-time protection.

You’ll leave this role with:

• Exposure to high-stakes LLM safety deployments
• Published frameworks or scoring methods used by enterprises
• Mentorship from technical founders operating at the bleeding edge of AI safety

Logistics & Application

• Start Date: Rolling
• Duration: 12–16 weeks
• Compensation: Research stipend
• Location: Remote-first; flexible for global candidates
• To Apply: Share a jailbreak or eval idea you’d love to run against GPT-4.