Seniority: Experienced individual contributor · Location: San Francisco, California, or fully remote · Work model: Remote, hybrid, or onsite; onsite preferred · Employment type: Full-time · Engagement: US employees are engaged on a full-time W-2 basis; international hires are engaged as contractors
About the company
Our client is a biotechnology research company building rigorous scientific benchmarks for AI agents working across biology. The company collaborates with frontier AI labs and pharmaceutical organizations to evaluate whether agents can handle the complex, multi-layered reasoning required in real biological workflows. As part of an early team, you will have the opportunity to shape strategy across multiple benchmarks and contribute to work at the frontier of artificial intelligence and biomedical research. The culture values radical transparency, original expert judgment, respectful collaboration, and a shared focus on solving difficult problems.
About the role
This role will contribute to the technical approach for evaluating how AI agents can support diagnosis, treatment, research, and target discovery in rare and progressive diseases. You will work with scientists, physician-scientists, software engineers, and computational biologists to build ground-truth benchmark datasets that rigorously test the analytical and interpretative steps performed by expert scientists. The role focuses on rare and progressive diseases, including genetic, autoimmune, infectious, idiopathic, neurodegenerative, and chronic progressive conditions, and does not include cancer research.
What you'll do
- Contribute to the technical approach for evaluating how AI agents can support diagnosis, treatment, research, and target discovery in rare and progressive disease.
- Build ground-truth benchmark datasets drawn from scientific studies, professional experience, and novel work.
- Define and evaluate the critical analytical and interpretative steps that expert scientists perform.
- Design and troubleshoot relevant wet-lab or dry-lab experiments, or apply expertise in clinical diagnosis, treatment, and research.
- Work with scientists, physician-scientists, software engineers, and computational biologists to develop benchmark tasks and workflows.
- Help define the strategy across multiple benchmarks.
- Assess where AI agents and large language models fail in scientific or clinical practice.
- Use expert judgment to distinguish scientifically valid reasoning from outputs that are merely fluent or technically plausible.
Must-have
- Advanced degree, such as an MS, MD, PhD, or PharmD, in biology, genetics, physiology, computational biology, or a related biomedical field.
- At least 3 years of hands-on research experience in relevant domains, demonstrated through publications or industry experience.
- Proficiency in Python and/or R for biological data analysis and visualization, or expert-level experience in experimental or methods design.
- Deep scientific or clinical expertise in at least one relevant disease area, including rare genetic, autoimmune, infectious, or idiopathic diseases, neurodegeneration, or chronic progressive conditions such as COPD or MASH.
- Expertise in designing and troubleshooting relevant wet-lab or dry-lab experiments, or expertise in clinical diagnosis, treatment, and research.
Nice-to-have
- Strong existing opinions on where AI agents and large language models fail in science or clinical practice.
- Experience using agentic coding systems.
What's offered
- Fully remote work is available, with hybrid and onsite options in San Francisco.
- Onsite employees have access to an office, catered meals, and snacks.
- US employees receive health, dental, and vision insurance fully covered for the employee and 50% covered for dependents.
- US employees receive paid time off, starting at 12 days and increasing with promotion.
- US employees can participate in a 401(k) plan.
- The company supports defined individual-contributor and management growth paths, with progression based on output rather than tenure.
- The opportunity to shape scientific benchmarking strategy within an early team working with frontier AI labs and pharmaceutical organizations.
- A transparent, collaborative culture that values expert judgment, respectful debate, and solving problems together.
- Regular company outings, team off-sites, and an active community built around cycling, soccer, figure skating, boxing, running, martial arts, music, books, and game nights.
- Work authorization support is available for H-1B, O-1, and TN visas, as well as STEM OPT.
Hiring process
- Application review
- Take-home project completed through HackerRank
- Technical interview with a Member of Technical Staff
- Culture interview with the C-suite
- Offer and background check after acceptance