# Research Scientist at True Talent Solutions Partners

- Company: True Talent Solutions Partners
- Status: Open
- Workplace: On-site
- Location: San Francisco
- Level: Mid-level
- Discipline: AI & ML
- Employment: Full-time
- Posted: 2026-11-02
- Skills: AI, language models
- Apply: https://jobs.ashbyhq.com/true-talent-solutions-partners/f3528ec7-81b0-4058-9c9f-459f6220b87e

## Description

ABOUT THE ROLE

Develop benchmarks and evaluation methods for assessing advanced language models. This research and engineering role combines hands-on AI research with building practical evaluation systems, helping shape how models are tested before deployment.


WHAT YOU'LL DO

 - Evaluate newly released AI models across a range of capabilities and use cases.

 - Build benchmarks from the ground up, including dataset creation and coordination with labelers.

 - Develop research findings and contribute to publications on evaluation methods.

 - Improve automated methods for evaluating generated text.

 - Work with engineering teams to implement and scale evaluation approaches in production systems.

 - Collaborate with AI labs and enterprise customers to understand evaluation needs and define research initiatives.


WHAT WE'RE LOOKING FOR

 - At least 3 years of full-time AI or machine learning research experience, especially in benchmarking, model evaluation, NLP, or language models.

 - A track record of published research, ideally in NLP or benchmarking, alongside an interest in translating research into applied systems.

 - Strong Python skills and experience building clean, maintainable software, including HTTP services with Django, Flask, or similar frameworks.

 - Hands-on experience with PyTorch or TensorFlow, language modeling, and relevant AI/ML infrastructure and techniques.

 - Experience designing benchmarks and evaluation methodologies.

 - Strong computer science fundamentals and experience collaborating through Git workflows, code reviews, and cross-function

More True Talent Solutions Partners roles: https://deviantjobs.com/companies/true-talent-solutions-partners