# Software Engineer, RL Environments at True Talent Solutions Partners

- Company: True Talent Solutions Partners
- Status: Open
- Workplace: On-site
- Location: San Francisco
- Level: Mid-level
- Discipline: AI & ML
- Employment: Full-time
- Posted: 2026-11-03
- Skills: Python, TensorFlow, Kubernetes, Docker, Java
- Apply: https://jobs.ashbyhq.com/true-talent-solutions-partners/db588de3-2532-42a9-bc6f-21dbf56f0f65

## Description

ABOUT THE ROLE

Build datasets, evaluation rubrics, and reward signals that help improve large language models. You will work with AI research teams to test data-collection approaches, investigate model failure modes, and develop measurements that inform training runs.


WHAT YOU'LL DO

 - Design data slices that reveal meaningful model failure modes across areas such as finance, software, and enterprise workflows.

 - Build and refine evaluation rubrics and reward signals for RLHF and RLVR pipelines.

 - Model annotator behavior and run experiments to improve model capabilities.

 - Develop quantitative methods to measure dataset quality, diversity, and downstream impact.

 - Create and manage real-world and synthetic data pipelines.

 - Work with research teams to turn training objectives into concrete data and evaluation specifications.


WHAT WE'RE LOOKING FOR

 - One to four years of hands-on experience designing machine learning data-collection strategies and evaluation rubrics.

 - Familiarity with large language model training and evaluation methods.

 - Experience creating data slices to reveal model failure modes across domains such as finance, code, or enterprise workflows.

 - Ability to design lightweight experiments, work through ambiguous results, and draw actionable insights.

 - A practical, builder-oriented approach and interest in how data quality and structure shape model behavior.


COMPENSATION & BENEFITS

Base salary is $260,000 to $290,000 annually. Visa sponsorship is available.


LOCATION

On-site in San Francisco, California.

More True Talent Solutions Partners roles: https://deviantjobs.com/companies/true-talent-solutions-partners