T
True Talent Solutions Partners

Member of Technical Staff

San Francisco·$200K–300K

inference systems · language models

ABOUT THE ROLE

Join a small AI infrastructure team building efficient inference systems for open-source language models. You will optimize models and serving infrastructure across diverse hardware, helping improve performance, reliability, and cost per token.

WHAT YOU'LL DO

  • Ship support for new open-source models, tuning them for latency and throughput.
  • Optimize inference serving with techniques such as batching, KV caching, speculative decoding, and quantization.
  • Write and tune kernels using CUDA, HIP, and Triton across a range of accelerator platforms.
  • Design, deploy, and operate heterogeneous clusters across infrastructure vendors.
  • Run production inference across a mixed hardware fleet, improving reliability, observability, and cost efficiency.
  • Work with customers to understand technical needs and solve infrastructure problems.

WHAT WE'RE LOOKING FOR

  • Typically 1 to 6 years of experience in backend, infrastructure, systems, or low-level engineering, including experience at a technology company or early-stage startup.
  • Strong computer science fundamentals, including computer architecture, memory hierarchy, and operating system internals.
  • Experience owning a challenging system end to end, notable technical achievements, or evidence of rapid growth.
  • Systems engineering experience beyond application-level or full-stack development, with a non-research-focused background.
  • Familiarity with CUDA, Triton, or GPU programming; experience with HIP, TPU, or AWS Trainium is also relevant.
  • Knowledge of inference optimization technique
Read the rest on jobs.ashbyhq.com