Client Roles
ABOUT THE ROLE
Join a small AI infrastructure team building systems that improve the performance and efficiency of large language model inference. You will work across software and hardware, with broad ownership from customer needs through production deployment.
WHAT YOU'LL DO
- Deliver support for new open-source models, tuning them for latency and throughput.
- Optimize inference serving, including batching, KV caching, speculative decoding, and quantization.
- Write and tune kernels using CUDA, HIP, and Triton across a range of accelerator platforms.
- Design, deploy, and operate clusters spanning multiple hardware vendors.
- Run production inference across heterogeneous fleets, improving reliability, observability, and cost efficiency.
- Work with customers to understand problems and develop practical systems solutions.
WHAT WE'RE LOOKING FOR
- Typically 1 to 6 years of relevant experience, including strong early-career candidates with substantial internship or project experience.
- Experience in backend, infrastructure, or systems engineering, with demonstrated work beyond application-level development.
- Strong understanding of computer architecture, memory hierarchy, and operating system internals.
- Familiarity with GPU programming or inference optimization, such as CUDA, Triton, quantization, batching, KV caching, or speculative decoding.
- Experience deploying or operating infrastructure at scale, such as clusters or large GPU fleets, is valuable.
- Strong computer science fundamentals, ownership of challenging technical work, and effe