AI
ABOUT THE ROLE
Join a small AI infrastructure team working to improve the performance and energy efficiency of large language model inference. You will build and operate systems that serve open-source models across a range of hardware, with ownership spanning low-level optimization through production deployment.
WHAT YOU'LL DO
- Deliver support for new open-source models, tuning them for latency and throughput.
- Optimize inference serving, including batching, KV cache management, speculative decoding, and quantization.
- Write and tune GPU kernels using CUDA, HIP, and Triton across heterogeneous hardware.
- Design, deploy, and operate clusters spanning multiple hardware vendors.
- Run production inference systems, improving reliability, observability, and cost per token.
- Work with customers to understand needs and solve infrastructure challenges.
WHAT WE'RE LOOKING FOR
- Typically 1 to 6 years of experience in backend, infrastructure, or systems engineering, including low-level work beyond application development.
- Strong understanding of computer architecture, memory hierarchy, and operating system internals.
- Experience with CUDA, Triton, or GPU programming; familiarity with HIP and heterogeneous accelerators is useful.
- Knowledge of ML inference optimization, such as quantization, batching, KV caching, or speculative decoding.
- Experience deploying or operating infrastructure at scale, such as clusters or large GPU fleets.
- Strong computer science fundamentals and evidence of ownership, rapid learning, or accomplishment on challenging