White Circle
We're looking for an ML Infrastructure Enginee r to join White Circle , an AI Safety company building the policy enforcement and optimization layer for AI systems. Backed by $11M from senior leaders at OpenAI, Anthropic, HuggingFace, Mistral, and DeepMind, White Circle processes 100M+ API calls monthly and runs its own LLMs in production.
You will
- Build scalable RL and post-training pipelines, including smoke tuning runs for quality testing and ablations.
- Design data control systems for rollouts, replay, filtering, evaluation, and policy updates.
- Tune training and inference end-to-end for throughput: networking, memory, scheduling, data loading, storage, checkpointing, I/O.
- Build infrastructure for model iteration (experiment runs, artifacts, evals, dashboards, reproducibility, cost visibility) and inference infrastructure for post-training and eval loops.
- Build agentic development environments: coding-agent harnesses, tool integrations, runtime sandboxes, multi-agent orchestration.
Requirements
- Hands-on experience designing and running distributed RL/post-training systems at scale (rollouts, replay buffers, reward signals, policy updates, eval loops).
- Strong Python (concurrency, async, multiprocessing, performance optimization) and PyTorch or JAX.
- Debugging distributed GPU workloads across CUDA, drivers, containers, NCCL, networking, storage, and checkpointing.
- Profiling across the stack (py-spy, PyTorch profiler, Nsight, perf, tracing).
- Inference stacks: vLLM, SGLang, TensorRT-LLM, Dynamo, or custom serving.
- Ability to connect system metrics to m