Applied AI Research Engineer: Benchmarking & Performance Economics

### Description

This is an applied research role, not a machine learning science role. You will not be advancing the state of the art in model architecture. You will be producing decision-grade numbers: benchmarks, cost models, and calculators that determine what we build, what we tell customers, and what we are willing to claim in public.

The work sits at the intersection of three disciplines that rarely overlap in one person — systems engineering, inference-stack depth, and honest experimental design. If you have ever read a vendor benchmark and immediately knew which confound made it meaningless, this is the seat for you.### What you'll own

• The token economics of agentic execution. What it actually costs to run an autonomous agent to task completion, and how that compares to LangGraph, CrewAI, AutoGen, the OpenAI Agents SDK, Ray, and whatever exists six months from now. Tokens per call is the wrong unit and you know it, the unit is cost per completed task at a fixed success rate, and getting there means measuring steps, retries, tail latency, success rate, and run-to-run variance.
• Routing and model-selection economics. Where each routing strategy sits on the cost/quality frontier, what the router costs to run, and how a 5% misroute rate compounds into a 40% task failure rate across a thirty-step trajectory. Routing quality gets evaluated at the trajectory level here, not the request level.
• Small language model economics. The crossover analysis: at what volume, task narrowness, and quality tolerance does a fine-tuned 3B beat a frontier API call? You own the full stack: data curation, training, evaluation, serving, and the ongoing cost of drift and retraining, and the judgment to say "don't train this" when the numbers say so.
• GPU and serving-stack modeling. A parameterized calculator that takes model size, quantization, batch size, context distribution, KV cache footprint, and concurrency, and returns th