AI tools for engineers — five jobs, one stack, measured adoption

Every AI tool promises to make engineers faster. Very few are measured.

The 2026 landscape of AI tools for engineers is overwhelming: coding agents, model gateways, observability platforms, evaluation frameworks, self-hosted serving layers. Most teams adopt them in waves, license a dozen, and cannot answer the only question that matters — which ones actually earn their place?

The useful way to think about it is five jobs, not fifty tools. Every serious engineering stack needs something that does each job, and the specific product is secondary to whether the job is covered. This page maps the five jobs and the honest assessment of what covers them.

The failure mode is tool-churn: adopting per hype-cycle instead of per job, so the stack never stabilises and nothing is measured. The fix is the five-job frame and a two-week measurement before any team-wide adoption.

How the stack fits together

The five jobs connect in a specific order. The gateway is the spine: every application calls one API, so every request can be budgeted, logged and routed. The observability layer reads the gateway's logs. The evaluation harness gates what ships. The coding agent and the serving layer sit at the two ends — where work enters and where models run.

This shape is what makes adoption measurable. Because everything crosses the gateway, the token cost of each tool is attributable. Because evaluation gates releases, quality regressions surface in CI instead of in front of users. The stack is not a pile of tools; it is a pipeline with an audit trail.

The gateway as the measurement point

Every tool crosses the same API, so cost and usage are attributable per team.

# LiteLLM config — one API for the whole engineering stack
model_list:
  - model_name: coding-agent
    litellm_params:
      model: anthropic/claude-opus-5
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: chat-default
    litellm_params:
      model: vllm/llama-3.3-70b-instruct
      api_base: http://vllm-inference:8000/v1

router_settings:
  routing_strategy: usage-based-routing
  team_limits:
    platform: {tpm: 1000000}
    qa:       {tpm: 200000}

Three numbers that decide the stack

Each comes out of the two-week measurement. None of them can be read off a vendor demo.

• Time saved per task, measured — the difference between a tool that accelerates real work and one that accelerates demos.
• Token cost per completed task, attributed through the gateway — the number that makes the budget conversation honest.
• Error rate before and after, from the evaluation harness — the number that decides whether a tool stays or goes.

Adopt per job, measure per job, cut per job. The tools will change; the discipline is the stack.

Go deeper on the individual layers

• MLflow vs. Langfuse — Observability and evaluation — the two layers teams conflate.
• LiteLLM vs. OpenRouter — The gateway decision behind the whole stack.
• Claude Code cost per developer — Measuring the coding agent — the eight metrics that matter.