The Roles That Exist on a Real Production AI Team

The titles in 2026 have started to stabilize, even if the comp bands and responsibilities still vary by company. The distinction that matters most is between people who ship to production (AI engineers, ML engineers, data engineers, AI ops) and people who advance the state of the art (applied researchers, research engineers). Most companies need a lot of the first group and a small number of the second, in that order.

• AI Engineer: builds with LLMs, RAG, tool use, agents, evaluation pipelines, and observability in production code. Closest to a senior backend engineer who has gone deep on the LLM stack
• ML Engineer: trains, fine-tunes, evaluates, and serves custom models. Owns model lifecycle, feature pipelines, and inference infrastructure. Distinct from researcher because the deliverable is production performance, not papers
• Data Engineer (AI focus): ingestion pipelines, feature stores, embedding stores, retrieval infrastructure, data quality monitoring. Most AI teams underhire this role and pay for it in eval quality
• AI Platform Engineer / ML Platform Engineer: builds the internal tools the AI engineers use, including serving infrastructure, eval frameworks, observability, prompt registry, and model gateway
• AI Product Manager: defines AI features, owns the eval rubric, runs the user research, and decides what ships. Different skill set from a traditional PM because the work has to embrace probabilistic outputs
• AI Ops / AI SRE: monitoring, incident response, cost optimization, reliability of AI systems in production. Often a senior SRE who has gone deep on AI workloads
• Applied Researcher: prompt engineering, eval design, novel pattern exploration, model comparison studies. Valuable when the company is at the frontier; expensive overhead when the company is not
• Research Engineer / ML Researcher: pre-training, novel architecture work, foundation model development. Only relevant if the company is building its own foundation models, which is rare outside a small number of labs

Hiring Sequence by Stage

The right sequence depends on what product shape the company is building. For most companies (integrating off-the-shelf foundation models into production workflows), the first hire is an AI Engineer, not a researcher. The sequence below assumes that integration-first product shape, which describes 80%+ of AI teams in 2026.

• Stage 1 (zero to first feature, 1-2 engineers): one senior AI Engineer with shipped production LLM experience. Hire someone who has owned an eval pipeline, not just written prompts
• Stage 2 (3-5 engineers): add an AI Product Manager and a Data Engineer focused on retrieval infrastructure. The PM owns the eval rubric. The data engineer owns the retrieval layer
• Stage 3 (6-8 engineers): add a second AI Engineer and an AI Ops or AI SRE. By this stage, observability and cost monitoring become full-time work
• Stage 4 (8-12 engineers): split into product-facing AI engineers and a small platform group of 2-3 building shared eval, observability, and serving infrastructure
• Stage 5 (12-20 engineers): formalize the platform team, add an AI Platform Manager, embed product AI engineers in feature teams, hire an ML Engineer if custom models are now a real need
• Avoid hiring researchers first if there is no production AI yet; the researcher will be a paid spectator for 6-12 months and either leave or build a research project nobody wants
• Avoid hiring an "AI generalist" at scale; by stage 3 the roles should be specialized so the engineers can build deep expertise

When to Stand Up an ML Platform / AI Platform Team

Around 8-12 AI engineers, every team builds its own eval rig, its own observability, its own serving abstraction, and its own prompt registry. Productivity stalls because everyone is rebuilding the same infrastructure. The platform team is the structural fix: a small group (typically 2-4 engineers) that builds shared internal tools so product teams can move faster.

The platform team is a force multiplier when scoped narrowly and a tax when scoped broadly. The right charter is "make AI engineers 2x more productive on shared concerns" (eval, observability, serving, prompts), not "control every AI decision."

• Trigger: 8-12 AI engineers and visible duplication of eval, observability, or serving infrastructure across teams
• Charter: shared eval framework, shared observability and tracing, shared model gateway, shared prompt registry, shared retrieval infrastructure
• Size: start with 2-3 engineers and an experienced platform lead, scale to 4-6 around 15-20 AI engineers
• Anti-pattern: platform team owns no shared infrastructure and instead becomes a bottleneck for AI architecture decisions
• Anti-pattern: platform team builds elaborate internal frameworks that wrap thin LLM APIs and add complexity without value
• Good signal: product AI engineers actively want to use platform-team tools because they speed up shipping
• Bad signal: product AI engineers build shadow versions of platform tools because the official ones are too slow, too opinionated, or too brittle
• Reporting: platform team typically reports into the same VP or CAIO as the product AI engineers, not into a separate infrastructure org

Centralized vs Embedded vs Hybrid AI Team Models

Three org structures recur in 2026. Centralized: a single AI team owns all AI features across the company. Embedded: AI engineers sit inside product teams, with no central AI function. Hybrid: a small central platform group plus AI engineers embedded in product teams. The right model depends on company size, product shape, and how strategic AI is to the business.

• Centralized AI team: works at 1-8 engineers when the AI surface is narrow and the work is concentrated. Easier to share infrastructure and patterns. Breaks down when AI becomes cross-cutting across product surfaces
• Embedded AI engineers: works at 4-12 engineers when AI is integrated deeply into multiple product surfaces. Better product alignment. Slower infrastructure investment because nobody owns shared concerns
• Hybrid (most common at 12+ engineers): small central platform team owns shared infrastructure, AI engineers embedded in product teams own feature delivery. Captures the best of both with overhead cost of governance
• Reporting structure: a Chief AI Officer or VP of AI works when AI is strategic and cross-cutting. A Head of AI under the CTO works when AI is one of several engineering disciplines
• Sizing rule: do not invest in a central platform team until product duplication is visible and painful. Centralizing too early creates a bottleneck
• Promote internally where possible: senior AI engineers with 18-24 months at the company make better embedded leads than external hires unfamiliar with the codebase
• Beware shadow AI teams: when the official AI function is too slow or too opinionated, product teams hire their own AI engineers and rebuild infrastructure. Visible warning sign that the model is broken

How to Interview AI Engineers

Interview signals for AI engineers in 2026 are different from interview signals for backend engineers. The strong signals are evaluation thinking, failure-mode storytelling, and production engineering depth applied to non-deterministic systems. The weak signals are notebook fluency, prompt-engineering trivia, and the ability to recite framework names.

• Ask the candidate to walk through a real AI system they shipped: architecture diagram, eval rubric, failure modes encountered, what they changed and why
• Test for evaluation thinking: how did they measure success, how did they catch regressions, how did they distinguish a prompt change from a model change
• Probe failure stories: what broke in production, how they detected it, how they recovered, what they changed in the system to prevent recurrence
• Check for production engineering depth: how do they handle retries, idempotency, partial failures, timeouts, cost spikes, vendor outages
• Look for tool-use experience: have they shipped agents with real tool calls, not just chat completion. If yes, what went wrong and how did they fix it
• Skip the algorithm whiteboard: tree traversals do not predict AI engineering quality. A 90-minute pair-debugging session on a real LLM trace is far more predictive
• Test eval design directly: give them a small dataset and a prompt, ask them to design a rubric and an eval set. The structure of their answer reveals more than any resume claim
• Beware "AI thought leader" candidates: blog posts and conference talks are not production experience. Always ground the interview in shipped systems with measurable outcomes

Comp Bands and Hiring Difficulty in 2026

AI engineering comp has separated from general software comp over the last 24 months, particularly at the senior level. The hardest roles to fill are senior AI engineers with 3+ years of production LLM experience and ML platform engineers who have shipped internal eval frameworks at scale. Hiring timelines have stretched to 4-9 months for senior roles outside of major tech hubs.

• Senior AI Engineer (US): $250K-$450K base + bonus + equity in 2026, $500K-$900K total comp at top AI-native companies
• Senior ML Engineer (US): $280K-$500K base, with research-adjacent roles reaching $700K+ total at top labs
• AI Product Manager (US): $200K-$350K base + bonus, premium for candidates with shipped AI product experience
• AI Platform Engineer (US): $260K-$450K base, hardest role to fill because the talent pool is thin
• UK senior AI engineer: £120K-£220K base, total comp £180K-£350K, narrower than US but tightening
• EU senior AI engineer: €110K-€200K base, Berlin/Amsterdam/Paris cluster at top, with broader range across continent
• Time to hire: 4-9 months for senior roles, 2-4 months for mid-level, longer outside major hubs
• Build vs poach: training a senior backend engineer into an AI engineer takes 6-12 months and a serious internal investment; cheaper than external hiring at scale but slower

Common AI Team Scaling Mistakes

The same mistakes recur across companies scaling AI teams from 2 to 20. Most are fixable if caught in the first 6 months. The hardest to fix are early hiring mistakes that compound over years.

• Hired a researcher first when the work was integration: researcher spends 6 months looking for a problem, leaves frustrated, the team has no production AI to show for the headcount
• Hired three AI engineers before any AI shipped to production: the engineers spin without clear ownership, ship demos, and burn 12 months before someone notices
• No AI product manager: AI engineers own both the eval rubric and the feature definition, conflate the two, ship features without a credible quality bar
• No platform team at 12+ engineers: every product team rebuilds eval, observability, and serving, duplication is invisible to leadership but visible in velocity numbers
• Platform team too big too early: 5 engineers building elaborate internal frameworks for 3 product engineers, ratio is inverted
• Hired generalists at scale: by stage 3, lack of role specialization means nobody develops deep expertise and the team plateaus on quality
• Org placement is wrong: AI team reports into a non-technical executive (CMO, COO) who cannot evaluate technical tradeoffs or defend the team in budget cycles
• No promotion path for AI engineers: senior AI engineers cap out and leave because there is no visible ladder above them, and the company has to externally re-hire at higher comp

How Mahmoud Helps Engineering Leaders Scale AI Teams

My team-scaling work runs as either a fixed-scope assessment (4-8 weeks) or a longer fractional or advisor engagement (3-12 months). The assessment produces a written org plan: current state, target shape at 12 and 24 months, hiring sequence, comp bands, role descriptions, and the platform-vs-product split. The ongoing engagement adds hands-on hiring support: job specs, sourcing through my network, calibration interviews, and reference calls.

I do not place candidates and I do not collect placement fees. The work is independent of any recruiter relationship. That independence is what makes the hiring advice useful, because the recommendation can be "do not hire yet" without conflict.

• Phase 1: assessment of the current AI team, product shape, and 12-24 month roadmap
• Phase 2: target org structure (centralized, embedded, hybrid), hiring sequence, platform-vs-product split, comp bands
• Phase 3: role descriptions, sourcing channels, interview rubrics, calibration with existing leadership
• Optional ongoing: hiring loop participation, candidate calibration, reference calls, monthly review of pipeline
• No placement fees: independent of any recruiter relationship, recommendation can be "do not hire" without conflict
• Internal promotion pathing: the engagement includes a senior IC ladder for AI engineers and a manager ladder for AI platform leadership
• Knowledge transfer: documentation lives in the company's systems, so the program survives my exit

FAQ