Enterprise AI Platform on Azure
From GenAI experiments to a governed AI platform in production
GenAI proofs of concept pile up, yet few reach production: security, cost, compliance and operations lag behind. I design enterprise AI platforms on Azure that let every team ship use cases and agents with confidence.
Who is it for?
Innovation leads, CIOs and platform teams moving from isolated use cases to a shared AI platform, or who must frame how teams use agents and tools (MCP) with bounded permissions.
What I deliver
- Reference architecture: Microsoft Foundry, models (Azure OpenAI and open models), AKS, APIM, private networking
- Agent gateway (agentgateway): Entra ID identity per token, caps, routing and model fallback
- Portable guardrails (NeMo Guardrails, Content Safety) in fail-closed mode, logging and traceability
- Enterprise MCP: internal tools exposed to agents with human approval and bounded permissions
- Microsoft Foundry: hosted agents, guardrails and regex blocklists, model catalog, fine-grained RBAC per project and agent, quotas
- “Agent Ready” team onboarding: self-service, quotas, no tickets
- AI FinOps: tokens, costs and tools measured per person, team and model
Method in 4 steps
- 1
Use cases and framing
Prioritization of use cases, data involved, security and compliance requirements, split of responsibilities between the platform and the teams.
Deliverable: Governance framework and prioritized use cases
- 2
Reference architecture
Choice of gateway (APIM, LiteLLM or agentgateway), keyless identity chain, models (Foundry or self-hosted) and guardrails. Validated with security, data and legal.
Deliverable: Reference architecture and ADRs
- 3
Industrialization
Platform on AKS as IaC, gateway with per-tool MCP authorization and token caps, guardrails, MCP servers, OpenTelemetry GenAI metrics and cost dashboards. Guardrail testing.
Deliverable: Production platform with cost tracking
- 4
Adoption
Onboarding of the first team with no ticket and bounded permissions (“Agent Ready”), agents published in a catalog, training and shared lessons learned.
Deliverable: Onboarding path and first published agents
Microsoft Foundry: the capabilities I put into production
I know Microsoft Foundry (formerly Azure AI Foundry) end to end, from agent hosting to access and quota governance.
Agent hosting
Foundry Agent Service runs your agents (Agent Framework, LangGraph, Semantic Kernel or custom code): container or source code, a dedicated Entra identity per agent, its own endpoint, a per-session isolated sandbox, immutable versions and traces in Application Insights.
Guardrails and custom regex
Input and output content filters, Prompt Shields, protected material, PII detection, and custom blocklists using exact match or regex (up to 10,000 terms per list), attached to deployments.
Managing available models
Multi-vendor catalog (Azure OpenAI, Anthropic, Mistral, Meta, Cohere…), choice of deployment type (standard, provisioned, batch, managed compute) and of the model scope open to each project.
Fine-grained RBAC
Dedicated roles: Foundry Agent Consumer to call an agent, Foundry User to build, Project Manager, Account Owner and Owner. Assigned at account, project or agent scope, following least privilege.
Quotas and usage tracking
Tokens-per-minute quotas per subscription, region and model, the portal Quota view, PTUs for reserved capacity, and per-project caps (TPM and total quota) through the AI Gateway.
Observability and control plane
OpenTelemetry tracing of agents, costs and usage per deployment, and centralized governance of the agent and model fleet in the Foundry control plane.
No API key, end to end
The identity chain of the platform I built at TF1: no secret to store, rotate or leak.
- 1
Entra ID
OIDC: the user authenticates
- 2
Agent gateway
the JWT is validated, access is authorized per tool
- 3
Workload Identity
the pod's identity, no secret
- 4
Microsoft Foundry
local authentication disabled
Stack: agentgateway · kagent · kmcp · agentregistry · MCP servers · KAITO and vLLM · OpenTelemetry GenAI · Prometheus · Grafana
Proof points
Enterprise AI in production
secure, governed Microsoft Foundry + AKS architecture delivered for TF1
5 articles & talks
“Agentic AI: to Azure and beyond” series (agentgateway, guardrails, MCP, FinOps)
Built-in governance
identity, caps, guardrails and cost handled at the architecture level
- Read the guide
Guide · 18 min read
Taking an AI agent platform to production on Azure with Microsoft Foundry and API Management
Guide: take AI agents to production on Azure with Microsoft Foundry and API Management as the AI gateway. Key-less identity, token quotas, MCP, guardrails.
- Read the guide
Guide · 16 min read
Governing enterprise AI agents: identity, guardrails, tools, quotas and evaluation
Guide: govern enterprise AI agents on Azure. Entra agent identity, per-tool MCP authorisation, guardrails, quotas, continuous evaluation, inventory.
- Read the guide
Guide · 15 min read
Generative AI FinOps on Azure: understanding, measuring and holding token costs
Guide: control generative AI costs on Azure. Token cost model, deployment types, PTUs and reservations, caching, per-team quotas, showback.
Cost, timeline and deliverables of a typical engagement
Three formats, from the shortest to the most complete. Durations are orders of magnitude observed on my engagements; the exact scope and quote are set after the first 30-minute call, within 48 hours.
5 to 10 days
AI platform scoping
Prioritised use cases, security and compliance requirements, Microsoft Foundry / model / gateway choices, token cost estimate, three-stage roadmap.
4 to 8 weeks
Reference architecture
Architecture document: key-less identity (Entra ID, Workload Identity), AI gateway (API Management or agent gateway), guardrails, agent and MCP tool catalogue, OpenTelemetry GenAI observability, token FinOps.
3 to 6 months
Production rollout and adoption
Platform deployed as infrastructure as code, first agents in production with their guardrails, cost and quality dashboards, platform team trained to operate and onboard new agents.
Fixed price for scoping and architecture, capped time-and-materials for the production rollout. The daily rate and conditions (remote, on site, duration) are stated in the proposal. No licence resale and no managed services: you own the code and the tenant.
- Knowledge transfer included at every step (pair reviews, workshops, documentation)
- Code and pipelines delivered in your repository, under your licence
- Weekly progress review with decision makers
Frequently asked questions
Is Microsoft Foundry enough as an enterprise platform?
It is a central building block, but an enterprise platform adds per-token identity, a gateway, portable guardrails, governed tool exposure (MCP), observability and FinOps. That whole is what I design.
What does your Microsoft Foundry expertise cover?
Agent hosting with Foundry Agent Service, guardrails (content filters, exact-match or regex blocklists), management of available models and their deployments, fine-grained RBAC (Foundry User, Agent Consumer, Project Manager, per project or per agent) and tokens-per-minute quota tracking.
Can we use models other than Azure OpenAI?
Yes. The architecture routes to several providers and open models hosted on AKS, with fallback based on cost, latency or compliance.
How do we control agents acting on our systems?
Through an enterprise MCP layer: tools exposed with bounded permissions, human approval on sensitive actions (e.g. tickets), full logging and per-team caps.
Discuss this with me
A first 30-minute call is free and without commitment.
Other services
Related articles
Hands-on Azure, AKS, FinOps and agentic AI write-ups published on blog.smartcloudarchitect.fr.
All articles on the blog· 9 min read
FinOps de l'IA agentique : mesurer jetons, coûts et outils par personne, équipe et modèle· 12 min read
Des MCP d'entreprise pour les agents : Azure MCP Server, Microsoft Learn et une couche tickets avec approbation humaine· 12 min read
« Agent Ready » : onboarder une équipe sur une plateforme agentique, sans ticket et avec des droits bornés· 11 min read
Garde-fous IA portables : NeMo Guardrails derrière agentgateway, en fail-closed