Production Template¶

Production template and agentic operating model

A reusable ML service template with governed AI-assisted engineering¶

The ML Service Template is the strongest artifact in this portfolio. It packages the production lessons from the monorepo into a starter system for ML services: FastAPI serving, training/serving parity, CI/CD, Docker, Kubernetes, Terraform examples, observability hooks, runbooks and an explicit agentic governance model. The template also encodes 38 anti-patterns with corrective actions, SLSA L2 supply-chain security practices, closed-loop monitoring with statistical promotion gates, native-cloud edge protection (Cloud Armor / AWS WAF+Shield, Cloudflare optional), and a self-auditing documentation-coherence system that keeps its own version, ADR count and governance surface honest across every document — a gate, not a suggestion.

Open the template repo Read the quick start Review technical evidence

Serving baseline FastAPI scaffold Predict, batch predict, health, readiness, metrics and model operations.

Cloud posture GKE + EKS ready Kubernetes overlays and Terraform examples for both paths.

Failure modes 38 anti-patterns Corrective actions for serving, HPA, SHAP, IAM, CI/CD, scaffolding and adoption risks.

Supply chain SLSA L2 posture Security scanning, build hygiene and provenance-oriented release practices.

Agentic governance AUTO / CONSULT / STOP Rules for when agents can act, ask or halt for safety.

Monitoring loop Promotion gates Closed-loop monitoring ideas tied to statistical quality gates.

How To Read This Template¶

Recruiter view

Reusable system, not only a project¶

The important signal is that portfolio lessons became a repeatable starter system for future ML services, with defaults, guardrails and documentation.

Technical lead view

Inspect the operating contracts¶

The best evidence is in the rules, skills, workflows, manifest and anti-pattern catalog that constrain how services are generated and operated.

Team adoption view

Can another engineer use it?¶

The template is designed around quick start, service scaffold, tests, deployment artifacts and reviewable AI-assisted workflows.

Code Review Shortcuts¶

Service Dockerfile K8s deployment CI template Deploy GCP Agent manifest Agent rules Agent skills Agent workflows

Why This Is More Than A Template¶

The template is not only a folder structure. It is an operating contract for starting ML services with fewer avoidable mistakes. It defines what a generated service should contain, how it should be validated, which deployment paths it can follow and how AI agents are allowed to participate in the work.

The important point is the second-order artifact: I did not only build three portfolio services. I extracted the reusable production system behind them.

Technical reviewer signal

The most distinctive part is the agentic operating model: canonical rules, skills, workflows and risk escalation are written as code-adjacent governance, not left as informal prompting.

Production Scaffold Contract¶

1. Service API

FastAPI by default¶

The scaffold includes prediction endpoints, batch prediction, readiness, health, metrics, error envelopes, auth hooks and model metadata.

2. ML lifecycle

Training to serving parity¶

Feature engineering, schema validation, model loading and inference contracts are treated as one path instead of separate notebook and API worlds.

3. Quality gates

CI before adoption¶

YAML checks, workflow checks, scaffold tests, smoke paths, pre-commit hooks and targeted validations protect the generated service.

4. Runtime

Docker and Kubernetes¶

Containers, Kustomize overlays, readiness probes and deployment defaults are part of the service contract from the beginning.

5. Observability

Metrics and operations hooks¶

Prometheus-compatible metrics, prediction logging, tracing options, drift checks and retraining workflows are built into the template story. A six-station coverage audit (edge, infra, inference, models, logs/traces, business KPIs) closed the real gaps it found: executor-saturation metrics, a business-KPI dashboard, and a structured access log that actually correlates a request to its trace on every call, not only on failures.

6. Cloud path

GCP and AWS patterns¶

The template documents GKE/EKS deployment expectations, identity patterns, artifact registries and operational runbooks without pretending local tests are cloud proof. An opt-in edge layer (Cloud Armor / AWS WAF+Shield, Cloudflare optional) sits in front of the Ingress once an adopter wires it in.

Agentic Operating Model¶

Canonical rules

One source of truth¶

AGENTS.md, agent context and the manifest define the behavior contract. Adapter files point to canonical rules instead of drifting into parallel policy.

Risk matrix

AUTO / CONSULT / STOP¶

Low-risk work can be automated, ambiguous production work requires user consultation, and destructive or safety-sensitive actions must stop.

Workflow skills

Repeatable MLOps actions¶

New service creation, drift checks, retraining, rollback, cost review, release, incident and security workflows are encoded as reusable agent skills.

Reviewability

Agents leave evidence¶

The model is designed around validation logs, changelogs, ADRs, runbooks and explicit test commands so agent-assisted work remains auditable.

Rules, Skills And Workflows¶

Rules (18)

Context-aware engineering constraints ¶

Rules cover Python serving, training, Kubernetes, Terraform, Docker, GitHub Actions, monitoring, data validation, security, API contracts, Copier template lifecycle, documentation coherence and edge protection (Cloud Armor / AWS WAF+Shield / optional Cloudflare). The goal is to make failure modes harder to reintroduce.

Skills (26)

Reusable MLOps procedures ¶

Skills include new service creation, EDA, deploy to GKE/EKS, drift checks, model retraining, release checklist, rollback, cost audit, security audit, incident response, stack-profile switching, adopter onboarding, documentation-coherence enforcement, CI-green verification, dual-axis PR review, systematic bug diagnosis, pre-scaffold ML problem spec capture, blameless incident postmortems and edge-protection coverage auditing.

Workflows (18)

Slash-command operating paths ¶

Workflows such as /new-service, /incident, /release, /drift-check, /retrain, /rollback, /secret-breach, /stack-switch, /onboard, /doc-coherence, /ci-green and /edge-setup turn repeatable MLOps work into auditable steps.

Anti-Pattern Catalog¶

Serving

No `uvicorn --workers N` under Kubernetes¶

Corrective action: one worker per pod, HPA for horizontal scaling, and ThreadPoolExecutor for CPU-bound inference.

Autoscaling

No memory-based HPA for ML pods¶

Corrective action: use CPU as the scaling signal because loaded models keep a fixed memory footprint even when traffic drops.

Async APIs

No direct `model.predict()` in async endpoints¶

Corrective action: move CPU-bound prediction work behind asyncio.run_in_executor() so request handling stays responsive.

Explainability

No TreeExplainer for StackingClassifier¶

Corrective action: use KernelExplainer with a predict-proba wrapper in the original feature space.

Packaging

No model artifacts baked into Docker images¶

Corrective action: keep images immutable and load model artifacts through runtime storage patterns such as init containers.

Security

No static cloud credentials in production paths¶

Corrective action: use Workload Identity on GCP and IRSA on AWS, with CI/deploy/runtime identities separated by purpose.

Adoption safety

No cloud credentials in a "local" stack profile¶

Corrective action: a local profile must structurally refuse cloud credentials, Kubernetes and Docker — enforced by a contract test and a runtime guard in make deploy, not just a naming convention.

Release safety

No promoting or deploying on red/missing CI¶

Corrective action: a read-only skill verifies CI status before release or a staging/prod deploy; overriding a red or missing signal requires explicit human approval and an audit-trail entry (D-36).

Edge exposure

No public Ingress without edge protection¶

Corrective action: a production overlay must wire in Cloud Armor or AWS WAF+Shield (Cloudflare optional) before going live; disabling an existing WAF/rate-limit rule is a STOP-class action in every environment, no exceptions (D-38).

Adoption Engineering & Self-Auditing Documentation¶

Governance only counts if adopting the template is actually easy. A later pass (v0.19.0–v0.20.0) rebuilt the scaffolding path on Copier (so copier update can pull template improvements into an already-adopted service), added local-first stack profiles so a reviewer can evaluate the whole train → serve → drift loop without provisioning a cluster, and mapped the production layout to the Cookiecutter Data Science vocabulary for practitioners coming from a notebook-first background.

The differentiated piece is the last one: a documentation coherence system that treats "the docs agree with reality" as a CI-enforced contract, not a hope.

Technical reviewer signal

An independent audit of this exact work found and fixed a real bug before release: an onboarding flow was validating its output against the wrong JSON schema and would have failed on first use. The fix, and four other real defects, are documented in the release notes instead of quietly folded in — that disclosure habit is the same one the template asks of its adopters.

Scaffolding

Copier, not `cp` + `sed`¶

A custom Jinja delimiter ({@ @}) avoids collisions with the literal ${{ }} GitHub Actions syntax the template ships. Two anti-patterns (D-33, D-34) keep the scaffolder from regressing to manual substitution.

Local-first profiles

`local` / `staging` / `prod`¶

Chosen at scaffold time; local runs the full loop with zero cloud dependencies. Switching profiles is a reviewable, CONSULT-mode operation, never a silent edit.

Recognizable layout

A generated CCDS mapping¶

A documentation-only view translates the production directory layout into Cookiecutter Data Science vocabulary — no directories renamed, no production path touched.

Self-auditing docs

A CI gate for the docs themselves ¶

One deterministic script enforces a single source of truth for release version, anti-pattern count, agentic-surface counts and ADR numbering across every document — a gate, not a suggestion.

Enterprise Governance & Compliance Posture¶

A later benchmarking pass compared the template against NIST AI RMF, ISO/IEC 42001, the EU AI Act and frontier open-source scaffolds — not against the author's other repos. The finding: the template already produces the evidence these frameworks ask for (quality gates, fairness thresholds, an audit trail, human-in-the-loop approval). What it lacked was the map connecting that evidence to each framework's own vocabulary.

The discipline that matters most

The compliance document is explicit about what it is not: not a certification, not a substitute for legal review, not a claim that any template can be "AI Act compliant" — only a deployed, operated system can be evaluated. Declining to over-claim is itself the signal.

Compliance mapping

NIST AI RMF · ISO/IEC 42001 · EU AI Act ¶

Traces artifacts the template already produces — quality gates, the fairness DIR floor, the audit trail, AUTO/CONSULT/STOP human oversight — to each framework's own control questions. Descriptive, never certifying.

Supply chain

SHA-pinned CI + OpenSSF Scorecard¶

Every GitHub Action across every workflow is pinned by commit SHA, not a mutable tag. A dedicated Scorecard workflow scores the repo against OpenSSF's supply-chain criteria on every push.

Agentic governance pattern

Read is AUTO, override is STOP¶

The newest gate (D-36) separates "check whether CI is green" (always allowed, read-only) from "proceed despite red or missing CI" (an explicit human STOP-class approval, logged to the audit trail) — wired as a hard precondition into release and staging/prod deploy.

Portability

Escape hatches, not lock-in ¶

A documented swap matrix for cloud, experiment tracking, serving backend, IaC engine and scaffolding tool — so "agnostic to technologies" is a verifiable claim, not a slogan.

Edge protection

Native-cloud-first, Cloudflare optional ¶

Cloud Armor and AWS WAF+Shield Standard are the default per-cloud WAF and DDoS layer; Cloudflare stays available for genuinely concurrent multi-cloud deployments, but is never the default — the common case is one cloud, not a third-party account layered on top of it.

How It Compares¶

None of the well-known alternatives occupy quite the same spot. Each is genuinely strong at what it optimizes for — the comparison below is about fit, not a claim that this template is universally "better."

The honest version

This is a hardening baseline for a small-to-mid ML team that wants production defaults without adopting a control-plane. It is not a managed pipeline product, not a full MLOps platform, and not a drop-in replacement for an already-mature internal platform team.

Alternative	Strong at	What this template adds
Cookiecutter Data Science	The most recognized project-layout convention; huge familiarity	CCDS gives you a folder shape. It has no serving contract, no CI/CD, no Kubernetes/Terraform, no supply-chain security and no agentic governance — this template maps to CCDS vocabulary for practitioners coming from it (see Adoption Engineering above) rather than competing with it.
Kubeflow	Full ML platform: pipelines, serving, multi-tenancy, a real control plane	Kubeflow needs a dedicated operator and real operational investment before the first service ships. This template is a starter a small team can scaffold from cold in minutes, with the option to grow into a platform like Kubeflow later — not a prerequisite for one.
MLRun / ZenML / Metaflow	Strong pipeline/DAG orchestration abstractions, good experiment ergonomics	These orchestrate the ML workflow; none of them ship Kubernetes manifests, Terraform, signed-image CI/CD, or an agentic governance layer as part of the same starter — you still assemble the production surface yourself.
Cloud-native pipelines (Vertex AI Pipelines, SageMaker Pipelines)	Managed, low operational burden, deep integration with one cloud	Powerful, but cloud-locked — you build on the platform, not with portable source you own. This template ships working Terraform + Kustomize for both GCP and AWS, so the same service definition targets either.
Bespoke internal platform-engineering scaffolds	Tailored exactly to one company's stack; common in mature orgs	Usually unpublished, un-audited by outsiders, and rarely include a first-class agentic governance layer — that concern barely existed when most internal scaffolds were built. This template treats it as a testable, CI-gated contract from the start.

The throughline across every row: this template is the only common option that ships a complete infra-to-serving stack and a governed, contract-tested agentic development layer as one coherent, adoptable unit — not a platform to operate, and not a governance story left to prompting.

Multi-IDE Governance¶

Canonical source

A vendor-neutral body store¶

Rules, skills and workflows live once in agentic/ — not named after any single IDE, so a tool rebrand (this happened once already) can never strand the source of truth again.

Portable adapters

Devin, Cursor, Claude Code and Codex¶

Devin ingests full bodies, so .devin/ is a generated byte-for-byte mirror. Cursor, Claude Code and Codex read pointer files. Both kinds are regenerated by one script and never hand-edited.

Manifest

Cross-surface index ¶

The manifest maps rules, skills and workflows across IDE surfaces so the governance model can be validated instead of trusted by memory.

Behavior protocol

AUTO / CONSULT / STOP everywhere¶

Risk class is independent of the assistant being used. Low-risk work can run, ambiguous work asks, and destructive or production-sensitive work stops.

What A Reviewer Should Inspect¶

First scaffold

Can it generate a usable service?¶

A reviewer should be able to scaffold a service, run the local checks and see a coherent FastAPI project with tests and deployment artifacts.

Serving contract

Does inference match training?¶

The service should preserve feature parity, validate inputs and expose readiness based on model loading rather than a superficial health endpoint.

Governance contract

Can agents act safely?¶

The strongest technical signal is whether agent workflows are bounded by rules, manifests, validations and explicit escalation points.

Cloud realism

Are claims scoped honestly?¶

The template distinguishes local validation from real GKE/EKS validation, which keeps the documentation useful without overstating production proof.

What It Shows About Me¶

Signal	What it means
Product thinking	I turned portfolio lessons into a reusable starter system.
MLOps discipline	The template treats serving, testing, packaging and deployment as one system.
Governance mindset	AI-assisted engineering is bounded by explicit rules, not only prompts.
Operational honesty	Cloud validation, cost control and limitations are documented instead of hidden.
Documentation taste	The repo is designed so another engineer can adopt, review and improve it.

Where To Go Next¶

Repository