Multi-Cloud Deployment Evidence¶

Production deployment of 3 ML services on Google Cloud Platform (GKE) and Amazon Web Services (EKS). All data below is from live verification on 2026-03-13 (v3.5.3) — both clouds via nginx Ingress LoadBalancer.

Architecture¶

graph TB
    subgraph GCP["GCP — us-central1"]
        direction TB
        GKE["GKE Cluster · 3 nodes · v1.34.3"]
        subgraph GCP_ML["ML Services — HPA"]
            BC1["BankChurn :8000"]
            NL1["NLPInsight :8000"]
            CT1["ChicagoTaxi :8000"]
        end
        subgraph GCP_OBS["Observability"]
            PR1["Prometheus :9090"]
            GR1["Grafana :3000"]
            ML1["MLflow :5000"]
        end
        GKE --> GCP_ML
        GKE --> GCP_OBS
        ING1["nginx Ingress · 136.111.152.72"] --> GKE
        GCP_INFRA["Artifact Registry · GCS · Workload Identity · Terraform"]
    end

    subgraph AWS["AWS — us-east-1"]
        direction TB
        EKS["EKS Cluster · 3 nodes · v1.31"]
        subgraph AWS_ML["ML Services — HPA"]
            BC2["BankChurn :8000"]
            NL2["NLPInsight :8000"]
            CT2["ChicagoTaxi :8000"]
        end
        subgraph AWS_OBS["Observability"]
            PR2["Prometheus :9090"]
            GR2["Grafana :3000"]
            ML2["MLflow :5000"]
        end
        EKS --> AWS_ML
        EKS --> AWS_OBS
        ING2["nginx Ingress · Classic ELB"] --> EKS
        AWS_INFRA["ECR · S3 · IRSA · Terraform + Kustomize"]
    end

• Active pods per cloud: 3 ML APIs + Prometheus + Grafana + MLflow (6 Running) + drift-detection CronJob history (Completed, 0 resources)
• GCP: 8 pods visible (6 Running + 2 Completed CronJob runs) · AWS: 7 pods visible (6 Running + 1 Completed CronJob run)
• Both clouds: nginx Ingress path routing (/bankchurn, /nlpinsight, /chicagotaxi, /grafana, /mlflow)
• GCP: static IP via GCE LB · AWS: Classic ELB DNS (2026-03-13)

Verified Capabilities¶

Test Results (v3.5.3 — Verified 2026-03-13)¶

Unit Test Coverage (395+ total tests, 0 failures)¶

Smoke & Integration Tests (Live GKE + EKS, 2026-03-13)¶

Multi-Cloud Load Test Comparison (2026-03-13 — via LoadBalancer, 10 users, 90s)¶

Both tests run against real LoadBalancer IPs — GCP: 136.111.152.72, AWS: Classic ELB DNS. Same locustfile, same user count, same duration. Results are directly comparable.

Load Test — GCP (GKE, 4× e2-medium, nginx Ingress static IP)¶

Load Test — AWS (EKS, 3× t3.small, Classic ELB)¶

Stress Test — AWS (25 users, 60s — peak load)¶

Multi-Cloud Performance Comparison¶

Conclusions¶

Key finding: ML inference latency (BankChurn, NLPInsight) is cloud-agnostic — p50 is identical on both clouds because it is dominated by model compute, not network. The ChicagoTaxi delta is due to batch data lookup patterns between S3 and GCS, not Kubernetes.

AWS t3.small vs GCP e2-medium: AWS uses 50% less RAM per node (2GB vs 4GB) but achieves the same inference SLAs because ML models are CPU-bound at inference time. This validates the cost-optimization decision documented in ADR-013.

Production readiness: 0% failure rate on both clouds under 25 concurrent users. Both meet the SLA target of p95 < 500ms for primary inference services (BankChurn, NLPInsight).

Infrastructure Tests¶

Run: bash tests/infra/kubernetes/test_kubernetes.sh all && bash tests/infra/terraform/test_terraform.sh all

Model Performance (v3.0.0 — Python 3.11, sklearn 1.8.0)¶

Docker Image Sizes (v3.5.0 — Artifact Registry, 2026-03-05)¶

Optimizations: multi-stage build, --no-compile, aggressive cleanup (__pycache__, tests/ excluding numpy, pip/setuptools), no .so stripping (corrupts numpy 2.x). NLPInsight dropped from 1.4 GB (FinBERT/torch) to 267 MB (TF-IDF+LogReg, no torch dependency).

In-Pod Latency (measured inside container, zero network overhead, 2026-03-05)¶

These are the real production latencies — measured by executing benchmarks directly inside each pod, eliminating port-forward proxy overhead (~50-100ms). This is equivalent to what a service mesh (Istio/Linkerd) or internal cluster client would observe.

Why BankChurn is slower¶

BankChurn uses a StackingClassifier ensemble: 4 base learners (RandomForest, GradientBoosting, XGBoost, LightGBM) feed into a LogisticRegression meta-learner. Each prediction runs 5 models sequentially. A P50 of ~103ms is expected and acceptable for this architecture — enterprise SLA target is P95 < 500ms.

Load Test Results (Locust, 30 users, 120s, via port-forward, 2026-03-05)¶

Port-forward adds ~50-100ms overhead per request and serializes connections under concurrency. In-pod metrics above are the authoritative production latency numbers.

SLA Compliance: Error rate 0.0% < 1% ✅ · Zero application errors under 30-user concurrent load ✅

HPA Auto-Scaling Configuration¶

Live Cluster State¶

GCP — GKE (verified 2026-03-05)¶

Pods¶

Cluster¶

Node Resource Utilization¶

AWS — EKS (verified 2026-03-12)¶

Pods¶

Note on pod count: kubectl get pods shows 7 pods on AWS (6 Running + 1 Completed CronJob pod) and 8 pods on GCP (6 Running + 2 Completed CronJob pods). The Completed pods are drift-detection CronJob execution history — they consume zero CPU/RAM and are automatically garbage-collected when successfulJobsHistoryLimit (default: 3) is exceeded. The active service stack is identical on both clouds: 6 Running pods.

Cluster¶

Note: AWS uses Classic ELB (provisioned 2026-03-13) via nginx-ingress LoadBalancer service. Same enterprise pattern as GCP: LoadBalancer + nginx Ingress path-based routing. ELB DNS: a6ed6b93fdbf14be2853d91bd2086d6b-1565798194.us-east-1.elb.amazonaws.com

Prometheus Monitoring (16/16 targets UP, 0 DOWN)¶

MLflow is intentionally NOT scraped (no /metrics endpoint). Health monitored via K8s liveness probes. Node-exporter removed (not deployed; unnecessary for portfolio-scale cluster).

Alert Rules (16 rules loaded, all healthy)¶

All rules use real metrics from deployed APIs (process_resident_memory_bytes, per-service *_requests_total). No rules reference non-existent metrics (kube-state-metrics, cAdvisor, model_drift_score).

Grafana (2 Dashboards, all panels functional)¶

Performance Optimizations Applied¶

Fixes Applied (v3.5.0)¶

• BankChurn: SHAP is lazy — skipped by default on /predict, available via ?explain=true (~196ms)
• NLPInsight: Switched from FinBERT (2+ GB torch) to TF-IDF+LogReg (267 MB image, 2.3ms inference)
• All services: Uvicorn workers = 2, multi-stage Docker builds, python:3.11-slim-bookworm base
• Docker numpy 2.x fix: Removed .so stripping (corrupts compiled extensions), excluded numpy from tests/ deletion
• Dependencies: All pinned with ~= (compatible release) — numpy~=2.2.0, scikit-learn~=1.8.0
• HPA: CPU-only scaling (removed memory metric — fixed model footprint)
• ChicagoTaxi: Added predictions init container for batch data download from GCS

Resource Optimization Assessment¶

• GCP Nodes: 3× e2-medium (2 vCPU / 4 GB) — avg 16% CPU, 58% memory utilization
• AWS Nodes: 3× t3.small (2 vCPU / 2 GB) — tighter memory budget, all pods running successfully
• Cost-effective: Smallest viable instance types per cloud; upgrading only needed if P95 latency SLAs are missed under sustained load
• HPA: All 3 services scale 1→3 replicas on CPU target (70-75%), verified functional on both clouds

Security¶

Visual Evidence¶

Multi-Cloud (HERO)¶

GKE vs EKS	SHAP on EKS

GCP Production¶

GKE Workloads	Grafana Dashboard	MLflow Experiments

AWS Production¶

EKS Cluster	EKS Pods	ECR Repos	S3 Buckets

CI/CD & Security¶

Pipeline Green	Deploy GCP	Deploy AWS

Codecov Dashboard	GitHub Secrets

Terminal Evidence¶

kubectl Pods (GKE)	kubectl Pods (EKS)	Resource Usage

Health Checks (GKE)	Health Checks (EKS)	Services & Ingress

Infrastructure as Code¶

Terraform Structure	K8s Overlays	Terraform Tests

API Evidence¶

BankChurn Swagger	NLPInsight Swagger	ChicagoTaxi Swagger

BankChurn Prediction	NLPInsight Prediction	ChicagoTaxi Prediction

SHAP Response	Prometheus Metrics

Monitoring¶

Grafana ML Panels	Load Test Results	P95 Latency

MLflow Comparison

GIFs & Video¶

Demo	File	Description
	01-demo-prediccion.gif	ML predictions: BankChurn (SHAP) + NLPInsight + ChicagoTaxi
	02-hpa-autoscaling.gif	HPA auto-scaling under load (1→3 replicas)
	03-fairness-audit.gif	Fairness audit CLI (disparate impact ratios)

Video: Portfolio Demo (3:30 min) — full multi-cloud walkthrough

Deployment Commands Reference¶

# === GCP (GKE) ===
gcloud container clusters get-credentials ml-portfolio-gke-production --region us-central1
kubectl get pods -n ml-portfolio
kubectl get hpa -n ml-portfolio
curl -s http://136.111.152.72/bankchurn/health | python3 -m json.tool

# === AWS (EKS) ===
export AWS_PROFILE=ml-portfolio
aws eks update-kubeconfig --name ml-portfolio-eks --region us-east-1
kubectl get pods -n ml-portfolio
kubectl get hpa -n ml-portfolio
kubectl get nodes -o wide
kubectl get svc,ingress -n ml-portfolio
ELB_DNS=$(kubectl get svc -n ingress-nginx ingress-nginx-controller -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
curl -s http://$ELB_DNS/bankchurn/health | python3 -m json.tool

# === Verify all services (either cloud) ===
for svc in bankchurn-predictor nlpinsight-analyzer chicagotaxi-pipeline; do
  echo "--- $svc ---"
  kubectl exec -n ml-portfolio deploy/$svc -- curl -sf http://localhost:8000/health
done

# === Run all tests ===
bash tests/infra/kubernetes/test_kubernetes.sh all
bash tests/infra/terraform/test_terraform.sh all
BANKCHURN_PORT=8000 NLPINSIGHT_PORT=8002 CHICAGOTAXI_PORT=8003 \
  python3 -m pytest tests/infra/smoke/test_smoke_services.py -v
python3 -m pytest tests/integration/test_smoke_k8s.py -v
python3 -m locust -f tests/load/locustfile.py --headless -u 10 -r 2 -t 120s --only-summary

# === Verify Classic ELB is provisioned ===
kubectl get svc -n ingress-nginx ingress-nginx-controller
# EXTERNAL-IP should show a6ed6b93...elb.amazonaws.com

Last Updated: 2026-03-14 (v3.5.3 — AWS Classic ELB LoadBalancer, load + stress tests verified on both clouds via real Ingress, multi-cloud parity confirmed: 0% failure rate, BankChurn p50 identical at 110ms)