Skip to content
View sunilmehta-si's full-sized avatar

Block or report sunilmehta-si

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
sunilmehta-si/README.md
Sunil Mehta. I build and scale the infrastructure AI platforms run on. 300+ TDX GPU nodes, 100M+ LLM requests a month, 99.95% uptime, 55% lower cloud cost.

sunilmehta.si  ·  Projects  ·  Experience  ·  Resume  ·  LinkedIn  ·  X  ·  Email

I'm Sunil Mehta, Founding Lead DevOps & MLOps Engineer at ORGN, based in Gurugram and open to remote roles. For 9+ years I have built and run the platforms AI products depend on, from multi-cloud Kubernetes and confidential GPU compute to secure delivery and cost governance.

In the open I build benchmarks and tools that answer questions a platform team has to answer, and I send the fixes I find upstream.

Measured in the open

Project What it asks What held
k8s-gpu-scheduler-benchmark
v0.5.0
Which Kubernetes GPU scheduler should a platform run? Out of the box only KAI bin-packs GPU pods by their GPUs. One setting made Volcano 26% faster and 26% cheaper per useful GPU-hour on the same trace.
k8s-inference-routing-benchmark
v0.6.0
How much does the router in front of LLM servers matter? Prefix-aware routing lifts cache hits from 64% to 80% and cuts median time to first token by 27%. Under load the router matters more and the routers differ less.
k8s-inference-autoscaling-benchmark
v0.8.0
What should an autoscaler follow when it scales LLM inference? On a real cluster no ramp from 30 seconds to 10 minutes was slow enough for the default HPA to keep requests on time. Pod startup is the constraint, a small warm pool is the cheapest fix, and a forecast helps on slow ramps only.
k8s-gpu-cost-scanner
v0.1.0
Which GPUs is the cluster paying for and not using? A read-only Go tool on the metrics a cluster already has. On a synthetic fleet backfilled into a real Prometheus in CI it finds every planted problem: 221.4 wasted GPU-hours in a day, each counted once, and the cost per million tokens of each model.
kube-diagnose
v0.5.0
Why is this Kubernetes workload broken? A read-only kubectl plugin that names the root cause with the evidence and the kubectl command behind each fact. On a real cluster in CI it named the cause of 87 of 87 broken namespaces in its first finding; k8sgpt named 33. record --redact makes a snapshot safe to share.
kubernetes-secure-mlops-pipeline
v0.8.0
Can every model in production be reproducible, signed and verified? A real kind cluster runs 24 checks on every push, and the gate blocked a 0.909 challenger against the 0.938 champion.
agentic-code-review-tool
v0.11.1
Scanner-grounded AI review for the whole pull request. 128 rules, each swept over real repositories before release. One rule went from 1,556 findings to 101.

Also: ai-devsecops-cicd, a secure delivery path for LLM and agent services · llm-inference-platform, LLM serving on Apple Silicon you can measure · jev-inference-router, an observable routing layer for local inference.

How the numbers are held

  1. One workload, every contender. The same trace is replayed through each scheduler, router or policy, so a difference belongs to the component and not to the traffic.
  2. Real components, in CI. Kueue, Volcano, KAI, the llm-d router, the HPA and KEDA run as released, on kind clusters in GitHub Actions. Anyone can run them again.
  3. Every number has a test. The datasets are committed, and tests hold each figure a README quotes to the data behind it.
  4. Only what repeats is claimed. A difference on a cluster counts when it holds over more than one run. One early reading reversed on the second run and was withdrawn in the open.

Shipped lately

When What
17 Nov 2026 kube-diagnose v0.5.0 13 new rules, from volumes a container may not use to pods on the node's DNS and CronJobs that skip runs: 87 of 87 faults named first on a real cluster; k8sgpt named 33.
12 Nov 2026 k8s-gpu-cost-scanner v0.1.0 A new tool: the GPUs an inference cluster pays for and does not use, and what each model costs per million tokens.
11 Nov 2026 k8s-inference-autoscaling-benchmark v0.8.0 A predictive policy: scaling one pod startup ahead helps on slow ramps, and warm capacity still wins the fast ones.
09 Nov 2026 kubernetes-secure-mlops-pipeline v0.8.0 Every pin is held back until its release is 14 days old, and reported when it drifts.
04 Nov 2026 agentic-code-review-tool v0.11.1 A false positive found by sweeping Inspect AI: 1,124 findings from one rule, down to 8.

The platform I run at ORGN

Ship Guard Run Watch & own
GitHub Actions, keyless through Workload Identity Federation Secret, code, dependency, image and infrastructure-code scanning 10+ GKE clusters for an engineering org of 50+ 99.95% uptime across 200+ services
Artifact Registry, images signed with cosign Kyverno and OPA at admission 300+ Intel TDX confidential GPU nodes Istio, Prometheus, Loki, OpenTelemetry, Grafana HA
Argo CD to every cluster Falco at runtime Rust LLM gateways, 100M+ requests a month $70K–$90K of monthly cloud spend owned
Protected Terraform workflows OAuth 2.1 / OIDC sign-on, migrated with zero downtime Infrastructure cost cut by 55%

This describes my professional work. Most of it lives in private organization repositories. The full experience, five roles since 2017, is on the site.

Upstream

Project Contribution Status
Argo CD Documented OCI repositories over plain HTTP (insecureOCIForceHttp, type: oci secrets) Merged
MLflow Model downloads no longer modify signed model directories, which broke signature verification. Reported, then fixed. Merged
NVIDIA KAI Scheduler Corrected the GPU fractioning docs to the names the subchart and operator use Merged
llm-d inference simulator Render requests no longer fail with EOF when vLLM closes an idle connection. Reported, then fixed. Merged
KServe Security: every Python lock moved off the certifi that still trusted the compromised emSign roots In review
Tesla vehicle-command Fixed key storage rejecting valid P-256 keys whose scalar starts with a zero byte, about one key in 256 In review
xAI Python SDK Shipped the missing py.typed marker, so type checkers use the SDK's annotations In review
X API CLI (xurl) The webhook server fails fast without an ngrok token instead of hanging In review
etcd-operator The operator no longer carries on without its TLS certificates when provisioning fails In review
KServe website Install docs: create the kserve namespace before applying the manifest In review
Kubeflow Pipelines A compiler bug: a multi-line default value makes the compiled pipeline YAML invalid. Fix ready. Reported

Tools

Infrastructure   Kubernetes · Terraform · Argo CD · Helm · GCP · AWS · DigitalOcean · Cloudflare
MLOps & serving   Kubeflow Pipelines · MLflow · KServe · vLLM
GPU scheduling   Kueue · Volcano · KAI Scheduler · kwok
Security   Intel TDX · OAuth 2.1 / OIDC · Kyverno · OPA · cosign · SLSA · OpenSSF model signing
Observability   Prometheus · Grafana · Loki · Tempo · OpenTelemetry · ClickHouse
Languages & data   Go · Rust · TypeScript · Python · PostgreSQL · Redis


Building AI platforms, developer tools or secure infrastructure?
sunilmehta.si · LinkedIn · X · Email

Pinned Loading

  1. agentic-code-review-tool agentic-code-review-tool Public

    Agentic, scanner-grounded code review for the whole PR: app code, infrastructure (Kubernetes, Docker, Terraform, GitHub Actions) and AI/LLM/agent code (OWASP LLM Top 10, MCP, agent skills).

    Python 1

  2. ai-devsecops-cicd ai-devsecops-cicd Public

    Reference DevSecOps pipeline for LLM and AI agent delivery: scan, sign, gate, deploy.

    Python

  3. k8s-gpu-scheduler-benchmark k8s-gpu-scheduler-benchmark Public

    Benchmark Kubernetes GPU schedulers (default, Kueue, Volcano, KAI) on identical AI workload traces: utilisation, fragmentation, queue wait, fairness and cost. No GPUs needed.

    Python 1

  4. k8s-inference-autoscaling-benchmark k8s-inference-autoscaling-benchmark Public

    Compare Kubernetes autoscaling signals (utilization, concurrency, queue depth, request rate) for LLM inference on identical bursty traces, with the HPA algorithm modelled exactly. No GPUs needed.

    Python

  5. k8s-inference-routing-benchmark k8s-inference-routing-benchmark Public

    Benchmark LLM request routing on Kubernetes (round-robin, load-aware, prefix-cache-aware with the llm-d Router) on identical chat and RAG traces, with simulated vLLM servers. No GPUs needed.

    Python

  6. kubernetes-secure-mlops-pipeline kubernetes-secure-mlops-pipeline Public

    MLOps pipeline on Kubernetes where every model is reproducible, evaluated against the champion, signed, verified at deploy, and rolled out with automatic rollback.

    Python