AutoScale OS: Intelligent Cloud Orchestration Platform System Design

Back to Portfolio

Scaling & Messaging Systems

Java, Kubernetes, Redis, AWS, and Terraform orchestration platform with explainable scale-to-zero, worker placement, health checks, and metrics export.

0 <-> N Replica Control | Cooldown + Readiness Guardrails

Live DemoRead Scale-to-Zero Design

Why This Project Matters

Shows platform-engineering depth beyond CRUD: workload eligibility, idle detection, safe 0-to-N recovery, readiness checks, and explainable scaling decisions.

Tech + Architecture Summary

  • Tech: Java, Kubernetes, Redis, AWS, Terraform, Prometheus, Scale to Zero
  • Architecture: autoscale-os.dev -> React operations UI -> Java orchestration API -> Redis state -> idle/backlog control loop -> 0-to-N Kubernetes worker scheduler -> Prometheus metrics + readiness.

Impact Metrics

  • Added a scale-to-zero path for eligible stateless worker pools after a configurable idle window while preserving control-plane and queue state.
  • Designed scale-from-zero recovery so new backlog raises desired capacity from 0 to N and readiness gates protect work until workers are healthy.
  • Applied cooldown, hysteresis, and per-workload minimum-replica rules so stateful or latency-sensitive services do not scale to zero accidentally.
  • Modeled deployment orchestration so deployment specs become placement plans, health summaries, and auditable platform events.

Core Problem

Reduce idle capacity to zero without losing queued work, scaling the wrong workload, or routing traffic before replacement workers are ready.

Build Notes

What I Owned

I built AutoScale OS to show how deployment policy, idle time, backlog, readiness, and capacity targets produce explainable scale-to-zero and scale-from-zero decisions.

Hard Lesson

The main lesson is that autoscaling is a safety problem, not just a formula. Downscaling and cap decisions need guardrails because infrastructure can fail faster than dashboards update.

Next Enhancement

Next I would add a replayable scaling scenario that shows the control loop before, during, and after a burst so reviewers can inspect the decision timeline.

High-Level Architecture

mermaid
graph LR
  UI[AutoScaleOS UI]-->API[Java Orchestration API]
  API-->Redis[(Redis State Store)]
  API-->Control[Autoscaling Control Loop]
  Control-->Idle[Idle Window + Backlog Trigger]
  Idle-->Scheduler[Worker Scheduler]
  Scheduler-->Workers[Kubernetes Workers: 0 to N]
  API-->Metrics[Prometheus /metrics]
  API-->Readiness[/readiness Health Surface]

Production-Grade Capabilities

  • Autoscaling control loop based on CPU, memory, backlog, current replicas, healthy workers, and desired capacity.
  • Opt-in scale-to-zero policy with idle windows, cooldown/hysteresis, queue wake-up, and readiness-gated recovery.
  • Worker scheduling model with CPU/memory headroom and availability-zone-aware placement language.
  • Prometheus-style `/metrics` and `/readiness` surfaces for operational validation.

Engineering Decisions

  • Aggressive scale-up absorbs backlog faster, but bounded recommendations reduce the risk of runaway capacity changes.
  • Scale-to-zero reduces idle allocation for eligible workers, but introduces cold-start latency for the first request after inactivity.
  • A zero minimum is unsafe for every workload, so stateful and latency-sensitive services retain an explicit replica floor.
  • Centralizing specs, metrics, plans, and events in Redis simplifies the demo control plane, but production systems would need stricter persistence and retention policies.
  • Combining UI and API in one deployable artifact improves reviewer accessibility, while keeping clear API boundaries preserves platform-engineering credibility.

Behavioral + Impact Signals

  • Framed autoscaling decisions around safety, health, and explainability.
  • Separated placement, readiness, event history, and metrics export into clear operational surfaces.
  • Chose Java/Kubernetes/Redis/AWS/Terraform stack elements that map directly to platform engineering roles.

Quality Guarantees

  • Scaling recommendations account for worker health before increasing desired capacity.
  • Only workloads explicitly marked scale-to-zero eligible may reach zero replicas.
  • Queue state survives zero worker capacity and triggers a guarded scale-from-zero transition.
  • Deployment events remain auditable from spec intake through placement and health summary.
  • Readiness and metrics surfaces expose operational state instead of relying only on UI screenshots.

Recent Upgrades

  • Added recruiter-readable scale-to-zero and scale-from-zero states with explicit workload eligibility and cold-start tradeoffs.
  • Added guardrails for idle windows, cooldown, queue-backed wake-up, readiness, and workload-specific replica floors.
  • Added AutoScale OS to the portfolio with live custom-domain access and a dedicated system-design page.
  • Positioned AutoScale OS as the Java/Kubernetes/Redis platform orchestration project for recruiters reviewing backend infrastructure fit.
  • Documented autoscaling, worker scheduling, deployment events, metrics, and readiness as the main evidence path.

Outcome Highlights

  • Published a live AutoScale OS deployment at autoscale-os.dev.
  • Built recruiter-readable panels for autoscaling control loop, deployment orchestration, worker scheduling, and cloud stack decisions.
  • Exposed `/metrics` and `/readiness` concepts as first-class operational surfaces.