All projects

Platform engineering

Nexus Platform

A production-style AWS platform built entirely as code: three environments, GitOps delivery, signed images, full observability, and admin access without an open port. Any environment can be destroyed and rebuilt from Git.

3repositories: infrastructure, GitOps, applications
37applications reconciled by Argo CD
26architecture decision records
~1 hto rebuild DEV from an empty cluster
0secrets in Git or Terraform state

Architecture

Three repositories, three layers

Nexus architecturenexus-platformTerraform · TerragruntAWS foundationVPC · EKS · RDS · Valkey · SQS · CloudFront · WAFbuildsnexus-gitopsHelm charts · Argo CDKubernetes platformIstio · Karpenter · Kyverno · Prometheus · Loki · Tempoappliesnexus-applicationsGo API · Java worker · CINexus applicationCanary releases of signed images onlyshipsVisitorsCloudFront → private NLB → IstioAdministratorsCloudflare Access → tunnel

What it is

Nexus is a small application, a Go API and a Java worker connected by an SQS queue, running on a platform that is built the way a production team would build it. The application is deliberately simple; the platform around it is the project.

How it fits together

  • Terraform and Terragrunt build the network, EKS, PostgreSQL, Valkey, SQS, and CloudFront from one set of units shared by every environment. Personal values live in a single ignored file, so anyone can deploy the project into their own account.
  • Argo CD receives the cluster facts from Terraform and installs 37 applications in four sync waves, from the service mesh to the application itself.
  • GitHub Actions tests, scans, and signs every image with cosign. Kyverno admits only images signed by that pipeline, and Argo Rollouts releases each version as a canary that Prometheus measures.
  • Visitors reach the site through CloudFront and WAF, which connect privately to an internal load balancer. Administrators reach Argo CD, Grafana, and the other tools through Cloudflare Access and a tunnel that only dials out.
  • Prometheus, Loki, Tempo, and Pyroscope collect metrics, logs, traces, and profiles; Sloth turns the service-level objective into burn-rate alerts.

How I work on it

Every significant choice is written down as an architecture decision record, including the alternatives I rejected. Problems found along the way are fixed in code, documented as incidents, and, where possible, covered by an alert so they cannot return silently.

Proof

See it running

Captured on 9 November 2026 from a DEV environment rebuilt from Git for the purpose and destroyed the same evening. Account IDs and personal addresses are blurred. Select a screenshot to open it at full size.

Delivery

Argo CD showing all 37 Nexus applications as Synced and Healthy.
37 applications, synced and healthyArgo CD after the rebuild: the whole platform, from the service mesh to the application, reconciled from Git.
GitHub Actions run with every job green, No leaks detected, and the API image signed with an SPDX SBOM.
Test, scan, sign, promoteEach push is tested, checked for leaked secrets and vulnerabilities, signed with cosign, given an SBOM, and promoted to DEV.
Argo Rollouts dashboard with all four canary steps complete and revision 2 marked stable.
A canary release, promotedThe new API version took 20% and then 50% of the traffic while Prometheus measured its success rate. All five checks passed, and it became the stable version.

Security

Trivy scan in GitHub Actions failing on CVE-2026-78661 in golang.org/x/net.
A vulnerable image stoppedA high-severity flaw in golang.org/x/net, published after the previous build, failed the pipeline. The update shipped in the next commit.
Policy Reporter showing 7 passing results and 0 failures in the nexus namespace.
Every policy passingKyverno checks that each running pod uses an image signed by CI and runs without root or extra privileges.
Cloudflare Access login page in front of the Nexus DEV Argo CD.
No public admin endpointArgo CD, Grafana, and the other tools sit behind Cloudflare Access and a tunnel that only dials out.

Observability

Grafana Nexus Overview with request rate, CPU, memory, a 0% server error ratio, and p95 latency during a load test.
Fifteen minutes of loadA k6 test held about 4.7 requests a second with no server errors and 95th-percentile latency under 10 ms.
Tempo trace of a POST request with 16 spans across istio-ingress, nexus-api, SQS, and nexus-worker.
One request across four servicesTempo follows a POST from the Istio gateway through the Go API and SQS to the Java worker that processed it.
Grafana Explore showing structured JSON logs from the Nexus worker in Loki.
Structured logsJSON logs collected by OpenTelemetry into Loki, searchable by any field.
Pyroscope CPU flame graph of the Nexus API.
Continuous profilingPyroscope shows where the API spends CPU under load, down to the function.
Grafana dashboard with RDS, ElastiCache, and SQS CloudWatch metrics.
AWS services beside the clusterPostgreSQL, Valkey, and SQS metrics from CloudWatch on the same dashboards as the cluster's own.
Email titled [FIRING] NexusAlertingTest from the nexus-dev-use1 alerts topic.
An alert that reached a personAlertmanager sends through SNS. This test alert proved the path end to end after the fix described in the incidents.

Cost

OpenCost cost allocation by namespace for the DEV cluster.
Cost per namespaceOpenCost divides the cluster's cost by namespace, so the most expensive component is obvious.
Goldilocks recommendations for the nexus-api container.
Sizing from real usageGoldilocks recommends about 15 millicores for the API, which requests 100, so requests can follow measured use.

From the terminal

Checked on the live cluster

Real commands and their output from the same session, with the registry address removed.

$ kubectl run unsigned -n nexus --image=<registry>/nexus/api:unsigned-demo --overrides=<restricted security context>Error from server: admission webhook "ivpol.validate.kyverno.svc-fail-finegrained-require-signed-nexus-images" denied the request: Policy require-signed-nexus-images failed: Nexus images must be signed by the nexus-applications CI workflow on main.
An image built on a laptop and pushed to the real registry, with the restricted security settings that Pod Security demands. Kyverno still refused it, because CI never signed it.
$ kubectl get externalsecrets -ANAMESPACE NAME STORETYPE STORE REFRESH INTERVAL STATUS READY LAST SYNCcloudflared cloudflared-token SecretStore aws-secrets-manager 1h SecretSynced True 63smonitoring alertmanager-slack SecretStore aws-secrets-manager 1h SecretSynced True 59mmonitoring grafana-admin SecretStore aws-secrets-manager 1h SecretSynced True 59mnexus nexus-database SecretStore aws-secrets-manager 1h SecretSynced True 24s$ kubectl get peerauthentication -ANAMESPACE NAME MODE AGEnexus nexus STRICT 120m
External Secrets reads every secret from AWS Secrets Manager, so none is in Git, and Istio accepts only mutual TLS between the Nexus services.
$ kubectl get prometheusservicelevels -n nexusNAME SERVICE DESIRED SLOS READY SLOS GEN OK GEN AGE AGEnexus-api nexus-api 2 2 true 2m29s 140m
Sloth turns the API's availability and latency objectives into recording rules and burn-rate alerts.
$ curl -sI https://nexus-dev.sergeabroyan.com/ | grep -iE '^(http|server|x-cache|via|strict|content-sec|x-frame|x-content|referrer)'HTTP/2 200content-security-policy: default-src 'self'; base-uri 'none'; frame-ancestors 'none'; form-action 'self'referrer-policy: strict-origin-when-cross-originx-content-type-options: nosniffx-frame-options: DENYserver: istio-envoyx-cache: Hit from cloudfrontvia: 1.1 ce90704f6d2bc1f19459aaf24b07365e.cloudfront.net (CloudFront)strict-transport-security: max-age=31536000; includeSubDomains; preload
The application answers through CloudFront, with HSTS, a Content Security Policy, and framing turned off.

Stack

What it is built with

Infrastructure

  • AWS
  • Terraform
  • Terragrunt
  • Cloudflare

Kubernetes

  • EKS
  • Karpenter
  • KEDA
  • Istio
  • Helm

Delivery

  • GitHub Actions
  • Argo CD
  • Argo Rollouts
  • release-please
  • Renovate

Security

  • Kyverno
  • cosign
  • Trivy
  • External Secrets
  • Cloudflare Access

Observability

  • Prometheus
  • Grafana
  • Loki
  • Tempo
  • OpenTelemetry
  • Pyroscope
  • Sloth

Source

Repositories

nexus-platformAWS infrastructure with Terraform and Terragruntnexus-gitopsEverything that runs in Kubernetes, reconciled by Argo CDnexus-applicationsGo API, Java worker, and their delivery pipeline