GCP & Cloud Run

Cloud Build pipeline, Cloud Run Pro API sizing, cost model, and substitution secrets for PYNE deployments.

This page

GCP & Cloud Run

Abstract

Production-shaped hosting for the Pro API targets Google Cloud Run, fed by Cloud Build (cloudbuild.yaml). The pipeline builds a container, pushes multi-tags to Container Registry, deploys a managed service, and can optionally compile the LSP binary in the same build graph.

This page codifies the as-checked-in config and the cost envelope from docs/gcp_cost_estimate.md — treat dollar figures as planning estimates, not invoices.

Conceptual model

Diagram

Rendering…

Interface surface

Substitutions (cloudbuild.yaml)

SubstitutionDefaultMeaning
_SERVICE_NAMEpynescript-pro-apiCloud Run service
_REGIONus-central1Deploy region
_REPOSITORYpynescriptImage path segment
_METADATA_KEY(secret/subst)Fernet key → CRYPTO_KEY for LSP stage
_PYNESCRIPT_VERSION"0.3.0" (stale default in YAML)OCI label; override to 0.3.14 at submit

Build steps

  1. build — docker build with BuildKit; tags $COMMIT_SHA + latest; cache-from latest
  2. push — docker push --all-tags
  3. deploy — gcloud run deploy with managed platform flags
  4. build-lsp — python:3.12-slim image runs scripts/build/ci_build.py --jobs=4 with CRYPTO_KEY=${_METADATA_KEY}

Cloud Run flags (as configured)

FlagValueRationale
--allow-unauthenticatedonPublic HTTP API (auth via API keys at app layer)
--min-instances0Scale to zero
--max-instances10Cap spend / blast radius
--memory512MiEvaluator headroom
--cpu1Single vCPU per instance
--concurrency10Parallel requests per instance
--timeout60sPlatform kill. Entrypoint gunicorn default is 120s (GUNICORN_TIMEOUT); set the env to 60 on Cloud Run or long evals die at the platform first.
--set-env-varsFLASK_ENV=productionApp mode

Machine / budget knobs

options:
  machineType: E2_HIGHCPU_8
  logging: CLOUD_LOGGING_ONLY
timeout: 1200s

Internals

PathRole
cloudbuild.yamlPipeline definition
Dockerfile target apiProduction image recipe used by Cloud Build
backend/app.pyHealth /, CORS, body size limit
docs/gcp_cost_estimate.mdScenario costs (April 2026 draft)

Cost envelope (planning)

ScaleRough monthlyNotes
Free / hobby~$0Within Cloud Run free tier if tiny
MVP (~10 pro users)~$50Cloud Run + micro SQL
Growth (~50)~$120Add Redis caching
Scale (~200)~$350Dominated by evaluator CPU-sec

Dominant cost driver: script execution CPU time (~2–5 CPU-sec per typical backtest over 1k bars). Memory ~200–400MB per concurrent Python worker.

Free-tier leverage (GCP)

  • Cloud Run: 2M requests, 400K CPU-sec, 200K GB-sec / month
  • Cloud Build: 120 build-minutes / day
  • Artifact Registry / GCS small free slices

Invariants & edge cases

  1. App-layer auth still required when Cloud Run allows unauthenticated invoke — see Security.
  2. Scale-to-zero cold starts add latency; min-instances=1 trades money for snappiness.
  3. LSP build step does not deploy the binary to Cloud Run; it is a release-adjacent compile in the same YAML.
  4. 512Mi may be tight for large OHLCV payloads — watch OOM kills; raise memory before raising concurrency.
  5. Cost estimate doc also lists Railway / Render / Fly / self-host alternatives if GCP is overkill.

Worked examples

Manual deploy sketch (operator)

gcloud builds submit --config cloudbuild.yaml \
  --substitutions=_METADATA_KEY="$METADATA_KEY"

Smoke health after deploy

curl -s "https://SERVICE-URL/" 
# expect JSON status healthy, service pynescript-pro-api

Failure modes

SymptomCauseFix
Deploy timeoutNuitka step + docker on 20 min budgetSplit LSP compile to release pipeline
429 / queuemax-instances or concurrencyRaise caps carefully; add queue
High billNo cache; chatty clientsRedis TTL; Cloud Tasks for backtests
Auth bypass illusionUnauthenticated Cloud RunEnforce API keys in Flask
Image pull errorsWrong project / registry permsIAM on GCR + Cloud Run service agent

See also