Cloud Run
Twelve services. Most use a zero minimum. Query and migration runners get low concurrency and strict maximums.
Northstar has twelve deployable backend processes, but most of them will sit idle during the internal pilot. Cloud Run keeps the service boundaries without paying for twelve servers all day.
Start in asia-south1. Keep the control data in that region. Route database reads and migration work through small, capped worker pools. Add regional workers only when customer database locations make the extra latency worth it.
Each Go command can remain a separate Cloud Run service. Only three routes need public traffic. Everything else should reject calls that do not carry a valid service identity.
| Process | Exposure | Initial size | Initial scaling | Reason |
|---|---|---|---|---|
api-gateway | Public through one hostname | 1 vCPU, 512 MiB | Min 1 in production, max 5, concurrency 40 | Browser entry point. One warm instance avoids a slow first screen. |
agent-gateway | Public MCP hostname | 1 vCPU, 512 MiB | Min 0, max 5, concurrency 20 | MCP requests are bursty. Keep a lower concurrency until query limits are proven. |
github-service | Public webhook route only | 1 vCPU, 512 MiB | Min 0, max 3, concurrency 40 | Verify the signature, store the event, enqueue work, and return fast. |
identity-service, access-service | Private | 1 vCPU, 512 MiB each | Min 0, max 3, concurrency 40 | Small control-plane requests. |
connection-service, credential-broker | Private | 1 vCPU, 512 MiB each | Min 0, max 3, concurrency 20 | These touch encrypted credentials. Keep the caller list narrow. |
metadata-service, audit-service | Private | 1 vCPU, 512 MiB each | Min 0, max 3, concurrency 40 | Control database reads and writes. |
migration-service | Private worker API | 1 vCPU, 1 GiB | Min 0, max 3, concurrency 5 | Builds source manifests and records run state. |
query-runner | Private | 1 vCPU, 1 GiB | Min 0, max 5, concurrency 4 | Every request can hold a connection to a customer database. |
liquibase-runner | Private | 2 vCPU, 2 GiB | Min 0, max 2, concurrency 1 | Liquibase uses Java and runs external processes. Isolate each run. |
connector-agent | Do not deploy centrally yet | Customer dependent | Disabled | Use it later for databases that have no public route. |
The control plane needs managed compute, a durable database, private networking, task delivery, container storage, secrets, and logs. Redis and Kubernetes add cost without solving a current problem.
Twelve services. Most use a zero minimum. Query and migration runners get low concurrency and strict maximums.
One control database per environment. Start staging as single-zone. Use regional HA, backups, and point-in-time recovery in production.
Move webhook and migration work off the request path. Use OIDC service identity and database-backed idempotency.
Build one image per process from a commit SHA. Promote the exact digest from staging to production.
Store short-lived, checksum-addressed migration bundles. Lifecycle rules should delete old bundles after the audit window.
Hold runtime secrets in Secret Manager. Replace the single AES environment key with KMS envelope encryption before customer use.
Give Cloud SQL a private address. Route database workers through a fixed egress IP when a target database requires allowlisting.
Alert on failed tasks, failed migrations, high database connections, repeated authorization failures, and old in-progress runs.
Publish sanitized reports now. Publish the product UI only after production authentication is in place.
The first deployment should protect databases, not chase a large request number. The following tiers are operating targets. They are not promises about provider limits.
| Tier | Working assumption | Database work | Control database | Change that unlocks it |
|---|---|---|---|---|
| Internal pilot | Up to 25 people, 50 saved targets, 100 migration runs per day | 10 active queries and 2 Liquibase runs at once | 2 vCPU, about 8 GiB RAM, single-zone staging | Current recommendation |
| Early customers | Up to 500 people, 1,000 saved targets, 1,000 migration runs per day | 100 active queries and 10 Liquibase runs across targets | 4 vCPU, about 16 GiB RAM, regional HA | Connection pooler, per-tenant limits, queue partitions |
| Regional product | Several thousand people and databases in more than one geography | Workers placed near target databases | Control data remains regional, reporting reads may use replicas | Regional runner pools and customer connector agents |
The service boundaries are usable. Five local assumptions still block a safe cloud deployment.
DEV_MODE creates a local browser session. Keep staging private behind an identity proxy, then add real SSO or product login before any customer can reach the API.
The migration service reads exact commits from /repos. Cloud Run will not have those folders. Fetch the approved GitHub commit with a GitHub App and store a checksum-addressed bundle.
A webhook must verify, persist, enqueue, and return. Cloud Tasks should run previews and applies. Existing idempotency keys and Liquibase history already provide a good base for retries.
Use one service account per process and Cloud Run identity tokens. Keep the internal token only as a temporary staging control.
The SQL files under backend/deploy/postgres/init need a real migration job. Convert them into a versioned changelog and run it before each service release.
Eight control services each open up to eight PostgreSQL connections per instance. Set small per-service pools, cap Cloud Run instances, and give long migrations an asynchronous job timeout.
Each phase has a pass condition. A successful container deployment is not enough.
DEV_MODE by default.asia-south1.The repository now has a dedicated public-reports directory. Only files in that directory are uploaded. Internal docs, credentials, and source files stay out of the Pages deployment.
./scripts/deploy-public-reports.sh
cp report.html public-reports/ ./scripts/deploy-public-reports.sh
public-reports becomes public. Remove credentials, customer names, internal hostnames, database data, account numbers, and private screenshots before publishing.The local Wrangler OAuth login is suitable for manual publishing. CI should use a separate API token limited to Cloudflare Pages. Do not copy the local OAuth credential into GitHub Actions.
Approve an isolated staging build on Cloud Run. Reject a production launch until authentication, remote repository fetch, asynchronous execution, service identity, and control database migrations are fixed.
New project, one region, Cloud Run, private Cloud SQL, Cloud Tasks, fixed egress, KMS, logs, and a test-only PostgreSQL target.
The current local login and mounted repository model are not safe or workable in a serverless production deployment.