Cyoda-Go

# Cyoda-Go

Lightweight, multi-node Go implementation of the Cyoda platform.
cyoda-go defines the API and integration contract; Cyoda Cloud aligns to it.

## Design Philosophy

**Correctness and consistency over availability.** When the two conflict,
cyoda-go fails closed: an operation that cannot be completed correctly is
rejected and rolled back — never committed partially, and never with a
substituted or fallback value standing in for the correct one. Do not weigh
availability as a reason to weaken an invariant, and do not propose
graceful-degradation fallbacks that return a wrong-but-available result. An
unavailable dependency a correct result requires (e.g. the compute node a
required Function/processor/criterion callout needs) fails the operation; it
does not downgrade it. See `.claude/rules/correctness-over-availability.md`.

## Development Gates

These are STOP-and-verify checkpoints. Do not proceed past a gate without completing it.

### Gate 1: TDD is mandatory
Do not write implementation code without a failing test driving it.
Use `superpowers:test-driven-development` skill for all feature and bugfix work.
See `.claude/rules/tdd.md` for the full protocol.

### Gate 2: E2E test coverage
When adding or changing user-facing behavior (API responses, workflow semantics, error codes),
add or update E2E tests in `internal/e2e/` to cover the change through the full HTTP stack.
E2E tests are self-contained: `TestMain` starts a PostgreSQL container via testcontainers-go
and an in-process `httptest.Server` with JWT auth — no external instance needed.
Run `make test-full`, which includes them (requires Docker running). To iterate on
E2E alone, `go test ./internal/e2e/...` is fine — but do not read a green from it
as whole-suite verification, and do not add `-v`.
For API/gRPC features the bar is **full coverage** — happy path AND every documented
status/error code on a running backend, plus a cross-backend parity scenario for
backend-agnostic behavior. See `.claude/rules/test-coverage.md` (the spec's error table is
the checklist). `superpowers:brainstorming` and `superpowers:writing-plans` auto-inject these
gates via a `PreToolUse` hook (`.claude/hooks/inject-skill-gates.sh`).

### Gate 3: Security by default
Never log credentials, tokens, secrets, or signing keys at any level.
Every data path must be verified for tenant isolation — no cross-tenant leakage.
Validate input at system boundaries. Sanitize output — no stack traces or internals in responses.

### Gate 4: Documentation hygiene
When changing env vars, update the relevant `cmd/cyoda/help/content/config/*.md` help topic, `README.md`, and `DefaultConfig()` together.
When changing public interfaces or developer workflow, check `README.md` and `CONTRIBUTING.md`.
When bumping the `cyoda-go-spi` pin in `go.mod`, when tagging a `cyoda-go` binary release, when changing the chart `version:` / `appVersion:`, or when out-of-tree-plugin pin guidance changes, update `COMPATIBILITY.md` in the same change.
When changing the `WorkflowConfigurationDto` import surface — DTO shape, validation strictness, accepted-input set, exporter stamping behaviour — follow `docs/workflow-schema-versioning.md`: bump rules, tightening-release decision rubric (MINOR retirement vs dual-shape acceptance), and the required commit-time checks. The doc is the canonical place to record the rationale for each bump.

### Gate 5: Verify before claiming done
Use `superpowers:verification-before-completion` skill before claiming work is complete.
Run `make test-full` and confirm green (root + every plugin submodule, including E2E). Run `go vet ./...` for static analysis.
A raw `go test ./...` is not sufficient: it reports `ok` for suites that never ran.
E2E tests spin up their own PostgreSQL + HTTP server automatically — just run them.
Do not claim work is done if any test — unit, integration, or E2E — is failing.

Race detector (`make race`) is a one-shot sanity check before PR creation,
not a per-step gate — see `.claude/rules/race-testing.md`. The target excludes
`internal/e2e` from the race scope (timeout-driven carve-out, rationale in the
rule) and CI runs the identical target, so local and CI stay in lock-step.

### Gate 6: Continuous improvement — resolve, don't defer
We strive for continual improvement of code quality and progressively reduce
technical debt. When you spot an issue — dead code, an outdated comment, a
weakened invariant, a missed test, a stale TODO — **resolve it now via
red/green TDD within reason**, do not delegate it to "later". The default is
to fix; deferring requires a reason (out of scope, architectural decision
needs human input, would balloon the change beyond reviewability).

"Within reason" means:
- Bounded in scope: the fix is comprehensible alongside the work that surfaced it.
- Reversible: small commits, each with its own failing test.
- Doesn't bypass other gates: still TDD, still reviewed, still verified.

If the fix is structural, requires a design decision, or would significantly
expand the change, **stop and surface the choice to the human** rather than
silently leaving it broken. Recording a `TODO(...)` is the last resort, not
the first response.

### Gate 7: Cloud-parity — cyoda-go leads the contract
cyoda-go **defines** the API and integration contract; Cyoda Cloud follows it.
Any change to the integration contract (API shape, wire/error semantics, entity
or workflow behaviour Cloud mirrors) must be reconciled with cyoda-cloud and the
change logged in `docs/cloud-parity/` — one file per feature/behaviour.

## External Storage Plugins

The Cassandra storage plugin lives in a separate repository:
- Repo: https://github.com/cyoda/cyoda-go-cassandra
- Local checkout: `../cyoda-go-cassandra`

When doing cross-plugin work (changing the SPI, adding parity tests, modifying
shared contracts), verify the change does not break the Cassandra plugin. The
parity test registry (`e2e/parity/registry.go`) is picked up by all backends
including Cassandra — new parity tests will surface there on their next
dependency update.

A storage backend (memory/sqlite/postgres) diverging from the others on the same
contract is a bug to fix, not an "accepted divergence"; cross-backend parity tests
guard that consistency, they don't define behaviour.

## Go Conventions

- Go 1.26+. Use `log/slog` exclusively — never `log.Printf` or `fmt.Printf`.
- Manual dependency injection via constructors. No DI frameworks.
- Wrap errors with context: `fmt.Errorf("failed to X: %w", err)`.
- Use `uuid.UUID` type, not `string`, for UUIDs.
- Config via env vars with `CYODA_` prefix and sensible defaults.
- 4xx errors: full domain detail with error code. 5xx: generic message + ticket UUID.
- Deferred work: `// TODO(plan-reference): description`. Run `make todos` to list all.

## Workflow

### New feature
brainstorming → writing-plans → subagent-driven-development → verification-before-completion → requesting-code-review → security-review → PR/merge

### Bugfix
test-driven-development → verification-before-completion → requesting-code-review → security-review → PR/merge

### Receiving review feedback
receiving-code-review

All workflow skills are in the `superpowers:` namespace.
Security review uses `antigravity-bundle-security-developer:cc-skill-security-review`.

Do not skip steps. Brainstorming prevents building the wrong thing.
TDD prevents shipping untested code. Verification prevents false "done" claims.
Review and security audit prevent defects reaching main.

**The review gates need a *fresh-context* reviewer, and this is a standing
request from the user to dispatch one.** `requesting-code-review` and the
brainstorming design review (`.claude/rules/gate-brainstorming.md`) both depend
on a reviewer that has not seen the working context; running them inline defeats
their purpose. Treat reaching either gate as the user having asked for a subagent
on that task — no need to ask again. If anything still prevents dispatching one,
**say so and stop — never drop the gate silently and mention it afterwards.**

## Common Commands

Plugin submodules (`plugins/memory`, `plugins/sqlite`, `plugins/postgres`)
each have their own `go.mod`; Go's `./...` recursion does **not** cross
module boundaries, so root-module commands miss them. `make test-full` is the
only target that covers the whole repo.

**Use `make test` and `make test-full`. Do not hand-roll `go test ./...`.**
A raw `go test` cannot tell you that a suite did not run: a `TestMain` that
calls `os.Exit(0)` — how the parity suites and `internal/e2e` opt out of
`-short` — prints `ok  pkg  2.80s` with zero tests executed, which is
indistinguishable from a pass. The make targets pipe through
`scripts/testreport`, which fails the run when a required suite executed
nothing and prints failures verbatim instead of burying them in `-v` output.

- **Iteration: `make test`** — unit + cross-backend parity, ~115s cold and
  ~85s warm against the test cache; the warm floor is the parity suites
  re-executing, which they always do. Excludes
  `internal/e2e` and the plugin submodules, and says so when it runs.
- **End of deliverable: `make test-full`** — everything, root + all three
  plugin submodules, ~15 min. This is the Gate 5 command.
- Race detector (CI-parity scope): `make race` — `go test -race` on every
  package except `internal/e2e`; the same scope CI runs. Use
  `go test -race -timeout=20m ./internal/e2e/...` separately if you need race
  coverage on E2E. Once before a PR, not per step.
- One package while iterating: `go test ./internal/domain/search/...` is fine
  — the tiers are for verification, not for every edit.

**Do not add `-count=1`.** The targets deliberately leave Go's test cache
enabled, so a re-run only re-executes packages whose inputs changed: measured
114s cold and 84s warm for `make test`, the warm figure being the parity suites
re-executing on every run. Forcing `-count=1` throws the rest away and
is the single easiest way to make verification slow again. Use it only when
you are specifically hunting a flake. The parity suites are the one exception,
and the make targets already run them uncached in their own invocation: their
fixtures build the server binary in a subprocess the test cache cannot see, so
a cached parity pass can be replayed over a server built from the old code.
- Coverage (root module): `go test -coverprofile=coverage.out ./...` — run
  inside each `plugins/*` for per-plugin coverage.

**Docker is a hard requirement for both tiers and there is no Docker-free
fallback.** `make preflight` checks that the daemon responds, that the probe
image is present, and that the VM disk can take a write — a full Docker disk
otherwise surfaces only as `container exited with code 1`. If pre-flight
fails, fix Docker and say so; do not drop to a narrower run.

- Build: `go build -o bin/cyoda ./cmd/cyoda`
- Tidy: `go mod tidy`
- Vet (root module): `go vet ./...` — the `per-module-hygiene` CI job vets each plugin separately