CLAUDE.md

# CLAUDE.md

You are building **Sovereign**, an autonomous cloud orchestration system. Read `SPEC.md` thoroughly before starting any work — it is the source of truth for architecture, data models, and acceptance criteria.

## Tech Stack

- **Backend:** Python, FastAPI, Pydantic, Redis (via `redis.asyncio` / redis-py)
- **Frontend:** Lit web components, [Web Awesome](https://webawesome.com/) components, plain CSS, Bun (not Node, not Vite)
- **LLM:** Google Gemini Flash via [DSPy](https://dspy.ai/) (typed signatures, not raw prompts)
- **Cloud:** Sprites (sprites.dev / Fly.io) for agent runtime, Redis for coordination
- **IDs:** nanoid (client-generated, server-validated for uniqueness)
- **Testing:** pytest, pytest-asyncio, httpx (for async test client)
- **No:** React, Next.js, Tailwind, Node.js, Vite

## Coding Style

- Type hints everywhere. Use `str | None` not `Optional[str]`.
- Pydantic models for all data shapes. Validate at the boundary.
- Async by default — async Redis, async FastAPI routes, async agent loops.
- Plain CSS. No utility classes. CSS custom properties for theming.
- Lit web components with standard conventions: properties, reactive state, shadow DOM.
- Minimal dependencies. Prefer the standard library. Justify every `pip install`.
- Small files. If a module exceeds ~200 lines, it probably wants to be two modules.

## Iteration Workflow

Development follows the V1 Roadmap in SPEC.md (Steps 1–7). Each step is one iteration. Work through them in order.

### For each iteration:

#### 1. Plan

Create an iteration document at `docs/NNN-step-name.md` (e.g., `docs/001-the-stream.md`).

The document starts with:

```markdown
# Step N: Name

Status: in_progress

## Goals
(What this step delivers — pull from SPEC.md acceptance criteria)

## Notes
(Living scratchpad — update this as you work)

## Spec Questions
(Anything the spec doesn't cover or gets wrong — these feed back into SPEC.md)

## Changelog
(Brief log of what you did and when)
```

#### 2. Write Tests

Write tests **before writing implementation code.** Tests should map directly to the acceptance criteria in SPEC.md for this step.

- One test file per module or major feature (e.g., `tests/test_outbox.py`, `tests/test_materializer.py`)
- Use `pytest` fixtures for Redis connections, test clients, and common AS2 objects
- Test the contract, not the implementation — tests should survive refactors
- Include at least one integration test that exercises the full request path (HTTP → outbox → Redis → materializer → read endpoint)

All tests will fail initially. This is correct.

#### 3. Implement

Loop:

1. Run the tests: `pytest -x` (stop on first failure)
2. Read the failure. Update `Notes` in the iteration doc with what you're addressing.
3. Write the minimum code to fix the current failure.
4. Run the tests again.
5. Repeat until all tests pass.

Do not write features that aren't covered by a test. If you realize you need something the tests don't cover, write the test first.

**After all tests pass:** make a git commit.

```
git add -A && git commit -m "step N: tests passing - [brief summary]"
```

#### 4. Smoke Test

Tests passing is necessary but not sufficient. After the test suite passes:

- Start the actual server: `python -m sovereign.main`
- Exercise the core path for this step manually (e.g., POST to `/outbox`, check `/activities`)
- For UI steps: open the browser and verify the components render and update
- If something is broken that the tests missed, write a test for it, then fix it

**After smoke test passes:** commit if there are changes.

#### 5. Analyze

Spawn two sub-agents to review the code. Each agent has a distinct role:

**Architecture Agent** — Reviews against the spec. Answers:
- Does the implementation match SPEC.md?
- Are the abstractions right? Will they hold up in later steps?
- Are there implicit assumptions that should be explicit?
- Is anything tightly coupled that should be decoupled?

**Quality Agent** — Reviews the code itself. Answers:
- Error handling: what happens when Redis is down, when input is malformed, when an agent crashes?
- Edge cases: empty streams, concurrent writes, malformed activities
- Dead code, unused imports, unclear naming
- Are there performance concerns (N+1 queries on Redis, unbounded reads)?

Synthesize both reports. Not every suggestion needs to be acted on — use judgment. Focus on things that would bite you in later steps or that violate the spec.

#### 6. Refactor

Apply the changes that matter from the analysis. Then:

1. Run the tests: `pytest`
2. If anything broke, fix it.
3. Run the tests again.
4. Maximum **two passes** through this analyze-refactor loop. Do not polish forever.

**After refactor:** commit.

```
git add -A && git commit -m "step N: refactored - [brief summary]"
```

#### 7. Close

- Update the iteration doc: set `Status: complete`, write a final `Changelog` entry.
- If you discovered spec issues (captured in `Spec Questions`), update `SPEC.md` now. Commit the spec changes separately:
  ```
  git add SPEC.md && git commit -m "spec: update from step N findings"
  ```
- Move on to the next step.

## Git Discipline

- **Branch:** Work on `main` for now. (Branching strategy is a post-V1 concern.)
- **Commit after tests pass** (Step 3) — this is your rollback point before refactoring.
- **Commit after refactor** (Step 6) — clean checkpoint.
- **Commit spec changes separately** — keep spec evolution traceable.
- **Commit messages:** `step N: [what happened]` for implementation, `spec: [what changed]` for spec updates.
- Never commit with failing tests.

## Testing Conventions

```python
# conftest.py fixtures you'll want:

@pytest.fixture
async def redis():
    """Fresh Redis connection with test prefix, cleaned up after each test."""

@pytest.fixture
async def client(redis):
    """httpx.AsyncClient wired to the FastAPI app with test Redis."""

@pytest.fixture
def make_activity():
    """Factory for valid AS2 activity dicts with sensible defaults."""

@pytest.fixture
def make_task():
    """Factory for valid sov:Task objects."""
```

- Test files mirror source files: `sovereign/outbox.py` → `tests/test_outbox.py`
- Use `make_activity()` and `make_task()` factories to avoid repeating AS2 boilerplate
- Test Redis state directly when verifying materialization — don't only test through HTTP
- Async tests: use `@pytest.mark.asyncio` and `async def test_*`

## Key Design Reminders

These are easy to forget mid-implementation. Revisit them each step.

- **`/outbox` is the only write path.** Everything that changes state goes through it. No backdoor writes to Redis.
- **Activities are the source of truth.** Materialized state is a cache. If Redis hashes get corrupted, replaying the stream rebuilds them.
- **AS2 vocabulary first.** Use standard Activity Streams properties before reaching for `sov:` extensions. Refer to the Data Models section in SPEC.md.
- **Agents are just processes.** They boot, connect to Redis, read their task, do work, post activities, and die. No daemon, no supervisor, no long-running connection pool.
- **The feed is the UI, the debugger, and the audit log.** If something happens and it's not in the feed, it didn't happen.

## When You're Stuck

- Re-read the relevant section of SPEC.md. The answer is usually there.
- Check the `Notes` section of your iteration doc — you may have already worked through this.
- If the spec is genuinely ambiguous or wrong, note it in `Spec Questions` and make a reasonable decision. Don't block.
- If a test is testing the wrong thing, fix the test. Tests serve the spec, not the other way around.

## Tech Guides

When working on a tech, framework, or package check `docs/lore/` for reference guides.