diff --git a/docs/sprites-exec-api.md b/docs/sprites-exec-api.md new file mode 100644 index 0000000..bfb8309 --- /dev/null +++ b/docs/sprites-exec-api.md @@ -0,0 +1,269 @@ +# Sprites Exec API: Opportunities for taskyou + +## Context + +PR #103 integrates Sprites as a cloud execution environment for Claude tasks. +PR #160 adds a web UI with per-user Sprites. + +Both PRs use the `sprites-go` SDK's `CommandContext()` method, which wraps the +exec WebSocket API. However, they layer tmux on top and use file-tailing for +hook communication. The exec API has native features that eliminate both. + +## Current Architecture (PR #103) + +``` +Local Sprite VM +───── ───────── +Executor tmux session "task-42" + │ └── claude --dangerously-skip-permissions + ├─ sprite.Command("tmux new-session") + ├─ sprite.Command("tail -f hooks.jsonl") ◄── hook script appends JSON + └─ polls "tmux has-session" every 2s +``` + +Problems: +1. tmux is redundant — exec sessions already persist across disconnections +2. Polling tmux every 2s wastes resources and adds latency +3. File-based hook streaming is fragile (race conditions, no backpressure) +4. No reconnection support if the local machine restarts +5. Setup requires installing tmux, Node.js, Claude CLI on the sprite + +## Exec API Capabilities + +The exec API (WSS + HTTP POST at `/v1/sprites/{name}/exec`) provides: + +| Feature | What it replaces | +|---------|-----------------| +| Persistent sessions | tmux sessions | +| `max_run_after_disconnect` | tmux detach behavior | +| Session listing (`GET /exec`) | `tmux has-session` polling | +| Session attach (`WSS /exec/{id}`) | `tmux attach` | +| Session kill (`POST /exec/{id}/kill`) | `tmux kill-session` | +| Port notifications | Manual port tracking | +| `ExitMessage` with exit code | Parsing tmux output | + +Additional APIs that help: + +| API | What it replaces | +|-----|-----------------| +| Filesystem (`/v1/fs/*`) | `sprite.Command("mkdir")`, `sprite.Command("cat")` | +| Services (`/v1/services/*`) | `nohup taskd &` + polling | +| FS Watch (`WS /v1/fs/watch`) | `tail -f /tmp/task-hooks.jsonl` | +| Network Policy (`/v1/policy`) | Manual iptables/allowlists | + +## Recommended Changes + +### 1. Replace tmux with native exec sessions + +**Before (PR #103):** +```go +// Start Claude in tmux on sprite +cmd := sprite.CommandContext(ctx, "tmux", "new-session", "-d", "-s", sessionName, + "sh", "-c", claudeCmd) +cmd.CombinedOutput() + +// Poll for completion +for { + checkCmd := sprite.CommandContext(ctx, "tmux", "has-session", "-t", sessionName) + if err := checkCmd.Run(); err != nil { + break // session ended + } + time.Sleep(2 * time.Second) +} +``` + +**After:** +```go +// Run Claude directly via exec API +cmd := sprite.CommandContext(ctx, "claude", + "--dangerously-skip-permissions", "-p", prompt) +cmd.Dir = workDir +cmd.Env = []string{ + fmt.Sprintf("TASK_ID=%d", task.ID), + fmt.Sprintf("WORKTREE_PORT=%d", task.Port), +} + +// Session persists if we disconnect (key feature!) +// Set max_run_after_disconnect to keep Claude running +cmd.SetMaxRunAfterDisconnect(0) // 0 = run forever + +// Stream stdout for log capture +stdout, _ := cmd.StdoutPipe() +cmd.Start() + +// cmd.SessionID() gives us the session ID for reconnection +sessionID := cmd.SessionID() +db.SetTaskSessionID(task.ID, sessionID) + +// Read output and wait for exit +scanner := bufio.NewScanner(stdout) +for scanner.Scan() { + db.AppendTaskLog(task.ID, "claude", scanner.Text()) +} +cmd.Wait() +``` + +Benefits: +- No tmux dependency on sprite +- No polling — `cmd.Wait()` blocks until exit +- Session ID stored for reconnection after crash +- `ExitMessage` gives exact exit code +- Stdout streamed directly, no file intermediary + +### 2. Reconnect to running sessions after restart + +The exec API's killer feature: sessions survive client disconnections. + +```go +// On executor restart, check for existing sessions +sessions, _ := client.ListSessions(ctx, spriteName) +for _, session := range sessions { + if isTaskSession(session) { + // Reattach to running Claude session + cmd := sprite.AttachSession(ctx, session.ID) + stdout, _ := cmd.StdoutPipe() + // Resume streaming logs... + } +} +``` + +This solves the current problem where restarting the local `task` daemon +loses track of running sprite tasks. + +### 3. Replace file-tailing hooks with FS Watch or direct streaming + +**Option A: FS Watch (recommended)** +```go +// Watch hooks file for changes via WebSocket +watcher := sprite.Filesystem().Watch(ctx, "/tmp/task-hooks.jsonl") +for event := range watcher.Events() { + // Process new hook events as they're written +} +``` + +**Option B: Direct stdout parsing** +Since Claude's output is now streamed directly through the exec WebSocket, +parse hook-like events from stdout instead of a sidecar file: +```go +// Claude's MCP server already writes structured output +// Parse it from the exec session's stdout stream +``` + +**Option C: Keep file-based hooks but use Filesystem API** +```go +// Read hooks file periodically via FS API (no exec needed) +data, _ := sprite.Filesystem().ReadFile("/tmp/task-hooks.jsonl") +``` + +### 4. Use Services API for the web API (PR #160) + +PR #160 runs `taskd` inside the sprite with `nohup ... &` and polls for +readiness. The Services API manages this properly: + +```go +// Create a service for the web API +service, _ := client.CreateService(ctx, spriteName, &sprites.ServiceConfig{ + Name: "webapi", + Command: "/usr/local/bin/taskd --web-api --addr :8080", +}) + +// Start it +client.StartService(ctx, spriteName, service.ID) + +// Get logs +stream, _ := client.GetServiceLogs(ctx, spriteName, service.ID) +``` + +Benefits: +- Auto-restart on crash +- Proper log management +- Clean start/stop lifecycle +- No polling for readiness + +### 5. Use Filesystem API for setup + +**Before:** +```go +// Install hook script via exec +installCmd := fmt.Sprintf(`cat > /usr/local/bin/hook << 'EOF'\n%s\nEOF`, script) +sprite.CommandContext(ctx, "sh", "-c", installCmd).Run() +``` + +**After:** +```go +// Write files directly via filesystem API +fs := sprite.Filesystem() +fs.WriteFile("/usr/local/bin/task-sprite-hook", []byte(hookScript), 0755) +fs.WriteFile("/root/.claude/settings.json", []byte(claudeSettings), 0644) +fs.MkdirAll("/workspace", 0755) +``` + +### 6. Use Network Policy for security + +```go +// Set network policy to restrict sprite access +client.SetPolicy(ctx, spriteName, &sprites.Policy{ + AllowedDomains: []string{ + "github.com", + "api.anthropic.com", + "api.openai.com", + "rubygems.org", + "registry.npmjs.org", + }, +}) +``` + +### 7. Use Port Notifications for dev servers + +When Claude starts a dev server inside the sprite, the exec API sends a +`PortNotificationMessage` with a proxy URL: + +```go +cmd.TextMessageHandler = func(data []byte) { + var notification sprites.PortNotificationMessage + json.Unmarshal(data, ¬ification) + if notification.Type == "port_opened" { + // notification.ProxyURL is the public URL + db.AppendTaskLog(task.ID, "system", + fmt.Sprintf("Dev server at %s", notification.ProxyURL)) + } +} +``` + +This enables the TUI to show clickable URLs when Claude starts servers. + +## Implementation Priority + +1. **Replace tmux with exec sessions** — Biggest simplification, removes a + dependency, enables reconnection. Affects `executor_sprite.go`. + +2. **Session reconnection on restart** — Critical for reliability. Affects + `executor.go` startup logic. + +3. **Filesystem API for setup** — Cleaner, no shell escaping issues. Affects + `sprite.go` setup code. + +4. **Network Policy** — Security improvement. Small change. + +5. **Port notifications** — Nice UX improvement. Small change. + +6. **Services API for PR #160** — Only relevant for the web UI PR. + +7. **FS Watch for hooks** — Nice to have, current file-tailing works. + +## SDK Version Note + +The `sprites-go` SDK at `v0.0.0-20260109202230-abba9310f931` (used in PR #103) +already has exec session support, filesystem operations, services, and session +management. The `Cmd` struct supports `SessionID`, `TextMessageHandler`, TTY, +and environment variables. No SDK upgrade needed — the features just need to +be used. + +## Summary + +The exec API eliminates the need for tmux on sprites entirely. Instead of +layering tmux → Claude with file-based hook streaming, we run Claude directly +as an exec session with native persistence, reconnection, and streaming. This +reduces setup complexity (no tmux/Node.js installation), improves reliability +(crash recovery via session reconnection), and simplifies the code +significantly.