commit f2ad80d · ran as cli · 69s · ~31k tokens · evidence sealed & signed (manifest.json)
24/30
A Go SDK package that builds cleanly and passes its full unit test suite, with well-documented, thoughtfully structured code (typed errors, careful WS frame handling, a wire-contract pinning test). Live integration against a real gateway was not exercised (tests skip without credentials), so README claims about cross-language wire-identical behavior are only partially checkable from this run. Code quality excerpts look professional, but assessment is necessarily limited to what was actually built and tested rather than production network behavior.
Strengths
- Clean build with go1.23, and the full unit test suite (git, commands, files, code, channel, contract-wire tests) passes, including nuanced git auth-injection and status-parsing edge cases
- Thoughtful error-type hierarchy (SolariError base with AuthError/PlanError/ConcurrencyLimitError/etc.) that maps cleanly onto errors.As matching, as documented in the README and reflected in errors.go
- Channel transport (channel.go) shows real engineering care: buffering partial WS frames, dispatching by frame shape, mutex-guarded pending/orphan maps, and clear doc comments explaining wire-protocol quirks
- contract_wire_test.go + testdata/wire-contract.json suggest a deliberate mechanism to pin the Go SDK's wire shape against a reference contract, catching 'silent drop' bugs when new CreateOptions fields aren't wired through
Concerns
- Live tests (TestLiveDesktop, TestLiveSandboxHasNoStreamURL) are skipped without SOLARI_API_KEY, so no evidence was observed of the SDK actually talking to a real gateway/control WebSocket — cross-language wire parity claims in the README remain unverified by this run
- All passing tests appear to exercise mocked/local transports rather than a live server, so concurrency-heavy code (async frame dispatch, CommandHandle buffering) is unit-tested but not proven under real network conditions
- No visibility into files.go/git.go/types.go contents beyond references — can't fully assess error handling depth or edge cases in those namespaces from the excerpts alone
The submission's own tests
go test ./... -v → PASS ✅
Security sweep
audit: n/a · secrets: no matches
Ask the candidate
- In channel.go you handle three frame kinds (JSON-RPC reply, v1 stream, v2 async) dispatched by shape — walk through a scenario where a v2 async STREAM frame arrives before its owning cmd.start reply, and how orphanFrames/maxOrphanFrames prevents a leak.
- CommandHandle.OnData buffers output until the first subscriber attaches, then replays it — what's the concurrency guarantee that a chunk delivered mid-replay isn't dropped or duplicated?
- contract_wire_test.go pins CreateOptions -> wire body via buildCreateBody; if you added a new field to CreateOptions today, what exactly would fail and how would you update wire-contract.json to keep it honest across the TS/Python SDKs?
Generated by Gauntlet — sandboxes & cloud browsers by Solari, judgment by Claude. Verify this evidence: npm run verify -- <this directory>