commit e9e88c1 · ran as cli · 90s · ~34k tokens · evidence sealed & signed (manifest.json)
21/30
The project builds and its offline unit tests pass, and the header design reflects a coherent, well-documented port of a TypeScript SDK's wire contract with sensible error typing and dependency injection for testability. However, the WebSocket-dependent control channel — the core interactive/streaming functionality the README advertises — was explicitly disabled for this run, so the most significant claims (live sandbox exec, streaming commands, git-over-exec) remain unverified. Code quality looks solid at the interface level but cannot be fully judged without seeing .cpp implementations or a live/gateway-driven run.
Strengths
- Builds cleanly with CMake FetchContent for all deps and compiles without warnings-as-errors surfaced in the log
- Offline test suite (test_contract_wire.cpp / test_main.cpp) actually passes (1/1 ctest, doctest-based)
- Header design is thoughtful: clear separation of transport-agnostic ControlChannel (byte-fed, unit-testable without sockets) from IXWebSocket wiring, typed error hierarchy mapped to gateway HTTP codes, and a CommandRunner injection pattern for git ops that mirrors the reference TS test approach
- Documentation in headers explains protocol framing (three frame kinds dispatched by shape) with enough precision to suggest real understanding, not just boilerplate comments
Concerns
- Only compiled and tested with -DSOLARI_WITH_WS=OFF, meaning the actual WebSocket control-channel path (commands.run, streaming exec, connect()) was never exercised live; the README's core promise ('.commands', 'sbx->connect()', streamed exec) is unverified
- No live gateway was reached in this review — all evidence is a single offline unit test binary passing; deliversClaims can only be partially credited since the bulk of README example code (sandbox creation, exec, file I/O, code.run charts) was never run end-to-end
- Only 1 test target reported by ctest (aggregated internally); can't tell how many actual assertions/cases ran or their granularity from the summary alone
- Source excerpts are headers only — no .cpp implementation bodies shown, so real error handling, resource cleanup, and concurrency correctness (mutexes/futures in ControlChannel) can't be verified from what's given
The submission's own tests
ctest --test-dir build --output-on-failure → PASS ✅
Security sweep
audit: n/a · secrets: no matches
Ask the candidate
- Walk us through what happens end-to-end when SOLARI_WITH_WS is ON and Sandbox::connect() is called — how does the background receive thread in ws_transport.cpp interact with the futures blocking on RPC replies in ControlChannel?
- The three frame kinds (JSON-RPC reply, v1 streamed exec, v2 async STREAM) are dispatched by shape in onMessage() — what's the disambiguation logic when a message could plausibly match more than one shape, and how did you test that boundary?
- GitOps takes an injected CommandRunner to avoid shelling out directly — walk through how args are escaped/passed to avoid injection when composing clone/commit/push commands, and what test coverage exists for that specifically?
Generated by Gauntlet — sandboxes & cloud browsers by Solari, judgment by Claude. Verify this evidence: npm run verify -- <this directory>