Skip to content

feat(execution): add ResumeRun to replay output after a dropped stream - #3

Merged
0xmhha merged 1 commit into
mainfrom
feat/run-output-replay
Oct 29, 2026
Merged

0xmhha merged 1 commit into
mainfrom
feat/run-output-replay

Conversation

@0xmhha

@0xmhha 0xmhha commented Oct 29, 2026

Copy link
Copy Markdown
Member

Summary

When the Run stream drops (mobile networks, gateway restarts, several gateway instances), the client can currently only read the final status with GetExecution, so output produced after the drop is lost. This adds ExecutionService.ResumeRun, which replays buffered events and then continues live.

Design

  • New RPC. ResumeRun(execution_id, after_seq) is a server stream. It replays events with seq > after_seq from a per-run buffer and then follows the live run until Finished. Clients de-duplicate by seq.
  • Buffer. The buffer holds at most 1 MiB per run in Redis and is kept until 5 minutes after the run finishes. Redis was chosen so that the replay works with several gateway and execution instances.
  • Evicted events. RunEvent.ReplayGap{first_available_seq} opens the stream when part of the range was evicted.
  • Nothing buffered. The call fails with FAILED_PRECONDITION and ErrorDetail.REASON_REPLAY_UNAVAILABLE.
  • Service API. playground.svc.v1.ExecutionInternalService.ResumeRun carries the gateway's CallContext.
  • Authorization. Replay is owner-checked like Cancel and GetExecution, and it counts toward the per-visitor call limits.
  • Docs. RPC_DESIGN.md and SECURITY_PROTOCOL.md are updated. The version is 0.4.0.

Testing

  • buf lint passes, and buf breaking --against main reports no breaking change.
  • go test ./... passes, including new validation cases: a valid resume request, and an execution id with path characters that is rejected.
  • TS package: build and 7 tests pass.

Not verified

  • The server and client implementations come in follow-up pull requests in playground-backend and playground-frontend.

When a Run stream dropped, clients could only read the final status
with GetExecution; output produced after the drop was lost. ResumeRun
reattaches to a run owned by the caller: the server replays buffered
events with seq greater than after_seq and then continues with live
events until Finished.

- RunEvent.ReplayGap is sent first when part of the requested range
  was evicted from the per-run buffer (1 MiB, kept 5 minutes after the
  run finishes).
- ErrorDetail.REASON_REPLAY_UNAVAILABLE reports that nothing is
  buffered any more.
- ExecutionInternalService gains the matching RPC with a CallContext.

The change is backwards compatible with 0.3.0. The npm package
version is 0.4.0.
@0xmhha
0xmhha merged commit fccba3b into main Oct 29, 2026
2 checks passed
@0xmhha
0xmhha deleted the feat/run-output-replay branch October 29, 2026 02:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant