[tyffical]

Motivation

• Local execution has to work on every Vite major the plugin supports (5 through 8), but CI only runs it against the Vite 6.4.3 that Jest loads.
• Jest can't load Vite 7+, which ships ESM only, so local execution on those versions has no automated coverage.
• Local execution's fresh process per call, parallel calls and sandbox need a check that runs the real dev server on each version, from a fresh install the way a customer gets it.
• Ticket: APPS-2961 Run local backend functions in parallel, with per-call realtime console logs.
• Stacks on [APPS-2961 Update: run local backend function calls in parallel, with per-call log prefixes]([APPS-2961] Update: run local backend function calls in parallel, with per-call log prefixes #603).
• Stack order, merged bottom up:
  1. [APPS-2961 Refactor: move local execution's side-effect-free helpers into their own module]([APPS-2961] Refactor: move local execution's side-effect-free helpers into their own module #604)
  2. [APPS-2961 Update: run each local backend function call in a fresh sandboxed process]([APPS-2961] Update: run each local backend function call in a fresh sandboxed process #608)
  3. [APPS-2961 Update: run local backend function calls in parallel, with per-call log prefixes]([APPS-2961] Update: run local backend function calls in parallel, with per-call log prefixes #603)
  4. [APPS-2961 Add: CI matrix running local execution against real Vite 5 to 8 installs]([APPS-2961] Add: CI matrix running local execution against real Vite 5 to 8 installs #606) (this PR)

Architecture

flowchart TD
    A["CI matrix leg (Vite N)"] -->|npm pack| B["datadog-vite-plugin.tgz"]
    A --> C["node --test local-execution.mjs"]
    C --> D["Temp project: vite@N + the tgz + @datadog/apps-backend, npm install"]
    B --> D
    D --> E["Real dev server with datadogVitePlugin, fake Datadog API via fetch"]
    E -->|"POST /__dd/executeAction"| G["Assertions on each response"]

Loading

• The suite runs the packed plugin's built entry by path, so it resolves vite from the temp project like a customer install.

Changes

6 changes across ci.yaml and local-execution.mjs

What changed	File
A matrix job, after the unit tests, that packs the plugin and runs the suite on Vite 5.4.21, 6.0.5, 6.4.3, 7.3.7 and 8.3.3.	ci.yaml
The job restores the unit tests' build cache without saving it, skips the install on a hit, checks out without credentials and has read-only permissions.	ci.yaml
An Apps local execution result job gives the matrix one stable check name.	ci.yaml
A node:test suite that installs the tgz with one exact Vite version in a temp project and drives a real dev server, with the plugin in test mode so it sends no telemetry.	local-execution.mjs
On Vite 6+ it checks fresh state per call, parallel calls, the permission model and prefixed logs; on Vite 5, kept state, one call at a time and the in-process guards.	local-execution.mjs
Both check $.Source through @datadog/apps-backend, at the fixture project's version.	local-execution.mjs

QA Instructions

• CI: Continuous Integration run for 676e2cef, whose Apps local execution (Vite …) jobs run the suite on each version.

yarn typecheck:all
# Expected: exit 0 ✅ VERIFIED
yarn lint
# Expected: exit 0 ✅ VERIFIED
yarn cli integrity
# Expected: exit 0, no changes ✅ VERIFIED
TMPDIR=/private/tmp yarn test:unit packages/plugins/apps
# Expected: Test Suites: 40 passed; Tests: 1041 passed ✅ VERIFIED

• The suite locally, after yarn build:all, from a checkout outside the temp dir:

cd packages/published/vite-plugin && npm pack --ignore-scripts --pack-destination /tmp/pack && cd -
mv /tmp/pack/datadog-vite-plugin-*.tgz /tmp/pack/datadog-vite-plugin.tgz
for v in 5.4.21 6.0.5 6.4.3 7.3.7 8.3.3; do
  DD_APPS_TEST_VITE_VERSION=$v DD_APPS_TEST_PLUGIN_TGZ=/tmp/pack/datadog-vite-plugin.tgz \
    node --test packages/tests/src/apps-vite-versions/local-execution.mjs
done
# Expected: each version passes 5 of 5 ✅ VERIFIED
node --test packages/tests/src/apps-vite-versions/local-execution.mjs
# Expected: fails with "Set DD_APPS_TEST_VITE_VERSION and DD_APPS_TEST_PLUGIN_TGZ." ✅ VERIFIED

Output for Vite 8.3.3

    ok 1 - module state is fresh per call in a child process, and kept in-process on Vite 5
    ok 2 - concurrent calls run in parallel in a child process, and one at a time on Vite 5
    ok 3 - the sandbox refuses writes outside the temp dir and subprocesses, and allows temp writes
    ok 4 - $.Source resolves through @datadog/apps-backend
    ok 5 - console output reaches the dev server's terminal, prefixed with its call in a child process
ok 1 - Local execution on Vite 8.3.3
# pass 5
# fail 0

Blast Radius

• No feature flag: CI and test code only; the plugin is unchanged.
• Low risk: five extra CI legs of about 2 minutes, which start after the unit tests and run beside the end-to-end shards.

Out of Scope / Follow-ups

4 items deferred

Item	Status	Next step
The suite loads the plugin's built entry by path, so the published exports map isn't covered.	Deliberate	npm pack keeps the repo's source-pointing exports; covering the map needs a pack that applies publishConfig.
The temp project's npm install isn't locked.	Accepted	The job has read-only permissions, no git credentials, skips install scripts and never writes the shared cache.
The matrix runs only after the unit tests pass, to reuse their build.	Accepted	Move the build into its own job if matrix signal is needed when unit tests fail.
The suite doesn't probe worker threads; the Jest suite does.	Deferred	Add a worker probe once its in-process guard message is pinned down.

Documentation

• Vite Environment API for runtimes
• Node.js test runner

🤖 Generated with Claude Code

[github-actions]

Live Debugger Runtime Benchmark

SDK-loaded dormant-probe runtime overhead, measured against an uninstrumented bundle in the same browser session.

Browser	Workload	Quality	Per-call overhead upper
chrome	Hot	clean	<= 2.54 ns
chrome	Tiny	clean	<= 0.01 ns
firefox	Hot	clean	<= 9.78 ns
firefox	Tiny	clean	<= 0.02 ns
safari	Hot	clean	<= 5.78 ns
safari	Tiny	clean	<= 0.42 ns

What do the Tiny and Hot workloads represent?

Browser Debugger SDK: 7.16.1 · built 2026-10-21 · S3 ETag 1426183a

Full diagnostics

browser  workload  quality  per-call overhead upper  overhead upper          95% CI         A/A diag        block CI  acf(1)   baseline  instrumented                         samples
-------  --------  -------  -----------------------  --------------  --------------  ---------------  --------------  ------  ---------  ------------  ------------------------------
chrome   Hot       clean                 <= 2.54 ns        <= 5.11%   2.51..2.54 ns  -0.03..-0.00 ns   2.51..2.54 ns   -0.06  52.355 ms     54.995 ms   102 (trim 20%, outliers 4.9%)
chrome   Tiny      clean                 <= 0.01 ns        <= 0.09%  -0.03..0.01 ns   -0.01..0.03 ns  -0.02..0.00 ns    0.01  54.965 ms     55.017 ms  102 (trim 20%, outliers 19.6%)
firefox  Hot       clean                 <= 9.78 ns       <= 38.85%   9.76..9.78 ns   -0.01..0.00 ns   9.76..9.78 ns    0.15  39.860 ms     55.340 ms   102 (trim 20%, outliers 7.8%)
firefox  Tiny      clean                 <= 0.02 ns        <= 0.16%   0.01..0.02 ns   -0.00..0.00 ns   0.01..0.02 ns   -0.17  50.320 ms     50.400 ms   102 (trim 20%, outliers 8.8%)
safari   Hot       clean                 <= 5.78 ns       <= 23.10%   5.77..5.78 ns   -0.00..0.00 ns   5.77..5.78 ns    0.03  44.940 ms     55.300 ms   102 (trim 20%, outliers 7.8%)
safari   Tiny      clean                 <= 0.42 ns       <= 11.15%   0.40..0.41 ns   -0.00..0.00 ns   0.40..0.42 ns    0.00  51.380 ms     56.970 ms   102 (trim 20%, outliers 2.0%)

Raw samples are in the live-debugger-runtime-bench-results artifact.

[tyffical]

@codex review

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-22T18:48:10.289942Z	676e2ce	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[Copilot]

🟢 Approval recommended

No blocking issues were identified in the CI configuration or test suite, and production code is unchanged.

No open issues 🎉

What changed in this PR

Adds real-install CI coverage for Apps local backend execution across Vite 5–8, complementing Jest coverage without changing production code.

Changes:

• Adds five Vite matrix legs and a stable aggregate status check.
• Adds integration tests for execution state, concurrency, sandbox restrictions, runtime context, and logs.

File	Description
packages/​tests/​src/​apps-vite-versions/​local-execution.mjs	Tests the packed plugin against fresh Vite installs.
.github/​workflows/​ci.yaml	Runs the version matrix using cached builds and read-only permissions.

_{Review effort: Balanced}

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: 1797661c4d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[datadog-prod-us1-6]

 

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 676e2ce | Docs | View more details | Give us feedback!}

[tyffical]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Bravo.

Reviewed commit: 676e2cef18

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[Copilot]

🟢 Approval recommended

No blocking issues were identified in these CI-only changes, and the linked CI run completed successfully.

No open issues 🎉

_{Review effort: Balanced}