benstull/vscode-cowriting-plugin
1
0
Fork 0
Code Issues 9 Pull Requests Actions Packages Projects Releases Wiki Activity

Host E2E undo tests fail (executeCommand undo non-functional in .vscode-test) — blocks #40 + makes main E2E red #54

New Issue
Open
opened 2026-06-13 16:06:30 +00:00 by benstull · 1 comment
benstull commented 2026-06-13 16:06:30 +00:00
Owner
Copy Link
Copy Source

Summary

The host E2E tests that drive vscode.commands.executeCommand("undo") —
undoMarks.test.ts (#38) and s40Provenance.test.ts (#40) — fail intermittently,
and are currently failing deterministically in the local .vscode-test
harness: the undo command does not restore the text, so the assertions
(undo restored …) fail before any product behavior is checked. This makes
main's E2E red in the affected environment and blocks verifying #40.

Diagnostic (session 0048 — conclusive)

An isolated probe (open doc → showTextDocument, asserted activeTextEditor === doc → delete via the active editor's editor.edit() → executeCommand("undo"))
left the deletion standing ("alpha charlie", bravo not restored). So it is:

  • NOT stale user-data — moving .vscode-test/user-data aside to force a fresh
    one did not help.
  • NOT the edit mechanism — both workspace.applyEdit and editor.edit fail.
  • NOT focus — the target doc was the verified active editor.

The unchanged #38 test fails identically on clean main, so it is not a
product regression — it is the undo command itself being non-functional in this
headless test instance (likely a VS Code-version / test-runner behavior).

Impact

  • main E2E is red in any environment where this reproduces.
  • #40 (restore exact attribution on undo/redo) cannot be verified/merged until
    undo works in E2E — s40-undo-provenance is implemented + unit-green but parked.

Likely fixes (to investigate)

  • Pin / change the VS Code test version in @vscode/test-electron (runTest.ts).
  • Trigger undo differently, or run the undo suites in a dedicated, freshly-spawned
    instance.
  • If undo is genuinely unsupported headless here, gate the undo-dependent E2E on a
    preflight undo-capability probe (skip-with-loud-warning when unavailable) so the
    suite isn't falsely red — but only as a backstop, never silently.

Source

Surfaced repeatedly (flake first noted session 0037); made deterministic and fully
diagnosed in session 0048 while implementing #40.

## Summary The host E2E tests that drive `vscode.commands.executeCommand("undo")` — `undoMarks.test.ts` (#38) and `s40Provenance.test.ts` (#40) — fail intermittently, and are **currently failing deterministically** in the local `.vscode-test` harness: the `undo` command does **not restore** the text, so the assertions (`undo restored …`) fail before any product behavior is checked. This makes `main`'s E2E red in the affected environment and **blocks verifying #40**. ## Diagnostic (session 0048 — conclusive) An isolated probe (open doc → `showTextDocument`, asserted `activeTextEditor === doc` → delete via the active editor's `editor.edit()` → `executeCommand("undo")`) left the deletion standing (`"alpha charlie"`, bravo not restored). So it is: - **NOT stale user-data** — moving `.vscode-test/user-data` aside to force a fresh one did not help. - **NOT the edit mechanism** — both `workspace.applyEdit` and `editor.edit` fail. - **NOT focus** — the target doc was the verified active editor. The unchanged #38 test fails identically on **clean `main`**, so it is not a product regression — it is the `undo` command itself being non-functional in this headless test instance (likely a VS Code-version / test-runner behavior). ## Impact - `main` E2E is red in any environment where this reproduces. - #40 (restore exact attribution on undo/redo) cannot be verified/merged until undo works in E2E — `s40-undo-provenance` is implemented + unit-green but parked. ## Likely fixes (to investigate) - Pin / change the VS Code test version in `@vscode/test-electron` (`runTest.ts`). - Trigger undo differently, or run the undo suites in a dedicated, freshly-spawned instance. - If undo is genuinely unsupported headless here, gate the undo-dependent E2E on a preflight undo-capability probe (skip-with-loud-warning when unavailable) so the suite isn't falsely red — but only as a backstop, never silently. ## Source Surfaced repeatedly (flake first noted session 0037); made deterministic and fully diagnosed in session 0048 while implementing #40.
benstull added the type/taskpriority/P2 labels 2026-06-13 16:06:30 +00:00
benstull commented 2026-06-13 16:17:15 +00:00
Author
Owner
Copy Link
Copy Source

Resilience fix shipped (#55, merged to main): a runtime preflight probe (undoCapable.ts) gates the #38 undo suite via suiteSetup — runs where undo works, skips with a loud warning (shows pending) where it doesn't. main E2E is no longer false-red in broken-undo environments. Verified locally: #38 suite skips (1 pending), both E2E passes exit 0.

Keeping this open for the underlying need: make undo actually work in the harness (or confirm it's an irreducible headless limitation) so #40's undo behavior gets real CI coverage. When s40-undo-provenance lands, apply the same suiteSetup guard to its suite. Also noticed: pretest:e2e should clean out/ (stale compiled *.test.js from other branches get run by the glob).

Resilience fix shipped (#55, merged to main): a runtime preflight probe (`undoCapable.ts`) gates the #38 undo suite via `suiteSetup` — runs where `undo` works, skips with a loud warning (shows `pending`) where it doesn't. **`main` E2E is no longer false-red** in broken-undo environments. Verified locally: #38 suite skips (1 pending), both E2E passes exit 0. Keeping this open for the underlying need: make `undo` actually work in the harness (or confirm it's an irreducible headless limitation) so #40's undo behavior gets real CI coverage. When `s40-undo-provenance` lands, apply the same `suiteSetup` guard to its suite. Also noticed: `pretest:e2e` should clean `out/` (stale compiled `*.test.js` from other branches get run by the glob).
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
priority/P0
Priority: highest — drop everything
 priority/P1
Priority: high
 priority/P2
Priority: medium
 priority/P3
Priority: low / someday
 type/bug
A defect in shipped behavior
 type/epic
Type: a large body of work spanning many features
 type/feature
Type: a deliverable capability under an epic
 type/story
Type: a user story — the smallest valuable increment
 type/task
Type: a concrete unit of work under a story
No Label priority/P2 type/task
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
benstull
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: benstull/vscode-cowriting-plugin#54
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?