feat(ui-automation): Add rs/1 runtime automation parity#416
Open
cameroncooke wants to merge 2 commits into
Open
feat(ui-automation): Add rs/1 runtime automation parity#416cameroncooke wants to merge 2 commits into
cameroncooke wants to merge 2 commits into
Conversation
Add batch execution, wait predicates, runtime snapshot refs, and screen-hash unchanged responses so agents can drive AXe with fewer process launches and less repeated snapshot output. Tighten action validation, stale-snapshot recovery, compact rendering, and fixture coverage so UI automation flows are easier for agents to use reliably. Co-Authored-By: OpenAI Codex <noreply@openai.com>
commit: |
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
Comment on lines
+345
to
+348
| snapshot_ui: { simulatorId: params.simulatorId }, | ||
| wait_for_ui: { simulatorId: params.simulatorId, predicate: 'settled' }, | ||
| }; | ||
| } |
There was a problem hiding this comment.
Bug: The wait_for_uiLogic function unconditionally sets nextStepParams, even when the operation fails, which is inconsistent with other tools that only suggest next steps on success.
Severity: MEDIUM
Suggested Fix
The assignment to ctx.nextStepParams in wait_for_uiLogic should be wrapped in a conditional check to ensure it only executes if the operation was successful. For example: if (!result.didError) { ctx.nextStepParams = { ... }; }.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: src/mcp/tools/ui-automation/wait_for_ui.ts#L345-L348
Potential issue: In `wait_for_ui.ts`, the `nextStepParams` context variable is
unconditionally set, regardless of whether the wait operation succeeded or failed (e.g.,
timed out). This behavior is inconsistent with the pattern in other tools like
`list_sims.ts` and `record_sim_video.ts`, where `nextStepParams` is only populated upon
successful completion. By suggesting next steps after a failure, the tool provides
misleading guidance to the user or agent, suggesting that the next logical action is
appropriate when the preceding one actually failed and needs to be addressed.
Did we get this right? 👍 / 👎 to inform future reviews.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Bugbot Autofix prepared fixes for both issues found in the latest run.
- ✅ Fixed: Validation artifacts with local paths committed to repo
- Removed entire out.nosync/ directory from git tracking and added it to .gitignore to prevent future commits.
- ✅ Fixed: Plan document embeds developer-local filesystem paths
- Replaced all machine-specific absolute paths with portable references or removed them entirely from the plan document.
Or push these changes by commenting:
@cursor push 07a7505ffa
Preview (07a7505ffa)
diff --git a/.gitignore b/.gitignore
--- a/.gitignore
+++ b/.gitignore
@@ -116,3 +116,6 @@
.derivedData
/.pr-learning
/repros
+
+# Validation artifacts
+out.nosync/
diff --git a/docs/plans/ui-automation-agent-optimization-2026-05-13.md b/docs/plans/ui-automation-agent-optimization-2026-05-13.md
--- a/docs/plans/ui-automation-agent-optimization-2026-05-13.md
+++ b/docs/plans/ui-automation-agent-optimization-2026-05-13.md
@@ -12,7 +12,7 @@
- `wait_for_ui` polls snapshots and records the latest usable runtime snapshot while waiting (`src/mcp/tools/ui-automation/wait_for_ui.ts`).
- Prior art: `04b055210b111c8a2d70bdf5c19ca4c6b0d2a479` added RS/1 runtime automation parity, including batch execution, wait predicates, runtime refs, and screen-hash unchanged responses.
-Transcript evidence from `/Users/cameroncooke/Desktop/out.nosync`:
+Transcript evidence from validation run:
- Settings toggles were handled as repeated `tap -> snapshot_ui -> tap` loops (`0058` through `0087`) instead of one snapshot plus one batch.
- Batch syntax was guessed and failed (`tap e7`, `tap --element-ref e7`, `tap-target e21`) before the agent fell back to coordinates (`0281` through `0286`, `0312` through `0327`).
- Scroll and sheet expansion produced repeated gesture/swipe/screenshot/snapshot loops (`0110` through `0144`).
@@ -83,9 +83,9 @@
Use Claude Code, not Codex, for validation runs. Configure Claude Code to use the local source-built XcodeBuildMCP server, then repeat the Weather-app task from `0001_user_message.md`.
Reference commands and artifacts:
-- Existing parser: `/Volumes/Developer/parse_claude_conversation.py`
-- Example source conversation: `/Users/cameroncooke/.claude/projects/-Volumes-Developer-XcodeBuildMCP-example-projects-Weather/b3d6cae2-e274-4a72-92ea-25eaf4f6fcff.jsonl`
-- Export command example: `python3 /Volumes/Developer/parse_claude_conversation.py <claude-jsonl>`
+- Existing parser: `parse_claude_conversation.py`
+- Example source conversation: `.claude/projects/<project-path>/<session-id>.jsonl`
+- Export command example: `python3 parse_claude_conversation.py <claude-jsonl>`
Acceptance signals:
- The settings-toggle sequence is reduced to one `snapshot_ui`, one structured `batch`, and at most one verification call.
@@ -106,11 +106,11 @@
## Validation Results
-Validation artifacts were written under `/Volumes/Developer/XcodeBuildMCP/out.nosync/validation-ui-automation-20260513-215522` with timestamped names. Prior Claude transcripts/exports were not deleted or overwritten.
+Validation artifacts were written under `out.nosync/validation-ui-automation-20260513-215522` with timestamped names. Prior Claude transcripts/exports were not deleted or overwritten.
### Automated checks
-All checks below were run with `XCODEBUILDMCP_AXE_SOURCE_PATH=/Volumes/Developer/AXe`:
+All checks below were run with `XCODEBUILDMCP_AXE_SOURCE_PATH` pointing to the AXe source build:
- Focused UI automation/config/factory tests: `npx vitest run src/mcp/tools/ui-automation/__tests__/batch.test.ts src/mcp/tools/ui-automation/__tests__/runtime-snapshot.test.ts src/mcp/tools/ui-automation/__tests__/snapshot_ui.test.ts src/mcp/tools/ui-automation/__tests__/tap.test.ts src/mcp/tools/ui-automation/__tests__/wait_for_ui.test.ts src/utils/__tests__/axe-helpers.test.ts src/utils/__tests__/config-store.test.ts src/utils/__tests__/project-config.test.ts src/utils/__tests__/session-aware-tool-factory.test.ts src/utils/responses/__tests__/next-steps-renderer.test.ts` passed: 10 files, 192 tests (`focused-vitest-20260513-215522.log`).
- `npm run lint` passed (`lint-20260513-215539.log`).
@@ -118,7 +118,7 @@
- `npm run typecheck` passed (`typecheck-20260513-215539.log`).
- `npm run build` passed (`build-20260513-215553.log`).
- `npm run test` passed: 186 files, 2049 tests (`test-20260513-215553.log`).
-- Post-review fixes were applied for switch batch delay handling and selector-scoped `gone` text waits. Final checks after those fixes passed: `npm run lint`, `npm run format:check`, `npm run typecheck`, `npm run build`, and `npm run test` (186 files, 2052 tests). The final full-test output was preserved at `/tmp/xcodebuildmcp-final-npm-test-20260513.txt`.
+- Post-review fixes were applied for switch batch delay handling and selector-scoped `gone` text waits. Final checks after those fixes passed: `npm run lint`, `npm run format:check`, `npm run typecheck`, `npm run build`, and `npm run test` (186 files, 2052 tests).
### AXe source-build proof
@@ -127,18 +127,18 @@
```json
{
"resolved": {
- "path": "/Volumes/Developer/AXe/.build/release/axe",
+ "path": "<axe-source-path>/.build/release/axe",
"source": "source"
},
"bundledEnvironment": {}
}-The resolved binary reported version staging-main-31-510d4df-dirty, proving the validation used the /Volumes/Developer/AXe source build rather than bundled or PATH fallback.
+The resolved binary reported version staging-main-31-510d4df-dirty, proving the validation used a local AXe source build rather than bundled or PATH fallback.
Claude Code E2E
-Claude Code ran the full original 18-step Weather/Safari task from /Users/cameroncooke/Desktop/out.nosync/0001_user_message.md against the local source-built XcodeBuildMCP server:
+Claude Code ran the full original 18-step Weather/Safari task against the local source-built XcodeBuildMCP server:
- MCP config:
claude-mcp-config-20260513-215817.json - Prompt:
claude-weather-safari-prompt-20260513-215817.md
@@ -153,9 +153,9 @@
{
"command": "node",
- "args": ["/Volumes/Developer/XcodeBuildMCP/build/cli.js", "mcp"],
+ "args": ["<xcodebuildmcp-path>/build/cli.js", "mcp"],
"env": {
- "XCODEBUILDMCP_AXE_SOURCE_PATH": "/Volumes/Developer/AXe",
+ "XCODEBUILDMCP_AXE_SOURCE_PATH": "<axe-source-path>",
"XCODEBUILDMCP_SENTRY_DISABLED": "1"
}
}
@@ -181,8 +181,5 @@
- Safari WebView contents, cookie/sign-in UI, and BBC in-page links were not exposed as tappable RS/1 targets, so Claude used the URL bar for Sport, Premier League, tables, and Brighton. This reached the equivalent end state but was not a real row/link click.
## References
-- Transcript folder: `/Users/cameroncooke/Desktop/out.nosync`
-- Validation artifact folder: `/Volumes/Developer/XcodeBuildMCP/out.nosync/validation-ui-automation-20260513-215522`
-- Parser: `/Volumes/Developer/parse_claude_conversation.py`
-- AXe workspace: `/Volumes/Developer/AXe`
+- Validation artifact folder: `out.nosync/validation-ui-automation-20260513-215522`
- Prior RS/1 commit: `04b055210b111c8a2d70bdf5c19ca4c6b0d2a479`
diff --git a/out.nosync/validation-ui-automation-20260513-215522/churn-analysis-94c0a294-37b0-453f-9ac6-774095a4ace0-20260513-221250.json b/out.nosync/validation-ui-automation-20260513-215522/churn-analysis-94c0a294-37b0-453f-9ac6-774095a4ace0-20260513-221250.json
deleted file mode 100644
--- a/out.nosync/validation-ui-automation-20260513-215522/churn-analysis-94c0a294-37b0-453f-9ac6-774095a4ace0-20260513-221250.json
+++ /dev/null
@@ -1,1008 +1,0 @@
-{
- "jsonl": "/Volumes/Developer/XcodeBuildMCP/out.nosync/validation-ui-automation-20260513-215522/claude-session-94c0a294-37b0-453f-9ac6-774095a4ace0-20260513-215817.jsonl",
- "total_lines": 425,
- "total_tool_calls": 138,
- "all_tool_counts": {
- "ToolSearch": 3,
- "mcp__xcodebuildmcp-dev__session_show_defaults": 1,
- "mcp__xcodebuildmcp-dev__build_run_sim": 2,
- "mcp__xcodebuildmcp-dev__snapshot_ui": 21,
- "mcp__xcodebuildmcp-dev__wait_for_ui": 23,
- "mcp__xcodebuildmcp-dev__screenshot": 19,
- "Bash": 2,
- "TodoWrite": 14,
- "mcp__xcodebuildmcp-dev__stop_app_sim": 1,
- "mcp__xcodebuildmcp-dev__launch_app_sim": 1,
- "mcp__xcodebuildmcp-dev__swipe": 15,
- "mcp__xcodebuildmcp-dev__tap": 16,
- "mcp__xcodebuildmcp-dev__batch": 1,
- "mcp__xcodebuildmcp-dev__gesture": 2,
- "Grep": 3,
- "Read": 2,
- "mcp__xcodebuildmcp-dev__type_text": 6,
- "mcp__xcodebuildmcp-dev__button": 1,
- "mcp__xcodebuildmcp-dev__key_press": 5
- },
- "xcodebuildmcp_tool_counts": {
- "session_show_defaults": 1,
- "build_run_sim": 2,
- "snapshot_ui": 21,
- "wait_for_ui": 23,
- "screenshot": 19,
- "stop_app_sim": 1,
- "launch_app_sim": 1,
- "swipe": 15,
- "tap": 16,
- "batch": 1,
- "gesture": 2,
- "type_text": 6,
- "button": 1,
- "key_press": 5
- },
- "batch_calls": [
- {
- "ordinal": 33,
- "timestamp": "2026-05-13T21:01:15.625Z",
- "id": "toolu_01FsZYoJ6ddwjVLWKMwYoyU6",
- "kind": "structured",
- "step_count": 7,
- "steps": [
- {
- "action": "tap",
- "elementRef": "e130",
- "postDelay": 0.25
- },
- {
- "action": "tap",
- "elementRef": "e133",
- "postDelay": 0.25
- },
- {
- "action": "tap",
- "elementRef": "e137",
- "postDelay": 0.25
- },
- {
- "action": "tap",
- "elementRef": "e140",
- "postDelay": 0.25
- },
- {
- "action": "tap",
- "elementRef": "e142",
- "postDelay": 0.25
- },
- {
- "action": "tap",
- "elementRef": "e145",
- "postDelay": 0.25
- },
- {
- "action": "tap",
- "elementRef": "e148",
- "postDelay": 0.25
- }
- ],
- "result": {
- "line": 112,
- "timestamp": "2026-05-13T21:01:20.872Z",
- "status": null,
- "error_code": null,
- "seq": null,
- "capture_count": null,
- "structured": null,
- "content": "{\"schema\":\"xcodebuildmcp.output.ui-action-result\",\"schemaVersion\":\"2\",\"didError\":false,\"error\":null,\"data\":{\"summary\":{\"status\":\"SUCCEEDED\"},\"action\":{\"type\":\"batch\",\"stepCount\":7},\"artifacts\":{\"simulatorId\":\"5213C8D8-61D0-4CD7-B468-B463C6206C7D\"}}}"
- }
- }
- ],
- "raw_string_batch_attempts": [],
- "errors": [],
- "settings_window_calls": [
- {
- "ordinal": 20,
- "timestamp": "2026-05-13T21:00:04.466Z",
- "name": "mcp__xcodebuildmcp-dev__swipe",
- "input": {
- "withinElementRef": "e7",
- "direction": "up",
- "distance": 800
- },
- "result": {
- "timestamp": "2026-05-13T21:00:06.036Z",
- "status": null,
- "error_code": null,
- "seq": null,
- "capture_count": null
- }
- },
- {
- "ordinal": 21,
- "timestamp": "2026-05-13T21:00:08.085Z",
- "name": "mcp__xcodebuildmcp-dev__snapshot_ui",
- "input": {},
- "result": {
- "timestamp": "2026-05-13T21:00:08.545Z",
- "status": null,
- "error_code": null,
- "seq": null,
- "capture_count": null
- }
- },
- {
- "ordinal": 22,
- "timestamp": "2026-05-13T21:00:11.517Z",
- "name": "mcp__xcodebuildmcp-dev__screenshot",
- "input": {
- "returnFormat": "base64"
- },
- "result": {
- "timestamp": "2026-05-13T21:00:13.199Z",
- "status": null,
- "error_code": null,
- "seq": null,
- "capture_count": null
- }
- },
- {
- "ordinal": 23,
- "timestamp": "2026-05-13T21:00:20.754Z",
- "name": "TodoWrite",
- "input": {
- "todos": [
- {
- "content": "Build and launch Weather app with mock data",
- "status": "completed",
- "activeForm": "Building and launching Weather app with mock data"
- },
- {
- "content": "Scroll to bottom of main screen",
- "status": "completed",
- "activeForm": "Scrolling to bottom of main screen"
- },
- {
- "content": "Tap settings button and reveal all options",
- "status": "in_progress",
- "activeForm": "Opening settings sheet"
- },
- {
- "content": "Invert all settings toggles",
- "status": "pending",
- "activeForm": "Inverting settings toggles"
- },
- {
- "content": "Close settings sheet",
- "status": "pending",
- "activeForm": "Closing settings sheet"
- },
- {
- "content": "Scroll to precipitation widget and tap",
- "status": "pending",
- "activeForm": "Tapping precipitation widget"
- },
- {
- "content": "Close precipitation widget",
- "status": "pending",
- "activeForm": "Closing precipitation widget"
- },
- {
- "content": "Open Location screen",
- "status": "pending",
- "activeForm": "Opening Location screen"
- },
- {
- "content": "Scroll location sheet to reveal all",
- "status": "pending",
- "activeForm": "Revealing all locations"
- },
- {
- "content": "Tap Portland location",
- "status": "pending",
- "activeForm": "Tapping Portland"
- },
- {
- "content": "Tap locations button again",
- "status": "pending",
- "activeForm": "Reopening locations"
- },
- {
- "content": "Search for a location",
- "status": "pending",
- "activeForm": "Searching for a location"
- },
- {
- "content": "Press hardware home button",
- "status": "pending",
- "activeForm": "Pressing home button"
- },
- {
- "content": "Open Safari",
- "status": "pending",
- "activeForm": "Opening Safari"
- },
- {
- "content": "Search for BBC.co.uk",
- "status": "pending",
- "activeForm": "Searching BBC.co.uk"
- },
- {
- "content": "Go to sport page",
- "status": "pending",
- "activeForm": "Navigating to sport page"
- },
- {
- "content": "Go to Premier League page",
- "status": "pending",
- "activeForm": "Navigating to Premier League"
- },
- {
- "content": "Go to tables",
- "status": "pending",
- "activeForm": "Going to tables"
- },
- {
- "content": "Click Brighton",
- "status": "pending",
- "activeForm": "Clicking Brighton"
- }
- ]
- },
- "result": {
- "timestamp": "2026-05-13T21:00:20.762Z",
- "status": null,
- "error_code": null,
- "seq": null,
- "capture_count": null
- }
- },
- {
- "ordinal": 24,
- "timestamp": "2026-05-13T21:00:23.387Z",
- "name": "mcp__xcodebuildmcp-dev__tap",
- "input": {
- "elementRef": "e113"
- },
- "result": {
- "timestamp": "2026-05-13T21:00:24.176Z",
- "status": null,
- "error_code": null,
- "seq": null,
- "capture_count": null
- }
- },
- {
- "ordinal": 25,
- "timestamp": "2026-05-13T21:00:26.779Z",
- "name": "mcp__xcodebuildmcp-dev__wait_for_ui",
- "input": {
- "predicate": "settled",
- "settledDurationMs": 500,
- "timeoutMs": 5000
- },
- "result": {
- "timestamp": "2026-05-13T21:00:28.153Z",
- "status": null,
- "error_code": null,
- "seq": null,
- "capture_count": null
- }
- },
- {
- "ordinal": 26,
- "timestamp": "2026-05-13T21:00:33.949Z",
- "name": "mcp__xcodebuildmcp-dev__swipe",
- "input": {
- "withinElementRef": "e123",
- "direction": "up",
- "distance": 400
- },
- "result": {
- "timestamp": "2026-05-13T21:00:35.629Z",
- "status": null,
- "error_code": null,
- "seq": null,
- "capture_count": null
- }
- },
- {
- "ordinal": 27,
- "timestamp": "2026-05-13T21:00:37.754Z",
- "name": "mcp__xcodebuildmcp-dev__snapshot_ui",
- "input": {},
- "result": {
- "timestamp": "2026-05-13T21:00:38.303Z",
- "status": null,
- "error_code": null,
- "seq": null,
- "capture_count": null
- }
- },
- {
- "ordinal": 28,
- "timestamp": "2026-05-13T21:00:42.712Z",
- "name": "mcp__xcodebuildmcp-dev__screenshot",
- "input": {
- "returnFormat": "base64"
- },
- "result": {
- "timestamp": "2026-05-13T21:00:43.315Z",
- "status": null,
- "error_code": null,
- "seq": null,
- "capture_count": null
- }
- },
- {
- "ordinal": 29,
- "timestamp": "2026-05-13T21:00:49.819Z",
- "name": "mcp__xcodebuildmcp-dev__swipe",
- "input": {
- "withinElementRef": "e115",
- "direction": "up",
- "distance": 500
- },
- "result": {
- "timestamp": "2026-05-13T21:00:51.550Z",
- "status": null,
- "error_code": null,
- "seq": null,
- "capture_count": null
- }
- },
- {
- "ordinal": 30,
- "timestamp": "2026-05-13T21:00:54.206Z",
- "name": "mcp__xcodebuildmcp-dev__screenshot",
- "input": {
- "returnFormat": "base64"
- },
- "result": {
- "timestamp": "2026-05-13T21:00:54.821Z",
- "status": null,
- "error_code": null,
- "seq": null,
- "capture_count": null
- }
- },
- {
- "ordinal": 31,
- "timestamp": "2026-05-13T21:01:01.034Z",
- "name": "mcp__xcodebuildmcp-dev__snapshot_ui",
- "input": {},
- "result": {
- "timestamp": "2026-05-13T21:01:01.575Z",
- "status": null,
- "error_code": null,
- "seq": null,
- "capture_count": null
- }
- },
- {
- "ordinal": 32,
- "timestamp": "2026-05-13T21:01:11.819Z",
- "name": "TodoWrite",
- "input": {
- "todos": [
- {
- "content": "Build and launch Weather app with mock data",
- "status": "completed",
- "activeForm": "Building and launching Weather app with mock data"
- },
- {
- "content": "Scroll to bottom of main screen",
- "status": "completed",
- "activeForm": "Scrolling to bottom of main screen"
- },
- {
- "content": "Tap settings button and reveal all options",
- "status": "completed",
- "activeForm": "Opening settings sheet"
- },
- {
- "content": "Invert all settings toggles",
- "status": "in_progress",
- "activeForm": "Inverting settings toggles"
- },
- {
- "content": "Close settings sheet",
- "status": "pending",
- "activeForm": "Closing settings sheet"
- },
- {
- "content": "Scroll to precipitation widget and tap",
- "status": "pending",
- "activeForm": "Tapping precipitation widget"
- },
- {
- "content": "Close precipitation widget",
- "status": "pending",
- "activeForm": "Closing precipitation widget"
- },
- {
- "content": "Open Location screen",
- "status": "pending",
- "activeForm": "Opening Location screen"
- },
- {
- "content": "Scroll location sheet to reveal all",
- "status": "pending",
- "activeForm": "Revealing all locations"
- },
- {
- "content": "Tap Portland location",
- "status": "pending",
- "activeForm": "Tapping Portland"
- },
- {
- "content": "Tap locations button again",
- "status": "pending",
- "activeForm": "Reopening locations"
- },
- {
- "content": "Search for a location",
- "status": "pending",
- "activeForm": "Searching for a location"
- },
- {
- "content": "Press hardware home button",
- "status": "pending",
- "activeForm": "Pressing home button"
- },
- {
- "content": "Open Safari",
- "status": "pending",
- "activeForm": "Opening Safari"
- },
- {
- "content": "Search for BBC.co.uk",
- "status": "pending",
- "activeForm": "Searching BBC.co.uk"
- },
- {
- "content": "Go to sport page",
- "status": "pending",
- "activeForm": "Navigating to sport page"
- },
- {
- "content": "Go to Premier League page",
- "status": "pending",
- "activeForm": "Navigating to Premier League"
- },
- {
- "content": "Go to tables",
- "status": "pending",
- "activeForm": "Going to tables"
- },
- {
- "content": "Click Brighton",
- "status": "pending",
- "activeForm": "Clicking Brighton"
- }
- ]
- },
- "result": {
- "timestamp": "2026-05-13T21:01:11.822Z",
- "status": null,
- "error_code": null,
- "seq": null,
- "capture_count": null
- }
- },
- {
- "ordinal": 33,
- "timestamp": "2026-05-13T21:01:15.625Z",
- "name": "mcp__xcodebuildmcp-dev__batch",
- "input": {
- "steps": [
- {
- "action": "tap",
- "elementRef": "e130",
- "postDelay": 0.25
- },
- {
- "action": "tap",
- "elementRef": "e133",
- "postDelay": 0.25
- },
- {
- "action": "tap",
- "elementRef": "e137",
- "postDelay": 0.25
- },
- {
- "action": "tap",
- "elementRef": "e140",
- "postDelay": 0.25
- },
- {
- "action": "tap",
- "elementRef": "e142",
- "postDelay": 0.25
- },
- {
- "action": "tap",
- "elementRef": "e145",
- "postDelay": 0.25
- },
- {
- "action": "tap",
- "elementRef": "e148",
- "postDelay": 0.25
- }
- ]
- },
- "result": {
- "timestamp": "2026-05-13T21:01:20.872Z",
- "status": null,
- "error_code": null,
- "seq": null,
- "capture_count": null
- }
- },
- {
- "ordinal": 34,
- "timestamp": "2026-05-13T21:01:23.163Z",
- "name": "mcp__xcodebuildmcp-dev__screenshot",
- "input": {
- "returnFormat": "base64"
- },
- "result": {
- "timestamp": "2026-05-13T21:01:23.739Z",
- "status": null,
- "error_code": null,
- "seq": null,
- "capture_count": null
- }
- },
- {
- "ordinal": 35,
- "timestamp": "2026-05-13T21:01:27.506Z",
- "name": "mcp__xcodebuildmcp-dev__snapshot_ui",
- "input": {},
- "result": {
- "timestamp": "2026-05-13T21:01:28.031Z",
- "status": null,
- "error_code": null,
- "seq": null,
- "capture_count": null
- }
- },
- {
- "ordinal": 36,
- "timestamp": "2026-05-13T21:01:31.879Z",
- "name": "mcp__xcodebuildmcp-dev__tap",
- "input": {
- "elementRef": "e126"
- },
- "result": {
- "timestamp": "2026-05-13T21:01:32.759Z",
- "status": null,
- "error_code": null,
- "seq": null,
- "capture_count": null
- }
- },
- {
- "ordinal": 37,
- "timestamp": "2026-05-13T21:01:35.902Z",
- "name": "mcp__xcodebuildmcp-dev__wait_for_ui",
- "input": {
- "predicate": "gone",
- "text": "Settings",
- "timeoutMs": 3000
- },
- "result": {
- "timestamp": "2026-05-13T21:01:39.013Z",
- "status": null,
- "error_code": null,
- "seq": null,
- "capture_count": null
- }
- },
- {
- "ordinal": 38,
- "timestamp": "2026-05-13T21:01:49.208Z",
- "name": "TodoWrite",
- "input": {
- "todos": [
- {
- "content": "Build and launch Weather app with mock data",
- "status": "completed",
- "activeForm": "Building and launching Weather app with mock data"
- },
- {
- "content": "Scroll to bottom of main screen",
- "status": "completed",
- "activeForm": "Scrolling to bottom of main screen"
- },
- {
- "content": "Tap settings button and reveal all options",
- "status": "completed",
- "activeForm": "Opening settings sheet"
- },
- {
- "content": "Invert all settings toggles",
- "status": "completed",
- "activeForm": "Inverting settings toggles"
- },
- {
- "content": "Close settings sheet",
- "status": "completed",
- "activeForm": "Closing settings sheet"
- },
- {
- "content": "Scroll to precipitation widget and tap",
- "status": "in_progress",
- "activeForm": "Tapping precipitation widget"
- },
- {
- "content": "Close precipitation widget",
- "status": "pending",
- "activeForm": "Closing precipitation widget"
- },
- {
- "content": "Open Location screen",
- "status": "pending",
- "activeForm": "Opening Location screen"
- },
- {
- "content": "Scroll location sheet to reveal all",
- "status": "pending",
- "activeForm": "Revealing all locations"
- },
- {
- "content": "Tap Portland location",
- "status": "pending",
- "activeForm": "Tapping Portland"
- },
- {
- "content": "Tap locations button again",
- "status": "pending",
- "activeForm": "Reopening locations"
- },
- {
- "content": "Search for a location",
- "status": "pending",
- "activeForm": "Searching for a location"
- },
- {
- "content": "Press hardware home button",
- "status": "pending",
- "activeForm": "Pressing home button"
- },
- {
- "content": "Open Safari",
- "status": "pending",
- "activeForm": "Opening Safari"
- },
- {
- "content": "Search for BBC.co.uk",
- "status": "pending",
- "activeForm": "Searching BBC.co.uk"
- },
- {
- "content": "Go to sport page",
- "status": "pending",
- "activeForm": "Navigating to sport page"
- },
- {
- "content": "Go to Premier League page",
- "status": "pending",
- "activeForm": "Navigating to Premier League"
- },
... diff truncated: showing 800 of 12059 linesYou can send follow-ups to the cloud agent here.
Reviewed by Cursor Bugbot for commit c0a5d33. Configure here.
| } | ||
| ], | ||
| "result_summary": null | ||
| } No newline at end of file |
Contributor
There was a problem hiding this comment.
Validation artifacts with local paths committed to repo
Medium Severity
The entire out.nosync/ directory — thousands of lines of validation session JSON, churn-analysis logs, parsed Claude transcripts, MCP config snapshots, and run metadata — is committed to the repository. The .nosync naming convention signals these files are not meant to be tracked. They contain machine-specific local paths (e.g. /Users/cameroncooke/, /Volumes/Developer/AXe), session IDs, and full tool-call traces that add significant repository bloat and leak local development environment details.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit c0a5d33. Configure here.
| - Validation artifact folder: `/Volumes/Developer/XcodeBuildMCP/out.nosync/validation-ui-automation-20260513-215522` | ||
| - Parser: `/Volumes/Developer/parse_claude_conversation.py` | ||
| - AXe workspace: `/Volumes/Developer/AXe` | ||
| - Prior RS/1 commit: `04b055210b111c8a2d70bdf5c19ca4c6b0d2a479` |
Contributor
There was a problem hiding this comment.
Plan document embeds developer-local filesystem paths
Low Severity
The References section hardcodes machine-specific absolute paths like /Users/cameroncooke/Desktop/out.nosync, /Volumes/Developer/parse_claude_conversation.py, and /Volumes/Developer/AXe. These are meaningless to other contributors and leak local development environment layout. Line 15 also references a local transcript path in the Background section.
Reviewed by Cursor Bugbot for commit c0a5d33. Configure here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an rs/1-oriented runtime UI automation contract on top of AXe. This introduces runtime snapshot refs with screen hashes, batch execution, wait predicates, stricter actionable-target handling, compact structured output schemas, and updated snapshots so agents can drive simulator UI with fewer unnecessary calls.
The branch was validated locally with
npm run format:check,npm run lint,npm run typecheck,npm run build, andnpm run testwith 186 files and 2025 tests passing. During implementation the UI automation flows were also smoke-tested against Settings, Safari, Contacts, and AXe Playground components.