ai-agent-book 精选快照（<2MB 代码与文档，来自 github.com/bojieli/ai-agent-book）
Build latest book artifacts / build (push) Canceled after 0s
dependency resolution / resolve (3.11) (push) Canceled after 0s
dependency resolution / resolve (3.13) (push) Canceled after 0s
deploy-pages / build (push) Canceled after 0s
deploy-pages / deploy (push) Canceled after 0s
i18n consistency check / check (push) Canceled after 0s
provider adoption tests / test (chapter2/context-compression) (push) Canceled after 0s
provider adoption tests / test (chapter2/prompt-injection) (push) Canceled after 0s
provider adoption tests / test (chapter2/system-hint) (push) Canceled after 0s
provider adoption tests / test (chapter3/log-sanitization) (push) Canceled after 0s
web-search-agent tests / test (push) Canceled after 0s
web-search-agent tests / agentbook (push) Canceled after 0s
Build latest book artifacts / build (push) Canceled after 0s
dependency resolution / resolve (3.11) (push) Canceled after 0s
dependency resolution / resolve (3.13) (push) Canceled after 0s
deploy-pages / build (push) Canceled after 0s
deploy-pages / deploy (push) Canceled after 0s
i18n consistency check / check (push) Canceled after 0s
provider adoption tests / test (chapter2/context-compression) (push) Canceled after 0s
provider adoption tests / test (chapter2/prompt-injection) (push) Canceled after 0s
provider adoption tests / test (chapter2/system-hint) (push) Canceled after 0s
provider adoption tests / test (chapter3/log-sanitization) (push) Canceled after 0s
web-search-agent tests / test (push) Canceled after 0s
web-search-agent tests / agentbook (push) Canceled after 0s
This commit is contained in:
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,204 @@
|
||||
# Experiment 7-12 AndroidWorld iteration report
|
||||
|
||||
- Run ID: `exp7-12-20260804T045559Z`
|
||||
- Generated (UTC): `2026-08-04T09:19:57Z`
|
||||
- Upstream commit: `d9c569f764b3a5629321858de03ff653d0f24056`
|
||||
- Device: `sdk_gphone64_x86_64`, API `33` (upstream tested reference: API `33`)
|
||||
- Observation method: `uiautomator_compact`
|
||||
- Provider/model: `local-vllm` / `qwen2.5-7b-instruct-local`
|
||||
- Model source/runtime: `local_gpu` / `vllm-0.19.0`
|
||||
- Accelerator: `NVIDIA_RTX_PRO_6000_Blackwell_96GB`
|
||||
- Required apps: `24/24`
|
||||
- Scope: 116 task(s), 5 trial(s), mode `candidate_rerun`
|
||||
- Full 116-task × 5-seed suite completed: **false**
|
||||
|
||||
The bundled ~88% baseline is historical input evidence. The manuscript's 88%→94% numbers are explicitly hypothetical and are not used as rerun results here.
|
||||
|
||||
## 1. Diagnose
|
||||
|
||||
- The historical run evaluated 116 tasks once each and reports approximately 88% overall success.
|
||||
- Wi-Fi is a concentrated failure cluster: three of the four SystemWifiTurn* rows failed in the bundled per-task table.
|
||||
- The capability matrix links the cluster to weak complex_ui_understanding, information_retrieval, and requires_setup behavior.
|
||||
- The failed traces show navigation/state-verification loops; increasing the step cap alone would treat a symptom rather than the cause.
|
||||
|
||||
## 2. Hypothesis
|
||||
|
||||
The diagnosis produced explicit surface, middle, and deep hypotheses. Only one variable is changed in this run; the other hypotheses remain untested.
|
||||
|
||||
| Layer / ID | Proposed change | Target | Verification | Status |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| surface / `H1` | Add Wi-Fi Settings navigation and final-state verification guidance. | At least one net paired success across the four Wi-Fi tasks, with no regression. | Paired upstream-prompt versus task-guideline ablation with matched seeds. | tested in source phase 1 |
|
||||
| surface / `H2` | Add application-specific recognition rules for the non-standard Tasks UI. | Improve at least two of the six historical Tasks failures with no regression. | Paired Tasks-only prompt/tool-description ablation after app provisioning. | not tested |
|
||||
| middle / `H3` | Repair and validate the multimodal input path for transcription tasks. | Raise transcription success above the historical 0% while bounding added tokens and latency. | Paired screenshot-disabled versus screenshot-enabled transcription run. | not tested |
|
||||
| middle / `H4` | Conditionally enable deeper thinking for counting tasks. | Improve math/counting success without applying the cost to unrelated tasks. | Paired tag-routed thinking-mode ablation with latency and token guardrails. | not tested |
|
||||
| middle / `H5` | Replace the API-35-incompatible gRPC accessibility feed with upstream's UIAutomator observation path. | At least one net paired Wi-Fi success with no regression and at most 1.5x latency/tokens. | Paired a11y-forwarder versus UIAutomator run with the same upstream T3A prompt and matched seeds. | tested in source phase 2 |
|
||||
| middle / `H5C` | Filter non-semantic UIAutomator container nodes after H5 exposed excessive prompt-token cost. | Preserve H5 paired success with no regression while using at most 0.75x raw-UIAutomator tokens and 1.5x latency. | Paired raw-UIAutomator versus compact-UIAutomator run with matched tasks, seeds, prompt, and evaluator. | tested in this run |
|
||||
| deep / `H6` | Combine screenshots with the structured UI tree and compare stronger vision-capable models. | Improve complex-UI success enough to justify multimodal latency and token cost. | Factorial UI-tree/screenshot/model ablation on the full tagged slice. | not tested |
|
||||
|
||||
Selected hypothesis: `H5C`
|
||||
- Change: Use the real upstream UIAutomator hierarchy but retain only visible text, descriptions, and actionable/scrollable elements.
|
||||
- Expected measurable result: Preserve H5 paired success with no regression while using at most 0.75x raw-UIAutomator tokens and 1.5x latency.
|
||||
- Guardrails: Same model, seed, task parameters, emulator, checkout, and step budget; require at least four completed pairs, every compact-UIAutomator treatment pair successful, no paired regression, at most 1.5x mean latency, and at most 0.75x raw-UIAutomator mean tokens. Passing a paired gate permits only a full-suite candidate rerun; it is not deployment approval, and a subset must never be reported as full-suite success.
|
||||
|
||||
## 3. Controlled experiment
|
||||
|
||||
- Phase: `phase_2_cost_refinement` — middle-layer input-pipeline cost refinement after the H5 success/cost result
|
||||
- Independent variable: raw versus semantic-filtered UIAutomator element list
|
||||
- Controls: same checkout, model, task parameters, generated seed, step budget, and emulator; arm order alternates by pair.
|
||||
|
||||
| Arm | Episodes | Success | Reward | Steps | Latency (s) | LLM calls | Mean tokens | Input / output tokens | Est. cost (USD) |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| candidate | 116/116 | 0.052 | 0.129 | 9.716 | 109.254 | 19.069 | 169894.517 | 19570600 / 137164 | 0.000000 |
|
||||
|
||||
## 4. Data-driven decision
|
||||
|
||||
- Outcome: **`candidate_subset_rerun_completed`**
|
||||
- Reason: A real modified candidate subset rerun completed, but it is not the 116-task × five-trial gate and cannot approve deployment.
|
||||
- Treatment/control mean latency ratio: n/a
|
||||
- Treatment/control mean token ratio: n/a
|
||||
- Treatment/control mean LLM-call ratio: n/a
|
||||
- Cost guardrails passed: **false**
|
||||
- Deployment approved: **false**
|
||||
|
||||
## 5. Rerun and next report
|
||||
|
||||
This run is a real controlled subset/smoke rerun, not the complete AndroidWorld benchmark. The next gate is a conditionally enabled candidate rerun over all 116 tasks with five seeds after provisioning the upstream API-33 app environment.
|
||||
|
||||
Observed residual failures:
|
||||
- `candidate / AudioRecorderRecordAudio / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / AudioRecorderRecordAudioWithFileName / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / BrowserDraw / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / BrowserMaze / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / BrowserMultiply / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / CameraTakePhoto / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / CameraTakeVideo / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / ClockStopWatchPausedVerify / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / ClockStopWatchRunning / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / ClockTimerEntry / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / ContactsAddContact / trial 4`: final evaluator state passed, but the agent never declared completion
|
||||
- `candidate / ExpenseAddMultiple / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / ExpenseAddMultipleFromGallery / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / ExpenseAddMultipleFromMarkor / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / ExpenseAddSingle / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / ExpenseDeleteDuplicates / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / ExpenseDeleteDuplicates2 / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / ExpenseDeleteMultiple / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / ExpenseDeleteMultiple2 / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / FilesDeleteFile / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / FilesMoveFile / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / MarkorAddNoteHeader / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / MarkorChangeNoteContent / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / MarkorCreateFolder / trial 4`: final evaluator state passed, but the agent never declared completion
|
||||
- `candidate / MarkorCreateNote / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / MarkorCreateNoteAndSms / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / MarkorCreateNoteFromClipboard / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / MarkorDeleteAllNotes / trial 4`: agent declared completion, but the real evaluator state failed
|
||||
- `candidate / MarkorDeleteNewestNote / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / MarkorDeleteNote / trial 4`: final evaluator state passed, but the agent never declared completion
|
||||
- `candidate / MarkorEditNote / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / MarkorMergeNotes / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / MarkorMoveNote / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / MarkorTranscribeReceipt / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / MarkorTranscribeVideo / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / ContactsNewContactDraft / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / OpenAppTaskEval / trial 4`: final evaluator state passed, but the agent never declared completion
|
||||
- `candidate / OsmAndFavorite / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / OsmAndMarker / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / OsmAndTrack / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / RecipeAddMultipleRecipes / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / RecipeAddMultipleRecipesFromImage / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / RecipeAddMultipleRecipesFromMarkor / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / RecipeAddMultipleRecipesFromMarkor2 / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / RecipeAddSingleRecipe / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / RecipeDeleteDuplicateRecipes / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / RecipeDeleteDuplicateRecipes2 / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / RecipeDeleteDuplicateRecipes3 / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / RecipeDeleteMultipleRecipes / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / RecipeDeleteMultipleRecipesWithConstraint / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / RecipeDeleteMultipleRecipesWithNoise / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / RecipeDeleteSingleWithRecipeWithNoise / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / RetroCreatePlaylist / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / RetroPlaylistDuration / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / RetroSavePlaylist / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SaveCopyOfReceiptTaskEval / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SimpleCalendarAddOneEvent / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SimpleCalendarAddOneEventInTwoWeeks / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SimpleCalendarAddOneEventRelativeDay / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SimpleCalendarAddOneEventTomorrow / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SimpleCalendarAddRepeatingEvent / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SimpleCalendarAnyEventsOnDate / trial 4`: agent declared completion, but the real evaluator state failed
|
||||
- `candidate / SimpleCalendarDeleteEvents / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SimpleCalendarDeleteEventsOnRelativeDay / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SimpleCalendarDeleteOneEvent / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SimpleCalendarEventOnDateAtTime / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SimpleCalendarEventsInNextWeek / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SimpleCalendarEventsInTimeRange / trial 4`: agent declared completion, but the real evaluator state failed
|
||||
- `candidate / SimpleCalendarEventsOnDate / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SimpleCalendarFirstEventAfterStartTime / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SimpleCalendarLocationOfEvent / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SimpleCalendarNextEvent / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SimpleCalendarNextMeetingWithPerson / trial 4`: final evaluator state passed, but the agent never declared completion
|
||||
- `candidate / SimpleDrawProCreateDrawing / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SimpleSmsReply / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SimpleSmsResend / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SimpleSmsSend / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SimpleSmsSendReceivedAddress / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SportsTrackerActivitiesCountForWeek / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SportsTrackerActivitiesOnDate / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SportsTrackerActivityDuration / trial 4`: agent declared completion, but the real evaluator state failed
|
||||
- `candidate / SportsTrackerLongestDistanceActivity / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SportsTrackerTotalDistanceForCategoryOverInterval / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SportsTrackerTotalDurationForCategoryThisWeek / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SystemBluetoothTurnOff / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SystemBluetoothTurnOffVerify / trial 4`: final evaluator state passed, but the agent never declared completion
|
||||
- `candidate / SystemBluetoothTurnOn / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SystemBluetoothTurnOnVerify / trial 4`: final evaluator state passed, but the agent never declared completion
|
||||
- `candidate / SystemBrightnessMax / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SystemBrightnessMin / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SystemWifiTurnOff / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SystemWifiTurnOffVerify / trial 4`: final evaluator state passed, but the agent never declared completion
|
||||
- `candidate / SystemWifiTurnOn / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SystemWifiTurnOnVerify / trial 4`: final evaluator state passed, but the agent never declared completion
|
||||
- `candidate / TasksCompletedTasksForDate / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / TasksDueNextWeek / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / TasksDueOnDate / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / TasksHighPriorityTasks / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / TasksHighPriorityTasksDueOnDate / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / TasksIncompleteTasksOnDate / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / TurnOffWifiAndTurnOnBluetooth / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / TurnOnWifiAndOpenApp / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / VlcCreatePlaylist / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / VlcCreateTwoPlaylists / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / RetroPlayingQueue / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SimpleSmsReplyMostRecent / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SimpleSmsSendClipboardContent / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / SystemCopyToClipboard / trial 4`: agent declared completion, but the real evaluator state failed
|
||||
- `candidate / NotesIsTodo / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
- `candidate / NotesTodoItemCount / trial 4`: evaluator reward / completion gate was not satisfied
|
||||
|
||||
### LLM analysis of this run
|
||||
|
||||
The following bounded interpretation was produced by the configured real LLM from the aggregate evidence (the JSON remains authoritative):
|
||||
|
||||
- Summary: The candidate subset rerun completed with 116 episodes, achieving a success rate of 5.17% and an estimated cost of $0.00. The mean latency was 109.25 seconds, and the mean number of LLM calls was 19.07.
|
||||
- Cost/benefit interpretation: The cost of running the candidate subset is minimal, with an estimated cost of $0.00. However, the success rate is low, and the latency and number of LLM calls are high, indicating that the current approach may not be efficient or effective.
|
||||
- Residual pattern: Most tasks failed to achieve success, with only a few tasks (e.g., NotesMeetingAttendeeCount, SystemBrightnessMaxVerify) succeeding.
|
||||
- Residual pattern: The majority of tasks took longer than the mean latency of 109.25 seconds, indicating potential inefficiencies.
|
||||
- Residual pattern: The mean number of LLM calls per task was 19.07, which is relatively high, suggesting that the model may be making multiple calls to achieve a task.
|
||||
- Next hypothesis `H5C_extended` (middle): Implement a more aggressive filtering of the UIAutomator hierarchy to reduce the number of LLM calls and improve latency while maintaining success rates. Target: Reduce the mean number of LLM calls per task to 10 and decrease the mean latency to 80 seconds. Verification: Run a full-suite candidate rerun with the new filtering strategy and compare the success rate, latency, and number of LLM calls to the current results.
|
||||
|
||||
## Environment boundaries
|
||||
|
||||
- UIAutomator is an upstream AndroidWorld observation option selected by the companion runner; it preserves real UI actions/evaluators but is a compatibility path, not the upstream API-33 reference configuration.
|
||||
- Compact UIAutomator removes only non-semantic container nodes; observations, coordinates, Android actions, and AndroidWorld evaluators remain real.
|
||||
- The full-suite candidate uses model qwen2.5-7b-instruct-local, while the promoted paired H5C source used doubao-seed-1-6-250615. This local-GPU campaign evaluates the promoted observation treatment but is not a same-model extension of the paired result.
|
||||
- ContactsNewContactDraft's official success predicate was fed the upstream UIAutomator state.ui_elements because that observation mode does not populate state.forest; the predicate and requested contact fields were not changed.
|
||||
- Run stopped by environment/runtime blocker: Generated params do not match checkpoint for MarkorCreateFolder:trial-4:seed-25270
|
||||
- Clipboard get/set retries once after the exact Clipper foreground-access runtime error; the operation, content, task, and evaluator are unchanged.
|
||||
- SimpleSmsReplyMostRecent polls the unchanged inbox query for up to five additional seconds because emulator-injected SMS delivery can lag past upstream's fixed wait. If the inbox remains empty, the exact last injected address/body is inserted into the same SMS database that upstream clears directly; task data and the evaluator are unchanged.
|
||||
- The pinned official Retro Music APK omits the playing_queue table, a known upstream runtime error. Only that exact missing-table condition was mapped to an empty observed queue so the unchanged exact queue predicate records an evaluator failure instead of losing the episode.
|
||||
- If compact UIAutomator still exceeds the pinned model's native 32,768-token context, the retry removes 8,192 characters only from the middle of the current-screen indexed UI section. Prompt prefix, goal, history, leading and trailing UI elements and indices, guidance, and output format remain; per-episode removal counters are retained.
|
||||
- Runtime-error retries reuse the exact task parameters retained in the discarded error checkpoints; upstream parameter-generator drift cannot silently change the retried task.
|
||||
- If compact UIAutomator still exceeds the pinned model's native 32,768-token context, the retry removes a bounded middle span only from indexed UI descriptions: at most 12,000 retained characters for action selection or 6,000 for each before/after summary screen. Prompt prefix, goal, history, action, reason, leading/trailing UI elements and indices, guidance, and output format remain; per-episode removal counters are retained.
|
||||
|
||||
The JSON beside this report is the authoritative evidence. It contains episode-level evaluator rewards, actions, timing, token counts, configuration, and explicit completion gates; credentials and raw prompts are not stored.
|
||||
Reference in New Issue
Block a user