docs: complete tool catalog, trajectory types, and browser automation details

This commit is contained in:
Nikketryhard
2026-02-14 04:22:13 -06:00
parent 7f5a0f51d3
commit 95cb65f1ae

View File

@@ -690,17 +690,162 @@ MODEL_STATUS_INFO
MODEL_STATUS_WARNING MODEL_STATUS_WARNING
``` ```
### Cortex Step Types ### Cortex Step Types (Complete Tool Catalog)
These are the internal names for the agent's tool-use steps: Every tool the agent can invoke, organized by category:
**File Operations:**
``` ```
CORTEX_STEP_TYPE_CODE_ACTION CODE_ACTION — edit/create files
CORTEX_STEP_TYPE_COMMAND_STATUS VIEW_FILE — read file contents
CORTEX_STEP_TYPE_COMPILE_TOOL_PYLINT VIEW_FILE_OUTLINE — view file structure
CORTEX_STEP_TYPE_GREP_SEARCH VIEW_CODE_ITEM — view specific code symbol
CORTEX_STEP_TYPE_RUN_COMMAND VIEW_CONTENT_CHUNK — view document chunk
CORTEX_STEP_TYPE_VIEW_CODE_ITEM LIST_DIRECTORY — list directory contents
FIND — find files
GREP_SEARCH — search file contents
CODE_SEARCH — semantic code search
FIND_ALL_REFERENCES — find all references to symbol
FILE_CHANGE — track file changes
DELETE_DIRECTORY — delete directory
MOVE — move/rename files
```
**Terminal / Commands:**
```
RUN_COMMAND — execute shell command
COMMAND_STATUS — check command status
SEND_COMMAND_INPUT — send input to running command
READ_TERMINAL — read terminal output
SHELL_EXEC — execute shell (lightweight variant)
DEFINE_NEW_ENV_VARIABLE — set environment variable
```
**Browser Automation:**
```
OPEN_BROWSER_URL — navigate to URL
READ_BROWSER_PAGE — read current page
BROWSER_CLICK_ELEMENT — click element
BROWSER_INPUT — type into element
BROWSER_SELECT_OPTION — select dropdown option
BROWSER_PRESS_KEY — keyboard press
BROWSER_SCROLL / _UP / _DOWN — scroll page
BROWSER_REFRESH_PAGE — refresh
BROWSER_RESIZE_WINDOW — resize viewport
BROWSER_MOVE_MOUSE — move cursor
BROWSER_MOUSE_DOWN / _UP — mouse events
BROWSER_MOUSE_WHEEL — scroll wheel
BROWSER_DRAG_PIXEL_TO_PIXEL — drag and drop
CLICK_BROWSER_PIXEL — click at pixel coordinates
CAPTURE_BROWSER_SCREENSHOT — screenshot
CAPTURE_BROWSER_CONSOLE_LOGS — console output
BROWSER_GET_DOM — get DOM tree
BROWSER_LIST_NETWORK_REQUESTS — list network requests
BROWSER_GET_NETWORK_REQUEST — get specific request
EXECUTE_BROWSER_JAVASCRIPT — run JS in page
LIST_BROWSER_PAGES — list open pages
BROWSER_SUBAGENT — spawn browser sub-agent
```
**Web & External:**
```
SEARCH_WEB — web search
READ_URL_CONTENT — fetch URL
READ_RESOURCE — read MCP resource
LIST_RESOURCES — list MCP resources
MCP_TOOL — invoke MCP tool
```
**Git & PR:**
```
GIT_COMMIT — create commit
POST_PR_REVIEW — post PR review
```
**Knowledge & Memory:**
```
MEMORY — store memory
RETRIEVE_MEMORY — recall memory
KNOWLEDGE_GENERATION — generate knowledge item
KNOWLEDGE_ARTIFACTS — manage knowledge artifacts
KI_INSERTION — insert knowledge item
BRAIN_UPDATE — update brain state
RETRIEVE_CONTENT — retrieve stored content
CONVERSATION_HISTORY — access conversation history
```
**Planning & Control Flow:**
```
PLANNER_RESPONSE — planner output
PLAN_INPUT — plan step input
TASK_BOUNDARY — mark task boundary
CHECKPOINT — checkpoint state
TRAJECTORY_CHOICE — choose trajectory branch
TRAJECTORY_SEARCH — search trajectories
TOOL_CALL_CHOICE — choose tool call
TOOL_CALL_PROPOSAL — propose tool call
PROPOSE_CODE — propose code change
CODE_ACKNOWLEDGEMENT — acknowledge code change
MANAGER_FEEDBACK — agent manager feedback
PROPOSAL_FEEDBACK — proposal feedback
CRITIQUE — self-critique step
FINISH — mark completion
WAIT — wait/pause
```
**Media & UI:**
```
GENERATE_IMAGE — generate image
SUGGESTED_RESPONSES — suggest user responses
NOTIFY_USER — notify user
EPHEMERAL_MESSAGE — temporary message
ERROR_MESSAGE — error display
SYSTEM_MESSAGE — system message
ADD_ANNOTATION — add UI annotation
CLIPBOARD — clipboard access
```
**Code Quality:**
```
COMPILE — compile/build
COMPILE_APPLET — compile applet
LINT_DIFF — lint changes
LINT_APPLET — lint applet
FINDINGS — code findings
```
**Workspace & Deployment:**
```
WORKSPACE_API — workspace operations
RESTART_DEV_SERVER — restart dev server
SET_UP_FIREBASE — Firebase setup
DEPLOY_FIREBASE — Firebase deployment
INSTALL_APPLET_DEPENDENCIES — install deps
INSTALL_APPLET_PACKAGE — install package
RUN_EXTENSION_CODE — run VS Code extension code
```
**Google-Internal (corp only):**
```
INTERNAL_SEARCH — internal search (Moma?)
MOMA — Google Moma system
MQUERY — Google MQuery
BLAZE_BUILD_TARGETS — Blaze build
BLAZE_TEST_TARGETS — Blaze test
BUILD_CLEANER — build cleaning
CIDER_AGENT_DUMMY — Cider agent placeholder
AGENCY_TOOL_CALL — agency tool
``` ```
### Cortex Generation Types ### Cortex Generation Types
@@ -718,7 +863,49 @@ CORTEX_GENERATION_TYPE_STATIC_TRAJECTORY
CORTEX_GENERATION_TYPE_VERIFY CORTEX_GENERATION_TYPE_VERIFY
``` ```
--- ### Cortex Trajectory Types
Different "modes" the agent can operate in:
```
CASCADE — standard cascade (user-initiated)
INTERACTIVE_CASCADE — interactive cascade (multi-turn)
AUTO_CASCADE — auto-triggered cascade
AUTO_CASCADE_MANAGER — manages auto-cascades
BACKGROUND_RESEARCH — background research thread
BROWSER — browser automation trajectory
DEEP_THINK — extended reasoning mode
KNOWLEDGE_GENERATION — knowledge item generation
APPLIER — code application trajectory
BRAIN_UPDATE — brain/memory update
CHECKPOINT — state checkpoint
CUSTOM_TOOL — custom tool execution
LLM_JUDGE — LLM-as-judge evaluation
PASSIVE_CODER — passive coding assistance
RETRIEVE_MEMORY — memory retrieval
SUPERCOMPLETE — tab-completion trajectory
TOOL_CALL_PROPOSAL — tool call proposal
TRAJECTORY_CHOICE — trajectory branching
USER_MAINLINE — user's main trajectory
USER_GRANULAR — granular user trajectory
ARTIFACT_SUMMARY — artifact summary generation
```
### Cortex Trajectory Sources
Where trajectories originate from:
```
INTERACTIVE_CASCADE — user-initiated cascade
CASCADE_CLIENT — cascade client
REFACTOR_FUNCTION — refactor action
EXPLAIN_PROBLEM — explain/debug action
PASSIVE_CODER — passive coding
REPLAY — trajectory replay
SDK / PYTHON_SDK — API/SDK access
EVAL / EVAL_TASK — evaluation system
ASYNC_CF / ASYNC_SL / ASYNC_CM / ASYNC_PRD / ASYNC_PRR — async operations
```
## Hardcoded URLs ## Hardcoded URLs