docs: complete tool catalog, trajectory types, and browser automation details
This commit is contained in:
@@ -690,17 +690,162 @@ MODEL_STATUS_INFO
|
|||||||
MODEL_STATUS_WARNING
|
MODEL_STATUS_WARNING
|
||||||
```
|
```
|
||||||
|
|
||||||
### Cortex Step Types
|
### Cortex Step Types (Complete Tool Catalog)
|
||||||
|
|
||||||
These are the internal names for the agent's tool-use steps:
|
Every tool the agent can invoke, organized by category:
|
||||||
|
|
||||||
|
**File Operations:**
|
||||||
|
|
||||||
```
|
```
|
||||||
CORTEX_STEP_TYPE_CODE_ACTION
|
CODE_ACTION — edit/create files
|
||||||
CORTEX_STEP_TYPE_COMMAND_STATUS
|
VIEW_FILE — read file contents
|
||||||
CORTEX_STEP_TYPE_COMPILE_TOOL_PYLINT
|
VIEW_FILE_OUTLINE — view file structure
|
||||||
CORTEX_STEP_TYPE_GREP_SEARCH
|
VIEW_CODE_ITEM — view specific code symbol
|
||||||
CORTEX_STEP_TYPE_RUN_COMMAND
|
VIEW_CONTENT_CHUNK — view document chunk
|
||||||
CORTEX_STEP_TYPE_VIEW_CODE_ITEM
|
LIST_DIRECTORY — list directory contents
|
||||||
|
FIND — find files
|
||||||
|
GREP_SEARCH — search file contents
|
||||||
|
CODE_SEARCH — semantic code search
|
||||||
|
FIND_ALL_REFERENCES — find all references to symbol
|
||||||
|
FILE_CHANGE — track file changes
|
||||||
|
DELETE_DIRECTORY — delete directory
|
||||||
|
MOVE — move/rename files
|
||||||
|
```
|
||||||
|
|
||||||
|
**Terminal / Commands:**
|
||||||
|
|
||||||
|
```
|
||||||
|
RUN_COMMAND — execute shell command
|
||||||
|
COMMAND_STATUS — check command status
|
||||||
|
SEND_COMMAND_INPUT — send input to running command
|
||||||
|
READ_TERMINAL — read terminal output
|
||||||
|
SHELL_EXEC — execute shell (lightweight variant)
|
||||||
|
DEFINE_NEW_ENV_VARIABLE — set environment variable
|
||||||
|
```
|
||||||
|
|
||||||
|
**Browser Automation:**
|
||||||
|
|
||||||
|
```
|
||||||
|
OPEN_BROWSER_URL — navigate to URL
|
||||||
|
READ_BROWSER_PAGE — read current page
|
||||||
|
BROWSER_CLICK_ELEMENT — click element
|
||||||
|
BROWSER_INPUT — type into element
|
||||||
|
BROWSER_SELECT_OPTION — select dropdown option
|
||||||
|
BROWSER_PRESS_KEY — keyboard press
|
||||||
|
BROWSER_SCROLL / _UP / _DOWN — scroll page
|
||||||
|
BROWSER_REFRESH_PAGE — refresh
|
||||||
|
BROWSER_RESIZE_WINDOW — resize viewport
|
||||||
|
BROWSER_MOVE_MOUSE — move cursor
|
||||||
|
BROWSER_MOUSE_DOWN / _UP — mouse events
|
||||||
|
BROWSER_MOUSE_WHEEL — scroll wheel
|
||||||
|
BROWSER_DRAG_PIXEL_TO_PIXEL — drag and drop
|
||||||
|
CLICK_BROWSER_PIXEL — click at pixel coordinates
|
||||||
|
CAPTURE_BROWSER_SCREENSHOT — screenshot
|
||||||
|
CAPTURE_BROWSER_CONSOLE_LOGS — console output
|
||||||
|
BROWSER_GET_DOM — get DOM tree
|
||||||
|
BROWSER_LIST_NETWORK_REQUESTS — list network requests
|
||||||
|
BROWSER_GET_NETWORK_REQUEST — get specific request
|
||||||
|
EXECUTE_BROWSER_JAVASCRIPT — run JS in page
|
||||||
|
LIST_BROWSER_PAGES — list open pages
|
||||||
|
BROWSER_SUBAGENT — spawn browser sub-agent
|
||||||
|
```
|
||||||
|
|
||||||
|
**Web & External:**
|
||||||
|
|
||||||
|
```
|
||||||
|
SEARCH_WEB — web search
|
||||||
|
READ_URL_CONTENT — fetch URL
|
||||||
|
READ_RESOURCE — read MCP resource
|
||||||
|
LIST_RESOURCES — list MCP resources
|
||||||
|
MCP_TOOL — invoke MCP tool
|
||||||
|
```
|
||||||
|
|
||||||
|
**Git & PR:**
|
||||||
|
|
||||||
|
```
|
||||||
|
GIT_COMMIT — create commit
|
||||||
|
POST_PR_REVIEW — post PR review
|
||||||
|
```
|
||||||
|
|
||||||
|
**Knowledge & Memory:**
|
||||||
|
|
||||||
|
```
|
||||||
|
MEMORY — store memory
|
||||||
|
RETRIEVE_MEMORY — recall memory
|
||||||
|
KNOWLEDGE_GENERATION — generate knowledge item
|
||||||
|
KNOWLEDGE_ARTIFACTS — manage knowledge artifacts
|
||||||
|
KI_INSERTION — insert knowledge item
|
||||||
|
BRAIN_UPDATE — update brain state
|
||||||
|
RETRIEVE_CONTENT — retrieve stored content
|
||||||
|
CONVERSATION_HISTORY — access conversation history
|
||||||
|
```
|
||||||
|
|
||||||
|
**Planning & Control Flow:**
|
||||||
|
|
||||||
|
```
|
||||||
|
PLANNER_RESPONSE — planner output
|
||||||
|
PLAN_INPUT — plan step input
|
||||||
|
TASK_BOUNDARY — mark task boundary
|
||||||
|
CHECKPOINT — checkpoint state
|
||||||
|
TRAJECTORY_CHOICE — choose trajectory branch
|
||||||
|
TRAJECTORY_SEARCH — search trajectories
|
||||||
|
TOOL_CALL_CHOICE — choose tool call
|
||||||
|
TOOL_CALL_PROPOSAL — propose tool call
|
||||||
|
PROPOSE_CODE — propose code change
|
||||||
|
CODE_ACKNOWLEDGEMENT — acknowledge code change
|
||||||
|
MANAGER_FEEDBACK — agent manager feedback
|
||||||
|
PROPOSAL_FEEDBACK — proposal feedback
|
||||||
|
CRITIQUE — self-critique step
|
||||||
|
FINISH — mark completion
|
||||||
|
WAIT — wait/pause
|
||||||
|
```
|
||||||
|
|
||||||
|
**Media & UI:**
|
||||||
|
|
||||||
|
```
|
||||||
|
GENERATE_IMAGE — generate image
|
||||||
|
SUGGESTED_RESPONSES — suggest user responses
|
||||||
|
NOTIFY_USER — notify user
|
||||||
|
EPHEMERAL_MESSAGE — temporary message
|
||||||
|
ERROR_MESSAGE — error display
|
||||||
|
SYSTEM_MESSAGE — system message
|
||||||
|
ADD_ANNOTATION — add UI annotation
|
||||||
|
CLIPBOARD — clipboard access
|
||||||
|
```
|
||||||
|
|
||||||
|
**Code Quality:**
|
||||||
|
|
||||||
|
```
|
||||||
|
COMPILE — compile/build
|
||||||
|
COMPILE_APPLET — compile applet
|
||||||
|
LINT_DIFF — lint changes
|
||||||
|
LINT_APPLET — lint applet
|
||||||
|
FINDINGS — code findings
|
||||||
|
```
|
||||||
|
|
||||||
|
**Workspace & Deployment:**
|
||||||
|
|
||||||
|
```
|
||||||
|
WORKSPACE_API — workspace operations
|
||||||
|
RESTART_DEV_SERVER — restart dev server
|
||||||
|
SET_UP_FIREBASE — Firebase setup
|
||||||
|
DEPLOY_FIREBASE — Firebase deployment
|
||||||
|
INSTALL_APPLET_DEPENDENCIES — install deps
|
||||||
|
INSTALL_APPLET_PACKAGE — install package
|
||||||
|
RUN_EXTENSION_CODE — run VS Code extension code
|
||||||
|
```
|
||||||
|
|
||||||
|
**Google-Internal (corp only):**
|
||||||
|
|
||||||
|
```
|
||||||
|
INTERNAL_SEARCH — internal search (Moma?)
|
||||||
|
MOMA — Google Moma system
|
||||||
|
MQUERY — Google MQuery
|
||||||
|
BLAZE_BUILD_TARGETS — Blaze build
|
||||||
|
BLAZE_TEST_TARGETS — Blaze test
|
||||||
|
BUILD_CLEANER — build cleaning
|
||||||
|
CIDER_AGENT_DUMMY — Cider agent placeholder
|
||||||
|
AGENCY_TOOL_CALL — agency tool
|
||||||
```
|
```
|
||||||
|
|
||||||
### Cortex Generation Types
|
### Cortex Generation Types
|
||||||
@@ -718,7 +863,49 @@ CORTEX_GENERATION_TYPE_STATIC_TRAJECTORY
|
|||||||
CORTEX_GENERATION_TYPE_VERIFY
|
CORTEX_GENERATION_TYPE_VERIFY
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
### Cortex Trajectory Types
|
||||||
|
|
||||||
|
Different "modes" the agent can operate in:
|
||||||
|
|
||||||
|
```
|
||||||
|
CASCADE — standard cascade (user-initiated)
|
||||||
|
INTERACTIVE_CASCADE — interactive cascade (multi-turn)
|
||||||
|
AUTO_CASCADE — auto-triggered cascade
|
||||||
|
AUTO_CASCADE_MANAGER — manages auto-cascades
|
||||||
|
BACKGROUND_RESEARCH — background research thread
|
||||||
|
BROWSER — browser automation trajectory
|
||||||
|
DEEP_THINK — extended reasoning mode
|
||||||
|
KNOWLEDGE_GENERATION — knowledge item generation
|
||||||
|
APPLIER — code application trajectory
|
||||||
|
BRAIN_UPDATE — brain/memory update
|
||||||
|
CHECKPOINT — state checkpoint
|
||||||
|
CUSTOM_TOOL — custom tool execution
|
||||||
|
LLM_JUDGE — LLM-as-judge evaluation
|
||||||
|
PASSIVE_CODER — passive coding assistance
|
||||||
|
RETRIEVE_MEMORY — memory retrieval
|
||||||
|
SUPERCOMPLETE — tab-completion trajectory
|
||||||
|
TOOL_CALL_PROPOSAL — tool call proposal
|
||||||
|
TRAJECTORY_CHOICE — trajectory branching
|
||||||
|
USER_MAINLINE — user's main trajectory
|
||||||
|
USER_GRANULAR — granular user trajectory
|
||||||
|
ARTIFACT_SUMMARY — artifact summary generation
|
||||||
|
```
|
||||||
|
|
||||||
|
### Cortex Trajectory Sources
|
||||||
|
|
||||||
|
Where trajectories originate from:
|
||||||
|
|
||||||
|
```
|
||||||
|
INTERACTIVE_CASCADE — user-initiated cascade
|
||||||
|
CASCADE_CLIENT — cascade client
|
||||||
|
REFACTOR_FUNCTION — refactor action
|
||||||
|
EXPLAIN_PROBLEM — explain/debug action
|
||||||
|
PASSIVE_CODER — passive coding
|
||||||
|
REPLAY — trajectory replay
|
||||||
|
SDK / PYTHON_SDK — API/SDK access
|
||||||
|
EVAL / EVAL_TASK — evaluation system
|
||||||
|
ASYNC_CF / ASYNC_SL / ASYNC_CM / ASYNC_PRD / ASYNC_PRR — async operations
|
||||||
|
```
|
||||||
|
|
||||||
## Hardcoded URLs
|
## Hardcoded URLs
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user