From 03f44bc1263cf72da62815e0fd820add9bdf0dde Mon Sep 17 00:00:00 2001 From: Nikketryhard Date: Wed, 18 Feb 2026 12:59:48 -0600 Subject: [PATCH] docs: document additional LS services, memory system, cascade configurations, supercomplete features, and browser automation policies. --- docs/ls-binary-analysis.md | 603 +++++++++++++++++++++++++++++++++++++ 1 file changed, 603 insertions(+) diff --git a/docs/ls-binary-analysis.md b/docs/ls-binary-analysis.md index 73933ff..5a59bde 100644 --- a/docs/ls-binary-analysis.md +++ b/docs/ls-binary-analysis.md @@ -505,6 +505,76 @@ Team/organization management. Handles licensing, seats, credits. - `UpdateCascadeWebSearchEnabled` — toggle web search - `SetUserApiProviderKey` — BYOK key management +### AnalyticsService (7+ methods) + +Separate analytics service for recording telemetry and training data. + +``` +BatchRecordCompletions +BatchRecordPrompts +RecordCommandUsage +RecordCompletions +RecordContextToPrompt +RecordCortexTrajectory +RecordCortexTrajectoryStep +``` + +**Notable:** Distinct from `ApiServerService.Record*` — likely a dedicated analytics pipeline. + +### UserAnalyticsService (5+ methods) + +Per-user analytics dashboard service. + +``` +Analytics +CascadeAnalytics +GetAnalytics +GetPreferredTimeZone +UserPageAnalytics +``` + +### KnowledgeBaseService (17+ methods) + +Enterprise knowledge base with multi-source ingestion. Uses OpenSearch backend. + +``` +AddGithubUsers +AddUsers +CancelKnowledgeBaseJobs +ConnectKnowledgeBaseAccount +DeleteKnowledgeBaseConnection +ForwardSlackPayload +GetConnectorInternalConfig +GetKnowledgeBaseConnectorState +GetKnowledgeBaseItemsFromScopeItems +GetKnowledgeBaseJobStates +GetKnowledgeBaseScopeItems +GetKnowledgeBaseWebhookUrl +IngestGithubData +IngestGoogleDriveData +IngestJiraData +IngestJiraPayload +IngestSlackData +IngestSlackPayload +KnowledgeBaseSearch +UpdateConnectorConfig +``` + +**Key insight:** Supports ingesting from **GitHub**, **Google Drive**, **Jira**, and **Slack** — full enterprise knowledge integration. + +### CodeIndexService (4+ methods) + +Code search and indexing. Uses OpenSearch for hybrid vector+keyword search. + +``` +GraphSearch +HybridSearch +OpenSearchAddRepository +OpenSearchGetIndex +``` + +**Notable:** `GraphSearch` suggests code graph traversal, `HybridSearch` combines vector embeddings with keyword matching. + ### ModelManagementService (8 methods) Self-hosted model management for hybrid deployments. @@ -935,6 +1005,539 @@ https://iamcredentials.googleapis.com/v1/ ← Service account impersonation --- +## Memory & Brain System + +The LS has a sophisticated memory system that persists across conversations. + +### Memory Sources + +``` +MEMORY_SOURCE_UNSPECIFIED +MEMORY_SOURCE_USER — manually created by user +MEMORY_SOURCE_CASCADE — extracted from cascade conversations +MEMORY_SOURCE_AUTO_CASCADE — automatically generated from cascades +``` + +### Memory Triggers + +``` +MEMORY_TRIGGER_UNSPECIFIED +MEMORY_TRIGGER_ALWAYS_ON — always active +MEMORY_TRIGGER_GLOB — glob pattern match on file paths +MEMORY_TRIGGER_MANUAL — only when explicitly invoked +MEMORY_TRIGGER_MODEL_DECISION — model decides when to use +``` + +### Memory Actions + +``` +MEMORY_ACTION_TYPE_UNSPECIFIED +MEMORY_ACTION_TYPE_CREATE +MEMORY_ACTION_TYPE_UPDATE +MEMORY_ACTION_TYPE_DELETE +``` + +### Brain Entry Types + +``` +BRAIN_ENTRY_TYPE_UNSPECIFIED +BRAIN_ENTRY_TYPE_PLAN — implementation plans +BRAIN_ENTRY_TYPE_TASK — task checklists +``` + +### Brain Update Triggers + +``` +BRAIN_UPDATE_TRIGGER_UNSPECIFIED +BRAIN_UPDATE_TRIGGER_USER_REQUESTED — user explicitly asks +BRAIN_UPDATE_TRIGGER_USER_NEW_INFO — user provides new info +BRAIN_UPDATE_TRIGGER_RESEARCH_NEW_INFO — research discovers new info +BRAIN_UPDATE_TRIGGER_SYSTEM_FORCED — system-initiated update +``` + +### Brain Filter Strategies + +``` +BRAIN_FILTER_STRATEGY_UNSPECIFIED +BRAIN_FILTER_STRATEGY_NO_MEMORIES — exclude all memories +BRAIN_FILTER_STRATEGY_NO_SYSTEM_INJECTED_STEPS — exclude system steps +``` + +--- + +## Expanded Cascade Configuration Keys + +Full list of all cascade config keys extracted from binary (60+ keys): + +### Core Behavior + +``` +CASCADE_ENFORCE_QUOTA — quota enforcement toggle +CASCADE_BASE_MODEL_ID — default model selection +CASCADE_DEFAULT_MODEL_OVERRIDE — model override +CASCADE_GLOBAL_CONFIG_OVERRIDE — global cascade config +CASCADE_MEMORY_CONFIG_OVERRIDE — memory subsystem config +CASCADE_PLAN_BASED_CONFIG_OVERRIDE — planner config +CASCADE_FREE_CONFIG_OVERRIDE — free-tier config +``` + +### Tool Access + +``` +CASCADE_ENABLE_MCP_TOOLS — MCP tool use +CASCADE_ANTIGRAVITY_BROWSER_TOOLS_ENABLED +CASCADE_ENABLE_PROXY_WEB_SERVER — proxy web server +CASCADE_ENABLE_IDE_TERMINAL_EXECUTION +CASCADE_VIEW_FILE_TOOL_CONFIG_OVERRIDE +CASCADE_USE_REPLACE_CONTENT_EDIT_TOOL +``` + +### Web Search + +``` +CASCADE_WEB_SEARCH_TOOL_ENABLED +CASCADE_WEB_SEARCH_TOOL_DISABLED +CASCADE_WEB_SEARCH_TOOL_UNSPECIFIED +CASCADE_WEB_APP_DEPLOYMENTS_ENABLED +``` + +### Extension Code Execution + +``` +CASCADE_RUN_EXTENSION_CODE_ENABLED +CASCADE_RUN_EXTENSION_CODE_DISABLED +CASCADE_RUN_EXTENSION_CODE_ONLY +CASCADE_RUN_EXTENSION_CODE_AUTO_RUN_ENABLED +CASCADE_RUN_EXTENSION_CODE_AUTO_RUN_MODEL_DECIDES +CASCADE_RUN_EXTENSION_CODE_AUTO_RUN_UNSPECIFIED +``` + +### Command Auto-Execution Policy + +``` +CASCADE_COMMANDS_AUTO_EXECUTION_UNSPECIFIED +CASCADE_COMMANDS_AUTO_EXECUTION_OFF — always ask +CASCADE_COMMANDS_AUTO_EXECUTION_AUTO — auto-run safe commands +CASCADE_COMMANDS_AUTO_EXECUTION_EAGER — auto-run most commands +``` + +### Memory & Learning + +``` +CASCADE_ENABLE_AUTOMATED_MEMORIES — auto-generate memories +CASCADE_USER_MEMORIES_IN_SYS_PROMPT +CASCADE_ENABLE_CUSTOM_RECIPES +CASCADE_BACKGROUND_RESEARCH_CONFIG_OVERRIDE +CASCADE_USE_EXPERIMENT_CHECKPOINTER +CASCADE_USE_SUBAGENT_CHECKPOINTER +``` + +### Autocomplete & Input + +``` +CASCADE_INPUT_AUTOCOMPLETE_ENABLED +CASCADE_INPUT_AUTOCOMPLETE_DISABLED +CASCADE_INPUT_AUTOCOMPLETE_UNSPECIFIED +CASCADE_RECIPES_AT_MENTION_VISIBILITY +``` + +### Run Status + +``` +CASCADE_RUN_STATUS_UNSPECIFIED +CASCADE_RUN_STATUS_IDLE +CASCADE_RUN_STATUS_BUSY +CASCADE_RUN_STATUS_RUNNING +CASCADE_RUN_STATUS_CANCELING +``` + +### Edit Importance (for code diffs) + +``` +CASCADE_EDIT_IMPORTANCE_UNSPECIFIED +CASCADE_EDIT_IMPORTANCE_LOW +CASCADE_EDIT_IMPORTANCE_MEDIUM +CASCADE_EDIT_IMPORTANCE_HIGH +``` + +### Seat Types + +``` +CASCADE_SEAT_TYPE_UNSPECIFIED +CASCADE_SEAT_TYPE_ENTRY — entry tier +CASCADE_SEAT_TYPE_STANDARD — standard tier +``` + +--- + +## Supercomplete (Tab Completion) System + +The "Supercomplete" system powers multi-line tab completions, distinct from basic autocomplete. + +### Trigger Conditions + +``` +SUPERCOMPLETE_TRIGGER_CONDITION_UNSPECIFIED +SUPERCOMPLETE_TRIGGER_CONDITION_TYPING +SUPERCOMPLETE_TRIGGER_CONDITION_AUTOCOMPLETE_ACCEPT +SUPERCOMPLETE_TRIGGER_CONDITION_AUTOCOMPLETE_PREDICTIVE +SUPERCOMPLETE_TRIGGER_CONDITION_SUPERCOMPLETE_ACCEPT +SUPERCOMPLETE_TRIGGER_CONDITION_SUPERCOMPLETE_PREDICTIVE +SUPERCOMPLETE_TRIGGER_CONDITION_TAB_JUMP_ACCEPT +SUPERCOMPLETE_TRIGGER_CONDITION_TAB_JUMP_EDIT +SUPERCOMPLETE_TRIGGER_CONDITION_TAB_JUMP_PREDICTIVE +SUPERCOMPLETE_TRIGGER_CONDITION_CURSOR_LINE_NAVIGATION +SUPERCOMPLETE_TRIGGER_CONDITION_FORCED +``` + +### Filters + +``` +SUPERCOMPLETE_FILTER_PREFIX_MATCH +SUPERCOMPLETE_FILTER_SUFFIX_MATCH +SUPERCOMPLETE_FILTER_INSERTION_CAP +SUPERCOMPLETE_FILTER_DELETION_CAP +SUPERCOMPLETE_FILTER_NO_OP +SUPERCOMPLETE_FILTER_REVERT +SUPERCOMPLETE_FILTER_PREVIOUSLY_SHOWN +SUPERCOMPLETE_FILTER_WHITESPACE_ONLY +SUPERCOMPLETE_FILTER_SCORE_THRESHOLD +``` + +### Config Keys + +``` +SUPERCOMPLETE_TEMPERATURE +SUPERCOMPLETE_MODEL_CONFIG +SUPERCOMPLETE_LINE_RADIUS +SUPERCOMPLETE_MIN_SCORE +SUPERCOMPLETE_MAX_INSERTIONS +SUPERCOMPLETE_MAX_DELETIONS +SUPERCOMPLETE_MAX_TRAJECTORY_STEPS +SUPERCOMPLETE_MAX_TRAJECTORY_STEP_SIZE +SUPERCOMPLETE_MAX_CONCURRENT_REQUESTS +SUPERCOMPLETE_FAST_DEBOUNCE +SUPERCOMPLETE_REGULAR_DEBOUNCE +SUPERCOMPLETE_RECENT_STEPS_DURATION +SUPERCOMPLETE_USE_CODE_DIAGNOSTICS +SUPERCOMPLETE_CODE_DIAGNOSTICS_TOP_K +SUPERCOMPLETE_DIAGNOSTIC_SEVERITY_THRESHOLD +SUPERCOMPLETE_DISABLE_TYPING_CACHE +SUPERCOMPLETE_DONT_FILTER_MID_STREAMED +SUPERCOMPLETE_ALWAYS_USE_CACHE_ON_EQUAL_STATE +SUPERCOMPLETE_CACHE_ON_PARENT_ID_KILL_SWITCH +SUPERCOMPLETE_ON_ACCEPT_ONLY +SUPERCOMPLETE_PRUNE_RESPONSE +SUPERCOMPLETE_PRUNE_MAX_INSERT_DELETE_LINE_DELTA +SUPERCOMPLETE_INLINE_PURE_DELETE +SUPERCOMPLETE_INLINE_RICH_GHOST_TEXT_INSERTIONS +SUPERCOMPLETE_USE_CURRENT_LINE +SUPERCOMPLETE_NO_ACTIVE_NODE +``` + +### Tab Jump (Cursor Navigation) + +``` +TAB_JUMP_ENABLED +TAB_JUMP_LINE_RADIUS +TAB_JUMP_CUMULATIVE_PROMPT_CONFIG +TAB_JUMP_FILTER_IN_SELECTION +TAB_JUMP_FILTER_INSERTION_CAP +TAB_JUMP_FILTER_DELETION_CAP +TAB_JUMP_FILTER_SCORE_THRESHOLD +TAB_JUMP_FILTER_WHITESPACE_ONLY +TAB_JUMP_FILTER_NO_OP +TAB_JUMP_FILTER_REVERT +TAB_JUMP_STOP_TOKEN_MIDSTREAM +TAB_JUMP_ON_ACCEPT_ONLY +TAB_JUMP_PRUNE_RESPONSE +TAB_JUMP_PRUNE_MAX_INSERT_DELETE_LINE_DELTA +TAB_JUMP_MIN_FILTER_RADIUS +``` + +--- + +## Browser Automation Policies + +### JS Execution Policy (user can control) + +``` +BROWSER_JS_EXECUTION_POLICY_UNSPECIFIED +BROWSER_JS_EXECUTION_POLICY_ALWAYS_ASK +BROWSER_JS_EXECUTION_POLICY_MODEL_DECIDES +BROWSER_JS_EXECUTION_POLICY_TURBO — never ask +BROWSER_JS_EXECUTION_POLICY_DISABLED +``` + +### JS Auto-Run Policy (cascade-level) + +``` +BROWSER_JS_AUTO_RUN_POLICY_UNSPECIFIED +BROWSER_JS_AUTO_RUN_POLICY_ENABLED +BROWSER_JS_AUTO_RUN_POLICY_DISABLED +BROWSER_JS_AUTO_RUN_POLICY_MODEL_DECIDES +``` + +### Browser Subagent Mode + +``` +BROWSER_SUBAGENT_MODE_UNSPECIFIED +BROWSER_SUBAGENT_MODE_MAIN_AGENT_ONLY +BROWSER_SUBAGENT_MODE_SUBAGENT_ONLY +BROWSER_SUBAGENT_MODE_SUBAGENT_PRIMARILY +BROWSER_SUBAGENT_MODE_BOTH_AGENTS +``` + +### Tool Set Mode (input/output level) + +``` +BROWSER_TOOL_SET_MODE_UNSPECIFIED +BROWSER_TOOL_SET_MODE_ALL_TOOLS +BROWSER_TOOL_SET_MODE_PIXEL_ONLY +BROWSER_TOOL_SET_MODE_ALL_INPUT_PIXEL_OUTPUT +``` + +### Ephemeral Options (what gets captured) + +``` +BROWSER_EPHEMERAL_OPTION_UNSPECIFIED +BROWSER_EPHEMERAL_OPTION_DOM +BROWSER_EPHEMERAL_OPTION_SCREENSHOT +``` + +### Action Waiting Reasons + +``` +BROWSER_ACTION_WAITING_REASON_UNSPECIFIED +BROWSER_ACTION_WAITING_REASON_PAGE_ACCESS +BROWSER_ACTION_WAITING_REASON_ACTION_PERMISSION +BROWSER_ACTION_WAITING_REASON_PAGE_ACCESS_AND_ACTION_PERMISSION +``` + +### Installation Status + +``` +BROWSER_INSTALLATION_STATUS_UNSPECIFIED +BROWSER_INSTALLATION_STATUS_NOT_INSTALLED +BROWSER_INSTALLATION_STATUS_IN_PROGRESS +BROWSER_INSTALLATION_STATUS_COMPLETE +BROWSER_INSTALLATION_STATUS_ERROR +``` + +--- + +## Sandbox System (macOS Seatbelt) + +The LS includes a `sandbox-wrapper.sh` that sandboxes command execution on macOS using Apple's Seatbelt framework. + +### Key Behaviors + +- **File writes**: Denied by default, allowed only to workspace dir and `/tmp` +- **Network**: Denied unless `--allow-network` flag is passed +- **Gitignore**: Reads `.gitignore` and `.agyignore` to deny access to ignored files +- **Process isolation**: Uses `sandbox-exec` with a generated Seatbelt profile +- **Violation detection**: Detects "Operation not permitted" errors and suggests disabling sandbox + +### `.agyignore` File + +Custom ignore file (like `.gitignore`) specifically for the sandbox. Patterns in `.agyignore` deny both read and write access within the sandbox. + +--- + +## Trainer / RLHF Infrastructure + +The binary contains extensive proto definitions for model training, suggesting the LS participates in or configures training runs. + +### Proto Package: `exa.trainer_pb` + +**Training Objectives:** + +``` +DPOConfig — Direct Preference Optimization +KTOConfig — Kahneman-Tversky Optimization +OnPolicyGRPOConfig — Group Relative Policy Optimization +KnowledgeDistillationConfig +PreferenceLoss — preference-based loss functions +``` + +**Model Architecture:** + +``` +BaseModelConfig +EncoderDecoderModelConfig +MixtureOfExpertsConfig — MoE architecture +MultiLatentAttentionConfig — MLA (DeepSeek-style?) +MultiTokenPredictionConfig +CrossAttentionConfig +SlidingWindowAttentionConfig +``` + +**Quantization:** + +``` +QuantizationConfig +QuantizationMethod +QuantizationPrecision +QuantizationPolicy / PolicySet +LinearQuantization +LayerNormQuantization +``` + +**Training Infrastructure:** + +``` +OptimizerType / OptimizerExpAvgDtype +CheckpointBackend (TORCH_DCP, ZARR) +ParallelismConfig / LayerParallelismConfig +RoPEEmbeddingConfig (Llama, DeepseekV scaling) +SGLangInferenceProviderConfig +MockedInferenceProviderConfig +``` + +**Experiment Management:** + +``` +ExperimentProject / ExperimentMetadata +SweepAxis / SweepItem +ScheduledModel / SchedulingGroupConfig +DeviceWorkerRoutingInfo +DraftTargetModelPair — speculative decoding? +``` + +### Key Proto Packages + +``` +exa.cortex_pb — agent/cascade core +exa.chat_pb — chat message/conversation protos +exa.trainer_pb — training/RLHF +exa.analytics_pb — telemetry +exa.api_server_pb — cloud API server +exa.seat_management_pb — user/team management +exa.user_analytics_pb — user analytics +exa.opensearch_clients_pb — knowledge base / code index +exa.language_server_pb — LS service +exa.extension_server_pb — extension server +exa.model_management_pb — model management +``` + +--- + +## MCP Integration Constants + +``` +MCP_SERVER_STATUS_UNSPECIFIED +MCP_SERVER_STATUS_PENDING +MCP_SERVER_STATUS_READY +MCP_SERVER_STATUS_ERROR +MCP_ADD_SERVER +MCP_SAVE_CONFIG +MCP_EXPAND_TOOLS +MCP_TOOL +MCP_RESOURCE +``` + +--- + +## Cortex Step Statuses + +``` +CORTEX_STEP_STATUS_UNSPECIFIED +CORTEX_STEP_STATUS_PENDING +CORTEX_STEP_STATUS_QUEUED +CORTEX_STEP_STATUS_GENERATING +CORTEX_STEP_STATUS_RUNNING +CORTEX_STEP_STATUS_WAITING +CORTEX_STEP_STATUS_DONE +CORTEX_STEP_STATUS_ERROR +CORTEX_STEP_STATUS_CANCELED +CORTEX_STEP_STATUS_CLEARED +CORTEX_STEP_STATUS_HALTED +CORTEX_STEP_STATUS_INTERRUPTED +CORTEX_STEP_STATUS_INVALID +``` + +### Step Sources + +``` +CORTEX_STEP_SOURCE_UNSPECIFIED +CORTEX_STEP_SOURCE_MODEL — generated by LLM +CORTEX_STEP_SOURCE_SYSTEM — system-generated +CORTEX_STEP_SOURCE_SYSTEM_SDK — SDK-generated +CORTEX_STEP_SOURCE_USER_EXPLICIT — user-initiated +CORTEX_STEP_SOURCE_USER_IMPLICIT — implied by user action +``` + +### Step Credit Reasons + +``` +CORTEX_STEP_CREDIT_REASON_UNSPECIFIED +CORTEX_STEP_CREDIT_REASON_LINT_FIXING_DISCOUNT — lint fixes are discounted +``` + +### Manager Feedback Status + +``` +CORTEX_STEP_MANAGER_FEEDBACK_STATUS_UNSPECIFIED +CORTEX_STEP_MANAGER_FEEDBACK_STATUS_APPROVED +CORTEX_STEP_MANAGER_FEEDBACK_STATUS_DENIED +CORTEX_STEP_MANAGER_FEEDBACK_STATUS_ERROR +``` + +### Error Categories + +``` +CORTEX_ERROR_CATEGORY_UNSPECIFIED +CORTEX_ERROR_CATEGORY_OVERALL +CORTEX_ERROR_CATEGORY_ACTION_PREPARE +CORTEX_ERROR_CATEGORY_ACTION_APPLY +``` + +### Compile Tools (Lint Integration) + +``` +CORTEX_STEP_COMPILE_TOOL_UNSPECIFIED +CORTEX_STEP_COMPILE_TOOL_PYLINT +``` + +--- + +## Onboarding Flow + +``` +ONBOARDING_PROGRESS_UNKNOWN +ONBOARDING_PROGRESS_NONE +ONBOARDING_PROGRESS_ELIGIBLE +ONBOARDING_PROGRESS_ONBOARDED + +ONBOARDING_ACTION_TYPE_UNSPECIFIED +ONBOARDING_ACTION_TYPE_CHAT +ONBOARDING_ACTION_TYPE_AUTOCOMPLETE +ONBOARDING_ACTION_TYPE_COMMAND + +ONBOARDING_PAGE_TYPE_UNSPECIFIED +ONBOARDING_PAGE_TYPE_HOTKEY +ONBOARDING_PAGE_TYPE_LENS +ONBOARDING_PAGE_TYPE_SEARCH +``` + +**Onboarding pages discovered:** + +``` +ONBOARDING_LANDING_PAGE_OPENED +ONBOARDING_AUTH_PAGE_OPENED +ONBOARDING_AUTH_MANUAL_PAGE_OPENED +ONBOARDING_CHOOSE_THEME_PAGE_OPENED +ONBOARDING_KEYBINDINGS_PAGE_OPENED +ONBOARDING_AGENT_CONFIG_PAGE_OPENED +ONBOARDING_IMPORT_PAGE_OPENED +ONBOARDING_MIGRATION_SCOPE_PAGE_OPENED +ONBOARDING_SETUP_PAGE_OPENED +ONBOARDING_TERMS_OF_USE_PAGE_OPENED +ONBOARDING_USAGE_MODE_PAGE_OPENED +``` + +--- + ## Protobuf Enum Numbers (from extension.js) Extracted from the compiled protobuf enum definitions in `extension.js`.