Initial commit: custom OpenClaw skills from docker-test
- workspace: capmetro-monitor, github-notifications, model-selector - workspace-security: vt-monitor, monitor-unauthorized - workspace-home: cron-manager, monitor-unauthorized - extensions: vt-sentinel (VT-Sentinel plugin) Includes sync.sh for pull/push, README, AGENTS.md, .gitignore.
This commit is contained in:
16
.devdoc.json
Normal file
16
.devdoc.json
Normal file
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"project": {
|
||||
"name": "openclaw-skills",
|
||||
"intent": "Backup and version control for custom OpenClaw skills synced from docker-test deployment.",
|
||||
"tier": 3,
|
||||
"status": "active",
|
||||
"ownership": "owned"
|
||||
},
|
||||
"documentation": {
|
||||
"outline_id": ""
|
||||
},
|
||||
"backlog": {
|
||||
"current_priority": "Initial setup",
|
||||
"next_steps": []
|
||||
}
|
||||
}
|
||||
13
.gitignore
vendored
Normal file
13
.gitignore
vendored
Normal file
@@ -0,0 +1,13 @@
|
||||
# Runtime state (never commit tokens, cache, or planning data)
|
||||
state.json
|
||||
seen-connections.json
|
||||
authorized-ips.json
|
||||
*.bak
|
||||
|
||||
# Skill memory/state directories
|
||||
**/memory/
|
||||
|
||||
# Editor and OS
|
||||
.DS_Store
|
||||
*.swp
|
||||
*.swo
|
||||
41
AGENTS.md
Normal file
41
AGENTS.md
Normal file
@@ -0,0 +1,41 @@
|
||||
# OpenClaw Skills — Agent Guidance
|
||||
|
||||
This repository is a **sync target** for custom OpenClaw skills from `docker-test:/opt/openclaw`. It is not the canonical source for runtime — docker-test is. This repo provides backup, version control, and a safe place to edit skills before pushing.
|
||||
|
||||
## For AI Agents
|
||||
|
||||
### What This Repo Is
|
||||
|
||||
- A mirror of custom skills (workspace, workspace-security, workspace-home, extensions)
|
||||
- Skills here are **not** part of OpenClaw's built-in defaults or ClawHub's public registry
|
||||
- Structure mirrors remote paths: `workspace/`, `workspace-security/`, `workspace-home/`, `extensions/`
|
||||
|
||||
### What to Do
|
||||
|
||||
1. **Syncing**: Use `./sync.sh pull` to fetch latest from docker-test. Use `./sync.sh push` to deploy local changes.
|
||||
2. **Editing skills**: Modify `SKILL.md` and scripts in the appropriate directory. Follow existing skill structure.
|
||||
3. **Never commit**: `state.json`, `seen-connections.json`, `authorized-ips.json`, `memory/` — these are runtime/token/cache data.
|
||||
4. **Testing**: After `./sync.sh push`, the user must restart OpenClaw or start a new session for changes to take effect.
|
||||
|
||||
### Key Paths
|
||||
|
||||
| Local | Remote |
|
||||
|-------|--------|
|
||||
| `workspace/*` | `/opt/openclaw/workspace/skills/*` |
|
||||
| `workspace-security/*` | `/opt/openclaw/state/workspace-security/skills/*` |
|
||||
| `workspace-home/*` | `/opt/openclaw/state/workspace-home/skills/*` |
|
||||
| `extensions/*` | `/opt/openclaw/state/extensions/openclaw-plugin-vt-sentinel/skills/*` |
|
||||
|
||||
### Sync Script
|
||||
|
||||
- `./sync.sh pull` — Download from docker-test (default)
|
||||
- `./sync.sh push` — Upload to docker-test
|
||||
- `REMOTE=host ./sync.sh pull` — Use different SSH host
|
||||
|
||||
Uses `scp` (rsync is not available on docker-test).
|
||||
|
||||
### Project Conventions
|
||||
|
||||
- Refer to `/home/b3nw/projects/AGENTS.md` for global agent entry point
|
||||
- User prefers SSH git remotes; origin is `gitea@git.local.ben.io:b3nw/openclaw-skills.git`
|
||||
- Do not add token, cache, or planning files to git
|
||||
80
README.md
Normal file
80
README.md
Normal file
@@ -0,0 +1,80 @@
|
||||
# OpenClaw Custom Skills
|
||||
|
||||
Backup and version control for custom OpenClaw skills deployed on `docker-test`. These skills extend the agent beyond the built-in defaults (ClawHub-installed or bundled).
|
||||
|
||||
## Purpose
|
||||
|
||||
- **Backup** — Preserve custom skills outside the deployment
|
||||
- **Version control** — Track changes, roll back, collaborate
|
||||
- **Sync** — Pull new skills from docker-test, push local edits back
|
||||
|
||||
## Directory Structure
|
||||
|
||||
Skills are organized by their source location on the remote:
|
||||
|
||||
| Directory | Remote Path | Skills |
|
||||
|-----------|-------------|--------|
|
||||
| `workspace/` | `/opt/openclaw/workspace/skills/` | capmetro-monitor, github-notifications, model-selector |
|
||||
| `workspace-security/` | `/opt/openclaw/state/workspace-security/skills/` | vt-monitor, monitor-unauthorized |
|
||||
| `workspace-home/` | `/opt/openclaw/state/workspace-home/skills/` | cron-manager |
|
||||
| `extensions/` | `/opt/openclaw/state/extensions/openclaw-plugin-vt-sentinel/skills/` | vt-sentinel |
|
||||
|
||||
Each skill is a folder containing `SKILL.md` and optional `scripts/`, `references/`, etc.
|
||||
|
||||
## Access Pattern
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- SSH access to `docker-test` (configured in `~/.ssh/config`)
|
||||
- OpenClaw deployed at `/opt/openclaw` on docker-test
|
||||
|
||||
### Sync Script
|
||||
|
||||
```bash
|
||||
# Pull skills FROM docker-test TO local (download new/updated)
|
||||
./sync.sh pull
|
||||
|
||||
# Push skills FROM local TO docker-test (upload your changes)
|
||||
./sync.sh push
|
||||
```
|
||||
|
||||
Override the SSH host:
|
||||
|
||||
```bash
|
||||
REMOTE=my-server ./sync.sh pull
|
||||
```
|
||||
|
||||
### What Gets Excluded
|
||||
|
||||
The sync script and `.gitignore` exclude runtime files:
|
||||
|
||||
- `state.json` — Per-skill state (tokens, seen IDs)
|
||||
- `seen-connections.json` — Connection tracking
|
||||
- `authorized-ips.json` — IP whitelist
|
||||
- `memory/` — Skill memory directories
|
||||
|
||||
These contain environment-specific or sensitive data and should never be committed.
|
||||
|
||||
## Custom Skills Inventory
|
||||
|
||||
| Skill | Source | Description |
|
||||
|-------|--------|-------------|
|
||||
| capmetro-monitor | workspace | Monitor CapMetro (Austin) service changes for Route 5/500 |
|
||||
| github-notifications | workspace | Check GitHub PRs and releases with smart filtering |
|
||||
| model-selector | workspace | Safely change agent primary/fallback models via LLM proxy |
|
||||
| vt-monitor | workspace-security | Parse VT-Sentinel activity from gateway logs |
|
||||
| monitor-unauthorized | workspace-security | Detect unauthorized WebSocket connections |
|
||||
| cron-manager | workspace-home | Manage cron and reminder workflows |
|
||||
| vt-sentinel | extensions | VirusTotal security scanner (ClawHub plugin) |
|
||||
|
||||
## First-Time Setup
|
||||
|
||||
1. Create the repository on Gitea (e.g. `b3nw/openclaw-skills`) if it does not exist.
|
||||
2. Push the initial commit: `git push -u origin main`
|
||||
|
||||
## Workflow
|
||||
|
||||
1. **Regular backup**: Run `./sync.sh pull` periodically to capture new skills or updates made directly on docker-test.
|
||||
2. **Edit locally**: Modify skills in this repo, commit, push to Gitea.
|
||||
3. **Deploy changes**: Run `./sync.sh push` to apply local edits to docker-test.
|
||||
4. **Restart OpenClaw** or start a new session so it picks up changes.
|
||||
197
extensions/vt-sentinel/SKILL.md
Normal file
197
extensions/vt-sentinel/SKILL.md
Normal file
@@ -0,0 +1,197 @@
|
||||
---
|
||||
name: vt-sentinel
|
||||
description: >-
|
||||
Security scanner using VirusTotal. Use when the user asks to scan a file for
|
||||
malware, check if a downloaded file or script is safe, verify a file hash
|
||||
against threat intelligence, or assess the security of any file. Returns both
|
||||
antivirus engine detections AND AI-powered Code Insight analysis (when
|
||||
available) in a single unified report.
|
||||
metadata:
|
||||
openclaw:
|
||||
emoji: "\U0001F6E1\uFE0F"
|
||||
---
|
||||
|
||||
# VT Sentinel — VirusTotal Active Protection
|
||||
|
||||
Protects OpenClaw users with **active prevention**, not just detection:
|
||||
|
||||
1. **Antivirus engines** — 60+ vendors check file hashes for known malware
|
||||
2. **AI Code Insight** — Gemini-based semantic analysis for scripts, skills, binaries
|
||||
3. **Active blocking** — Files detected as malicious are blocklisted and quarantined. Any subsequent attempt to execute them is automatically blocked before it runs.
|
||||
|
||||
Both AV and AI sources are always checked. The final verdict is the worst of the two. Malicious files are quarantined (renamed to `.QUARANTINED`) and added to a runtime blocklist that prevents their execution via `exec` or `bash` tools.
|
||||
|
||||
## Available Tools
|
||||
|
||||
### `vt_scan_file` — Full File Scan
|
||||
Classifies the file, computes SHA-256, checks VT (AV + Code Insight), uploads if unknown.
|
||||
|
||||
```
|
||||
vt_scan_file { "path": "/absolute/path/to/file" }
|
||||
```
|
||||
|
||||
### `vt_check_hash` — Quick Hash Lookup
|
||||
Fast check of a SHA-256 hash against VT. Returns AV detections + Code Insight if available.
|
||||
|
||||
```
|
||||
vt_check_hash { "hash": "e3b0c44298fc1c149afbf4c8996fb924..." }
|
||||
```
|
||||
|
||||
### `vt_upload_consent` — Confirm Sensitive File Upload
|
||||
When `vt_scan_file` returns `needs_consent`, relay the user's decision.
|
||||
|
||||
```
|
||||
vt_upload_consent { "path": "/path/to/document.pdf", "upload": true }
|
||||
vt_upload_consent { "path": "/path/to/document.pdf", "upload": false }
|
||||
```
|
||||
|
||||
## When to Use Which Tool
|
||||
|
||||
| Scenario | Tool |
|
||||
|----------|------|
|
||||
| User asks "is this file safe?" | `vt_scan_file` |
|
||||
| User provides a SHA-256 hash | `vt_check_hash` |
|
||||
| Evaluating a new SKILL.md or HOOK.md | `vt_scan_file` |
|
||||
| Checking a downloaded script or binary | `vt_scan_file` |
|
||||
| User said YES/NO to uploading a sensitive file | `vt_upload_consent` |
|
||||
|
||||
## Interpreting Results
|
||||
|
||||
Every result includes both AV detections and Code Insight (when available):
|
||||
|
||||
```
|
||||
File: example.sh
|
||||
Category: HIGH_RISK
|
||||
Verdict: MALICIOUS
|
||||
Detections: 12 malicious, 0 suspicious / 64 engines
|
||||
Code Insight (Code Insight): MALICIOUS
|
||||
Analysis: This script downloads and executes a remote payload...
|
||||
VT Link: https://www.virustotal.com/gui/file/...
|
||||
Summary: AV: 12/64 engines detected malware | AI: MALICIOUS — ...
|
||||
```
|
||||
|
||||
### Verdicts
|
||||
- **CLEAN** — No threats from AV engines or AI
|
||||
- **MALICIOUS** — AV engines and/or AI flagged the file. Warn the user immediately.
|
||||
- **SUSPICIOUS** — Some concerns raised. Recommend caution.
|
||||
- **PENDING** — File uploaded, analysis not yet available. Check again later.
|
||||
- **SKIPPED** — File classified as safe/media (auto-scan only; manual scans always check).
|
||||
- **NEEDS_CONSENT** — Sensitive file. Hash checked (not found). Ask user before uploading.
|
||||
|
||||
### Code Insight
|
||||
When present in the result, Code Insight provides:
|
||||
- **Source**: Analysis engine (e.g., "Code Insight", "palm")
|
||||
- **Verdict**: UNDETECTED / SUSPICIOUS / MALICIOUS
|
||||
- **Analysis**: Free-text description of what the file does
|
||||
|
||||
Code Insight works on any file type VT can analyze — scripts, skills, binaries (decompiled), documents with macros, etc.
|
||||
|
||||
## File Categories
|
||||
|
||||
Classification uses magic bytes and content analysis (never extensions alone):
|
||||
- **HIGH_RISK**: Binaries (PE, ELF, Mach-O), scripts (shebang/content patterns), ZIPs with executables → auto-scanned
|
||||
- **SEMANTIC_RISK**: SKILL.md, HOOK.md, AGENTS.md, SOUL.md, skill ZIPs → auto-scanned
|
||||
- **SENSITIVE**: PDF, Office docs, unknown ZIPs → hash checked, upload needs consent
|
||||
- **MEDIA/SAFE**: Images, video, audio, plain text → skipped in auto-scan, checked in manual scan
|
||||
|
||||
## Consent Flow for Sensitive Files
|
||||
|
||||
When `vt_scan_file` returns `NEEDS_CONSENT`:
|
||||
|
||||
1. Tell the user: the file's hash was checked (no match), but the file was NOT uploaded.
|
||||
2. Explain: uploading enables deep analysis (macros, embedded threats, AI), but content is shared with VirusTotal.
|
||||
3. Ask: "Would you like me to upload this file for a full scan?"
|
||||
4. Call `vt_upload_consent` with their answer.
|
||||
|
||||
## Admin Tools
|
||||
|
||||
### `vt_sentinel_status` — Current Status
|
||||
Shows effective configuration, monitored directories, policy matrix, active protections.
|
||||
|
||||
```
|
||||
vt_sentinel_status {}
|
||||
```
|
||||
|
||||
### `vt_sentinel_configure` — Change Config at Runtime
|
||||
Update any setting immediately. Changes persist to disk by default.
|
||||
|
||||
```
|
||||
vt_sentinel_configure { "preset": "privacy_first" }
|
||||
vt_sentinel_configure { "sensitiveFilePolicy": "hash_only", "notifyLevel": "threats_only" }
|
||||
vt_sentinel_configure { "watchDirsAdd": ["/extra/dir"], "excludeGlobs": ["*.log"] }
|
||||
vt_sentinel_configure { "blockMode": "log_only", "persist": "session" }
|
||||
```
|
||||
|
||||
**Presets**: `balanced` (default), `privacy_first` (hash-only, minimal logging), `strict_security` (auto-upload all, quarantine).
|
||||
|
||||
### `vt_sentinel_reset_policy` — Reset to Defaults
|
||||
Clears runtime overrides. Optionally clears first-run flags or blocklist.
|
||||
|
||||
```
|
||||
vt_sentinel_reset_policy {}
|
||||
vt_sentinel_reset_policy { "clearBlocklist": true }
|
||||
vt_sentinel_reset_policy { "clearFirstRun": true }
|
||||
```
|
||||
|
||||
### `vt_sentinel_help` — Quick-Start Guide
|
||||
Shows usage examples, privacy explanation, and available presets.
|
||||
|
||||
```
|
||||
vt_sentinel_help {}
|
||||
```
|
||||
|
||||
### `vt_sentinel_update` — Check for Updates
|
||||
Checks npm for a newer version and generates upgrade instructions.
|
||||
|
||||
```
|
||||
vt_sentinel_update {}
|
||||
vt_sentinel_update { "confirm": true }
|
||||
```
|
||||
|
||||
### `vt_sentinel_re_register` — Re-register Agent Identity
|
||||
Re-registers with VTAI using current identity settings. Creates a new `public_handle`.
|
||||
Use after changing `agentDisplayName` or other identity settings via `vt_sentinel_configure`.
|
||||
|
||||
```
|
||||
vt_sentinel_re_register {}
|
||||
vt_sentinel_re_register { "confirm": true }
|
||||
```
|
||||
|
||||
## Agent Identity
|
||||
|
||||
Each VT Sentinel instance registers with VTAI and appears on the agent leaderboard.
|
||||
By default, a unique name is auto-generated (e.g., `Sentinel-SwiftFalcon-a3f2`).
|
||||
|
||||
Configure identity via `vt_sentinel_configure`:
|
||||
- `agentDisplayName`: Custom display name for the leaderboard
|
||||
- `agentHumanAlias`: Human operator alias (no spaces)
|
||||
- `agentBio`: Short description (maps to VTAI `define_your_self`)
|
||||
- `agentContactEmail`: Optional contact email
|
||||
- `agentMetadataMode`: `minimal` (default, display name only) or `enhanced` (adds OS, preset info — no personal data)
|
||||
|
||||
After changing identity settings, use `vt_sentinel_re_register { "confirm": true }` to apply.
|
||||
|
||||
## Active Protection
|
||||
|
||||
VT Sentinel automatically protects the system in real-time:
|
||||
|
||||
1. **Auto-scan**: Every file downloaded or created by tools (`exec`, `write`, `web_fetch`) is automatically scanned
|
||||
2. **Blocklist**: Malicious and suspicious files are added to an in-memory blocklist
|
||||
3. **Quarantine**: Malicious files are renamed to `.QUARANTINED` so they cannot be executed
|
||||
4. **Execution blocking**: Any `exec`/`bash` command that references a blocked file is intercepted and prevented BEFORE execution
|
||||
5. **Command pattern inspection**: Commands are analyzed for dangerous patterns BEFORE execution, even when no file is involved:
|
||||
- **Pipe-to-shell**: `curl | bash`, `wget | sh`, `base64 -d | bash` — remote code execution without touching disk
|
||||
- **SSH key injection**: Appending to `authorized_keys` — backdoor persistence
|
||||
- **Data exfiltration**: Sending data to webhook.site, requestbin, pipedream, etc.
|
||||
- **Credential theft**: Piping `.env`, SSH keys, or AWS credentials to network tools
|
||||
|
||||
If you see a "BLOCKED" message, it means VT Sentinel prevented a potentially dangerous operation. Do NOT attempt to work around the block — inform the user about the threat.
|
||||
|
||||
## Constraints
|
||||
|
||||
- Always use absolute file paths
|
||||
- Never expose the VT API key in output
|
||||
- Rate limit: 4 requests/minute (free tier) — handled automatically
|
||||
- For SENSITIVE files, follow the consent flow — never upload without permission
|
||||
- If verdict is MALICIOUS, always warn the user prominently
|
||||
- Do not attempt to bypass quarantine or blocklist protections
|
||||
114
sync.sh
Executable file
114
sync.sh
Executable file
@@ -0,0 +1,114 @@
|
||||
#!/usr/bin/env bash
|
||||
# Sync OpenClaw custom skills between docker-test:/opt/openclaw and local openclaw-skills.
|
||||
# Run from this directory. Uses scp (rsync not available on docker-test).
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
REMOTE="${REMOTE:-docker-test}"
|
||||
OPENCLAW_BASE="/opt/openclaw"
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
cd "$SCRIPT_DIR"
|
||||
|
||||
# Runtime files to exclude from sync (tokens, cache, state)
|
||||
EXCLUDE_PATTERNS=(
|
||||
"state.json"
|
||||
"seen-connections.json"
|
||||
"authorized-ips.json"
|
||||
"memory"
|
||||
"*.bak"
|
||||
)
|
||||
|
||||
# Skills to skip during pull (duplicates or deprecated versions on remote)
|
||||
SKIP_SKILLS=(
|
||||
"workspace-home/monitor-unauthorized" # superseded by workspace-security version
|
||||
)
|
||||
|
||||
usage() {
|
||||
cat <<EOF
|
||||
Usage: $0 [direction]
|
||||
|
||||
Sync custom OpenClaw skills between $REMOTE:$OPENCLAW_BASE and local openclaw-skills.
|
||||
|
||||
direction: pull (default) | push
|
||||
pull - Sync FROM docker-test TO local (download new/updated skills)
|
||||
push - Sync FROM local TO docker-test (upload local changes)
|
||||
|
||||
Sources on remote:
|
||||
- workspace/skills/ (capmetro-monitor, github-notifications, model-selector)
|
||||
- state/workspace-security/skills/
|
||||
- state/workspace-home/skills/
|
||||
- state/extensions/openclaw-plugin-vt-sentinel/skills/
|
||||
|
||||
Environment:
|
||||
REMOTE - SSH host (default: docker-test)
|
||||
EOF
|
||||
}
|
||||
|
||||
pull_skill_dir() {
|
||||
local remote_path="$1"
|
||||
local local_path="$2"
|
||||
echo "Pulling $remote_path -> $local_path"
|
||||
mkdir -p "$local_path"
|
||||
scp -r "${REMOTE}:${OPENCLAW_BASE}/${remote_path}/"* "$local_path/" 2>/dev/null || true
|
||||
}
|
||||
|
||||
push_skill_dir() {
|
||||
local local_path="$1"
|
||||
local remote_path="$2"
|
||||
echo "Pushing $local_path -> $remote_path"
|
||||
ssh "$REMOTE" "mkdir -p ${OPENCLAW_BASE}/${remote_path}"
|
||||
for skill in "$local_path"/*/; do
|
||||
[ -d "$skill" ] || continue
|
||||
skill_name="$(basename "$skill")"
|
||||
scp -r "$skill" "${REMOTE}:${OPENCLAW_BASE}/${remote_path}/"
|
||||
done
|
||||
}
|
||||
|
||||
remove_runtime_files() {
|
||||
find . -name 'state.json' -delete 2>/dev/null || true
|
||||
find . -name 'seen-connections.json' -delete 2>/dev/null || true
|
||||
find . -name 'authorized-ips.json' -delete 2>/dev/null || true
|
||||
find . -type d -name 'memory' -exec rm -rf {} + 2>/dev/null || true
|
||||
find . -name '*.bak' -delete 2>/dev/null || true
|
||||
}
|
||||
|
||||
remove_skipped_skills() {
|
||||
for skip in "${SKIP_SKILLS[@]}"; do
|
||||
if [ -d "$skip" ]; then
|
||||
echo " Removing skipped skill: $skip"
|
||||
rm -rf "$skip"
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
pull() {
|
||||
echo "=== Pulling skills from $REMOTE ==="
|
||||
pull_skill_dir "workspace/skills" "workspace"
|
||||
pull_skill_dir "state/workspace-security/skills" "workspace-security"
|
||||
pull_skill_dir "state/workspace-home/skills" "workspace-home"
|
||||
pull_skill_dir "state/extensions/openclaw-plugin-vt-sentinel/skills" "extensions"
|
||||
remove_runtime_files
|
||||
remove_skipped_skills
|
||||
echo "Done. Run 'git status' to see changes."
|
||||
}
|
||||
|
||||
push() {
|
||||
echo "=== Pushing skills to $REMOTE ==="
|
||||
push_skill_dir "workspace" "workspace/skills"
|
||||
push_skill_dir "workspace-security" "state/workspace-security/skills"
|
||||
push_skill_dir "workspace-home" "state/workspace-home/skills"
|
||||
push_skill_dir "extensions" "state/extensions/openclaw-plugin-vt-sentinel/skills"
|
||||
echo "Done. Restart OpenClaw or start a new session to pick up changes."
|
||||
}
|
||||
|
||||
main() {
|
||||
local direction="${1:-pull}"
|
||||
case "$direction" in
|
||||
pull) pull ;;
|
||||
push) push ;;
|
||||
-h|--help) usage ;;
|
||||
*) echo "Unknown direction: $direction"; usage; exit 1 ;;
|
||||
esac
|
||||
}
|
||||
|
||||
main "$@"
|
||||
25
workspace-home/cron-manager/SKILL.md
Normal file
25
workspace-home/cron-manager/SKILL.md
Normal file
@@ -0,0 +1,25 @@
|
||||
---
|
||||
name: cron-manager
|
||||
description: Manage cron and reminder workflows with a strict no-guess process. Use when creating, updating, validating, listing, or auditing scheduled jobs, and default all deliveries to Discord channel 1348179994920095777 unless the user explicitly asks for another target.
|
||||
---
|
||||
|
||||
Use this skill as the only authority for cron/reminder actions.
|
||||
|
||||
Run a strict execution flow:
|
||||
1. Resolve the user intent (`list`, `create`, `update`, `delete`, `pause`, `resume`, `audit`).
|
||||
2. Confirm the delivery target; default to channel `1348179994920095777` unless explicitly overridden.
|
||||
3. Enumerate currently scheduled jobs before making changes.
|
||||
4. Apply only explicit user-requested changes.
|
||||
5. Re-list jobs and report the resulting state.
|
||||
|
||||
Enforce no-guess behavior:
|
||||
- Never invent CLI subcommands.
|
||||
- Check available command help first when command capability is uncertain.
|
||||
- Stop and report exact command limitations when the environment does not expose required verbs.
|
||||
- Create any new skill only through the `skill-creator` skill workflow.
|
||||
|
||||
Use concise confirmations that include:
|
||||
- Action performed
|
||||
- Effective target channel
|
||||
- Schedule or timing
|
||||
- Resulting job identity/name (if available)
|
||||
182
workspace-security/monitor-unauthorized/SKILL.md
Normal file
182
workspace-security/monitor-unauthorized/SKILL.md
Normal file
@@ -0,0 +1,182 @@
|
||||
---
|
||||
name: monitor-unauthorized
|
||||
description: >-
|
||||
Monitor and report unauthorized WebSocket gateway connections. Parses logs
|
||||
every 30 minutes, detects new and returning IPs, and manages per-IP Discord
|
||||
threads for tracking unauthorized access attempts.
|
||||
metadata:
|
||||
openclaw:
|
||||
emoji: "🚨"
|
||||
---
|
||||
|
||||
# Monitor Unauthorized Gateway Connections
|
||||
|
||||
Detects and tracks unauthorized WebSocket connection attempts to the OpenClaw gateway. Maintains per-IP Discord threads for ongoing tracking.
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
# Cron tool (every 30 minutes)
|
||||
bash skills/monitor-unauthorized/scripts/cron-wrapper.sh
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
### Scripts (data layer — no Discord/OpenClaw calls)
|
||||
|
||||
| Script | Purpose |
|
||||
|--------|---------|
|
||||
| `scripts/log-splitter.sh` | Extracts unauthorized connection entries from the gateway log into `/tmp/openclaw/unauthorized-connections.log` (incremental, byte-offset tracked) |
|
||||
| `scripts/check.sh` | Reads new entries from the unauthorized log, categorizes IPs as `new_ips` or `returning_ips`, updates `seen-connections.json` |
|
||||
| `scripts/index-threads.sh` | Manages the thread index — lookup, record, and staleness checks. The index maps IPs to their Discord thread session keys |
|
||||
| `scripts/cron-wrapper.sh` | Orchestrates the above scripts and outputs structured ACTION blocks for the agent |
|
||||
|
||||
### Agent (action layer — thread management via OpenClaw tools)
|
||||
|
||||
The agent parses the cron-wrapper output and handles all Discord thread operations using OpenClaw's built-in session and thread tools.
|
||||
|
||||
## Cron Behavior
|
||||
|
||||
1. `log-splitter.sh` extracts new unauthorized connection log lines (incremental)
|
||||
2. `check.sh` processes new entries, outputs JSON with `new_ips[]` and `returning_ips[]`
|
||||
3. `cron-wrapper.sh` formats ACTION blocks the agent must parse and act on
|
||||
4. **If no new activity**: script produces no output → agent replies `NO_REPLY`
|
||||
|
||||
## Parsing Cron Output
|
||||
|
||||
The cron-wrapper outputs ACTION blocks. Parse them as follows:
|
||||
|
||||
### `ACTION:INDEX_THREADS`
|
||||
|
||||
The thread index is missing or stale. You must refresh it before processing connections.
|
||||
|
||||
**Steps:**
|
||||
1. List all threads in the security Discord channel (`1471181304782389381`)
|
||||
2. For each thread whose name starts with `🚨` and contains `unauthorized gateway access`:
|
||||
- Extract the IP address from the thread name (format: `🚨 <ip> — unauthorized gateway access`)
|
||||
- Record it: `bash scripts/index-threads.sh record "<ip>" "<session_key>" "<thread_name>"`
|
||||
3. The session key format is: `agent:security:discord:channel:1471181304782389381:thread:<thread_name>`
|
||||
|
||||
### `ACTION:NEW_THREAD`
|
||||
|
||||
A new unauthorized IP was detected. Create a thread and report.
|
||||
|
||||
**Format:**
|
||||
```
|
||||
ACTION:NEW_THREAD
|
||||
IP:<ip_address>
|
||||
---
|
||||
<report content>
|
||||
---END_ACTION---
|
||||
```
|
||||
|
||||
**Steps:**
|
||||
1. Construct the thread name: `🚨 <ip> — unauthorized gateway access`
|
||||
2. Construct the session key: `agent:security:discord:channel:1471181304782389381:thread:<thread_name>`
|
||||
3. Use `sessions_send` to send the report content (between `---` and `---END_ACTION---`) to that session key. This creates the thread if it doesn't exist.
|
||||
4. Record the thread in the index: `bash scripts/index-threads.sh record "<ip>" "<session_key>"`
|
||||
|
||||
### `ACTION:UPDATE_THREAD`
|
||||
|
||||
A previously-seen IP has new connection attempts. Update the existing thread.
|
||||
|
||||
**Format:**
|
||||
```
|
||||
ACTION:UPDATE_THREAD
|
||||
IP:<ip_address>
|
||||
SESSION_KEY:<session_key or empty>
|
||||
---
|
||||
<update content>
|
||||
---END_ACTION---
|
||||
```
|
||||
|
||||
**Steps:**
|
||||
1. If `SESSION_KEY` is provided and non-empty, use it directly
|
||||
2. If `SESSION_KEY` is empty, construct it: `agent:security:discord:channel:1471181304782389381:thread:🚨 <ip> — unauthorized gateway access`
|
||||
3. Use `sessions_send` to post the update content to that session key
|
||||
4. Update the index: `bash scripts/index-threads.sh record "<ip>" "<session_key>"`
|
||||
|
||||
## Thread Index Management
|
||||
|
||||
The thread index (`memory/thread-index.json`) maps IPs to their Discord thread session keys. This avoids redundant thread creation and enables reliable updates.
|
||||
|
||||
### One-Time Bootstrap
|
||||
|
||||
On first run (or when the index is missing), the agent must:
|
||||
1. List all existing threads in channel `1471181304782389381`
|
||||
2. Identify threads matching the `🚨 <ip> — unauthorized gateway access` pattern
|
||||
3. Record each one via `scripts/index-threads.sh record`
|
||||
|
||||
### Ongoing Maintenance
|
||||
|
||||
- After creating a new thread (`ACTION:NEW_THREAD`), always `record` it in the index
|
||||
- After sending an update (`ACTION:UPDATE_THREAD`), always `record` it to refresh timestamps
|
||||
- The index auto-expires after 24 hours; the cron-wrapper will emit `ACTION:INDEX_THREADS` when a refresh is needed
|
||||
|
||||
### Index Script Commands
|
||||
|
||||
```bash
|
||||
# Check if index needs refresh
|
||||
bash scripts/index-threads.sh needs-refresh
|
||||
# Returns: "fresh" (exit 1), "stale" (exit 0), or "missing" (exit 0)
|
||||
|
||||
# Look up a thread by IP
|
||||
bash scripts/index-threads.sh lookup "1.2.3.4"
|
||||
# Returns: JSON with session_key, thread_name, etc. (exit 0 = found, exit 1 = not found)
|
||||
|
||||
# Record/update a thread entry
|
||||
bash scripts/index-threads.sh record "1.2.3.4" "agent:security:discord:channel:...:thread:..." "🚨 1.2.3.4 — unauthorized gateway access"
|
||||
|
||||
# Check index status
|
||||
bash scripts/index-threads.sh status
|
||||
# Returns: JSON with entry count, age, staleness
|
||||
```
|
||||
|
||||
## Storage Files
|
||||
|
||||
| File | Location | Purpose |
|
||||
|------|----------|---------|
|
||||
| `seen-connections.json` | Skill directory | All IPs ever seen — first_seen, last_seen, total_attempts, metadata |
|
||||
| `authorized-ips.json` | Skill directory | Whitelist — these IPs are silently skipped |
|
||||
| `memory/thread-index.json` | State directory | Maps IPs to Discord thread session keys |
|
||||
| `memory/unauth-splitter-offset` | State directory | Byte offset for log-splitter (gateway log) |
|
||||
| `memory/unauth-check-offset` | State directory | Byte offset for check (unauthorized log) |
|
||||
|
||||
## Authorized IPs (Whitelist)
|
||||
|
||||
Edit `authorized-ips.json` to suppress reporting for known IPs:
|
||||
|
||||
```json
|
||||
{
|
||||
"whitelist": ["127.0.0.1", "::1", "localhost", "192.168.1.100"]
|
||||
}
|
||||
```
|
||||
|
||||
## Log Files
|
||||
|
||||
| Log | Path | Contents |
|
||||
|-----|------|----------|
|
||||
| Gateway log | `/tmp/openclaw/openclaw.log` | Full OpenClaw gateway log (source) |
|
||||
| Unauthorized log | `/tmp/openclaw/unauthorized-connections.log` | Extracted unauthorized connection entries only |
|
||||
|
||||
## What This Monitors
|
||||
|
||||
Gateway WebSocket authorization failures containing `forwardedFor` IP addresses. Specifically:
|
||||
- Entries with `"forwardedFor"` in the JSON log
|
||||
- Entries with cause `"unauthorized"` or `"pairing-required"`
|
||||
|
||||
## Example Agent Flow
|
||||
|
||||
```
|
||||
1. Cron fires → bash scripts/cron-wrapper.sh
|
||||
2. Output contains ACTION:INDEX_THREADS → agent lists threads, records them
|
||||
3. Output contains ACTION:NEW_THREAD for IP 203.0.113.42 →
|
||||
a. agent constructs session key
|
||||
b. agent calls sessions_send with report content
|
||||
c. agent runs: bash scripts/index-threads.sh record "203.0.113.42" "<key>"
|
||||
4. Output contains ACTION:UPDATE_THREAD for IP 198.51.100.7 →
|
||||
a. agent uses SESSION_KEY from output (or constructs it)
|
||||
b. agent calls sessions_send with update content
|
||||
c. agent runs: bash scripts/index-threads.sh record "198.51.100.7" "<key>"
|
||||
5. No output → agent replies NO_REPLY
|
||||
```
|
||||
209
workspace-security/monitor-unauthorized/scripts/check.sh
Executable file
209
workspace-security/monitor-unauthorized/scripts/check.sh
Executable file
@@ -0,0 +1,209 @@
|
||||
#!/bin/bash
|
||||
# Incremental unauthorized connection checker
|
||||
# Reads the dedicated unauthorized-connections.log (populated by log-splitter.sh),
|
||||
# compares against seen-connections.json and authorized-ips.json,
|
||||
# and outputs structured JSON categorizing IPs as new or returning.
|
||||
#
|
||||
# Usage: bash check.sh
|
||||
#
|
||||
# Output JSON:
|
||||
# {
|
||||
# "timestamp": "2026-02-16T12:00:00Z",
|
||||
# "new_ips": [ { "ip": "...", "first_seen": "...", "reason": "...", ... } ],
|
||||
# "returning_ips": [ { "ip": "...", "new_attempts": 3, "latest": "...", ... } ],
|
||||
# "total_events": 12,
|
||||
# "whitelisted_skipped": 2
|
||||
# }
|
||||
|
||||
set -e
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
SKILL_DIR="$(dirname "$SCRIPT_DIR")"
|
||||
UNAUTH_LOG="/tmp/openclaw/unauthorized-connections.log"
|
||||
STATE_DIR="/home/node/.openclaw/workspace-security/memory"
|
||||
OFFSET_FILE="$STATE_DIR/unauth-check-offset"
|
||||
SEEN_FILE="$SKILL_DIR/seen-connections.json"
|
||||
AUTH_FILE="$SKILL_DIR/authorized-ips.json"
|
||||
|
||||
mkdir -p "$STATE_DIR"
|
||||
|
||||
# Initialize state files if missing
|
||||
[ ! -s "$SEEN_FILE" ] && echo '{}' > "$SEEN_FILE"
|
||||
[ ! -s "$AUTH_FILE" ] && echo '{"whitelist":["127.0.0.1","::1","localhost"]}' > "$AUTH_FILE"
|
||||
|
||||
# Check log exists
|
||||
if [ ! -f "$UNAUTH_LOG" ]; then
|
||||
echo '{"timestamp":"'"$(date -u +%Y-%m-%dT%H:%M:%SZ)"'","new_ips":[],"returning_ips":[],"total_events":0,"whitelisted_skipped":0}'
|
||||
exit 0
|
||||
fi
|
||||
|
||||
FILE_SIZE=$(stat -c%s "$UNAUTH_LOG")
|
||||
|
||||
# Read offset
|
||||
LAST_OFFSET=0
|
||||
if [ -f "$OFFSET_FILE" ]; then
|
||||
LAST_OFFSET=$(cat "$OFFSET_FILE")
|
||||
fi
|
||||
|
||||
# Handle log rotation / truncation
|
||||
if [ "$LAST_OFFSET" -gt "$FILE_SIZE" ]; then
|
||||
LAST_OFFSET=0
|
||||
fi
|
||||
|
||||
BYTES_NEW=$((FILE_SIZE - LAST_OFFSET))
|
||||
|
||||
if [ "$BYTES_NEW" -le 0 ]; then
|
||||
echo "$FILE_SIZE" > "$OFFSET_FILE"
|
||||
echo '{"timestamp":"'"$(date -u +%Y-%m-%dT%H:%M:%SZ)"'","new_ips":[],"returning_ips":[],"total_events":0,"whitelisted_skipped":0}'
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Load whitelist
|
||||
WHITELIST=$(jq -r '.whitelist[]' "$AUTH_FILE" 2>/dev/null | tr '\n' '|' | sed 's/|$//')
|
||||
|
||||
is_whitelisted() {
|
||||
local ip="$1"
|
||||
[[ "$ip" == "127.0.0.1" || "$ip" == "::1" || "$ip" == "localhost" ]] && return 0
|
||||
[ -n "$WHITELIST" ] && echo "$ip" | grep -qE "^($WHITELIST)$" && return 0
|
||||
return 1
|
||||
}
|
||||
|
||||
# Parse new log entries into per-IP aggregated data
|
||||
TMPFILE=$(mktemp)
|
||||
trap "rm -f $TMPFILE" EXIT
|
||||
|
||||
tail -c +"$((LAST_OFFSET + 1))" "$UNAUTH_LOG" > "$TMPFILE"
|
||||
|
||||
TOTAL_EVENTS=0
|
||||
WHITELISTED=0
|
||||
declare -A IP_EVENTS # ip -> JSON array of events
|
||||
declare -A IP_LATEST # ip -> latest timestamp
|
||||
declare -A IP_COUNT # ip -> count of events in this batch
|
||||
|
||||
while IFS= read -r line; do
|
||||
[ -z "$line" ] && continue
|
||||
TOTAL_EVENTS=$((TOTAL_EVENTS + 1))
|
||||
|
||||
# Extract fields from JSON log line
|
||||
PARSED=$(echo "$line" | jq -r '
|
||||
[
|
||||
.time // "",
|
||||
(.["1"].forwardedFor // ""),
|
||||
(.["1"].authReason // .["1"].reason // .["1"].cause // ""),
|
||||
(.["1"].origin // ""),
|
||||
(.["1"].userAgent // ""),
|
||||
((.["2"] // "") | tostring | (capture("remote=(?<r>[0-9.]+)") | .r) // "")
|
||||
] | @tsv
|
||||
' 2>/dev/null) || continue
|
||||
|
||||
IFS=$'\t' read -r ts ip reason origin ua remote <<< "$PARSED"
|
||||
[ -z "$ip" ] && continue
|
||||
|
||||
if is_whitelisted "$ip"; then
|
||||
WHITELISTED=$((WHITELISTED + 1))
|
||||
continue
|
||||
fi
|
||||
|
||||
# Build event JSON
|
||||
EVENT=$(jq -cn \
|
||||
--arg ts "$ts" \
|
||||
--arg ip "$ip" \
|
||||
--arg reason "$reason" \
|
||||
--arg origin "$origin" \
|
||||
--arg ua "$ua" \
|
||||
--arg remote "$remote" \
|
||||
'{timestamp:$ts, ip:$ip, reason:$reason, origin:$origin, user_agent:$ua, remote:$remote}')
|
||||
|
||||
# Aggregate by IP
|
||||
if [ -z "${IP_EVENTS[$ip]+x}" ]; then
|
||||
IP_EVENTS[$ip]="$EVENT"
|
||||
IP_COUNT[$ip]=1
|
||||
else
|
||||
IP_EVENTS[$ip]="${IP_EVENTS[$ip]}"$'\n'"$EVENT"
|
||||
IP_COUNT[$ip]=$(( ${IP_COUNT[$ip]} + 1 ))
|
||||
fi
|
||||
IP_LATEST[$ip]="$ts"
|
||||
|
||||
done < "$TMPFILE"
|
||||
|
||||
# Categorize IPs as new or returning
|
||||
NEW_IPS="[]"
|
||||
RETURNING_IPS="[]"
|
||||
|
||||
for ip in "${!IP_EVENTS[@]}"; do
|
||||
COUNT=${IP_COUNT[$ip]}
|
||||
LATEST=${IP_LATEST[$ip]}
|
||||
|
||||
# Get first event for details
|
||||
FIRST_EVENT=$(echo "${IP_EVENTS[$ip]}" | head -1)
|
||||
REASON=$(echo "$FIRST_EVENT" | jq -r '.reason')
|
||||
ORIGIN=$(echo "$FIRST_EVENT" | jq -r '.origin')
|
||||
UA=$(echo "$FIRST_EVENT" | jq -r '.user_agent')
|
||||
REMOTE=$(echo "$FIRST_EVENT" | jq -r '.remote')
|
||||
|
||||
# Check if IP was previously seen
|
||||
if jq -e --arg ip "$ip" 'has($ip)' "$SEEN_FILE" >/dev/null 2>&1; then
|
||||
# Returning IP — update seen record, add to returning list
|
||||
PREV_COUNT=$(jq -r --arg ip "$ip" '.[$ip].total_attempts // 0' "$SEEN_FILE")
|
||||
NEW_TOTAL=$((PREV_COUNT + COUNT))
|
||||
|
||||
STMP=$(mktemp)
|
||||
jq --arg ip "$ip" \
|
||||
--arg latest "$LATEST" \
|
||||
--argjson count "$NEW_TOTAL" \
|
||||
--argjson batch "$COUNT" \
|
||||
'.[$ip].last_seen = $latest | .[$ip].total_attempts = $count | .[$ip].attempts_this_batch = $batch' \
|
||||
"$SEEN_FILE" > "$STMP" && mv "$STMP" "$SEEN_FILE"
|
||||
|
||||
RETURNING_IPS=$(echo "$RETURNING_IPS" | jq \
|
||||
--arg ip "$ip" \
|
||||
--arg latest "$LATEST" \
|
||||
--argjson new_attempts "$COUNT" \
|
||||
--argjson total "$NEW_TOTAL" \
|
||||
--arg reason "$REASON" \
|
||||
--arg origin "$ORIGIN" \
|
||||
--arg ua "$UA" \
|
||||
--arg remote "$REMOTE" \
|
||||
'. += [{ip:$ip, latest:$latest, new_attempts:$new_attempts, total_attempts:$total, reason:$reason, origin:$origin, user_agent:$ua, remote:$remote}]')
|
||||
else
|
||||
# New IP — record it, add to new list
|
||||
STMP=$(mktemp)
|
||||
jq --arg ip "$ip" \
|
||||
--arg ts "$LATEST" \
|
||||
--arg reason "$REASON" \
|
||||
--arg origin "$ORIGIN" \
|
||||
--arg ua "$UA" \
|
||||
--arg remote "$REMOTE" \
|
||||
--argjson count "$COUNT" \
|
||||
'. + {($ip): {first_seen: $ts, last_seen: $ts, reason: $reason, origin: $origin, user_agent: $ua, remote: $remote, total_attempts: $count}}' \
|
||||
"$SEEN_FILE" > "$STMP" && mv "$STMP" "$SEEN_FILE"
|
||||
|
||||
NEW_IPS=$(echo "$NEW_IPS" | jq \
|
||||
--arg ip "$ip" \
|
||||
--arg first_seen "$LATEST" \
|
||||
--argjson attempts "$COUNT" \
|
||||
--arg reason "$REASON" \
|
||||
--arg origin "$ORIGIN" \
|
||||
--arg ua "$UA" \
|
||||
--arg remote "$REMOTE" \
|
||||
'. += [{ip:$ip, first_seen:$first_seen, attempts:$attempts, reason:$reason, origin:$origin, user_agent:$ua, remote:$remote}]')
|
||||
fi
|
||||
done
|
||||
|
||||
# Save new offset
|
||||
echo "$FILE_SIZE" > "$OFFSET_FILE"
|
||||
|
||||
# Output structured result
|
||||
jq -n \
|
||||
--arg ts "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
|
||||
--argjson new_ips "$NEW_IPS" \
|
||||
--argjson returning_ips "$RETURNING_IPS" \
|
||||
--argjson total "$TOTAL_EVENTS" \
|
||||
--argjson whitelisted "$WHITELISTED" \
|
||||
'{
|
||||
timestamp: $ts,
|
||||
new_ips: $new_ips,
|
||||
returning_ips: $returning_ips,
|
||||
total_events: $total,
|
||||
whitelisted_skipped: $whitelisted
|
||||
}'
|
||||
129
workspace-security/monitor-unauthorized/scripts/cron-wrapper.sh
Executable file
129
workspace-security/monitor-unauthorized/scripts/cron-wrapper.sh
Executable file
@@ -0,0 +1,129 @@
|
||||
#!/bin/bash
|
||||
# Unauthorized connection monitor — cron wrapper
|
||||
# Orchestrates: log-splitter → check → formatted output with agent instructions
|
||||
#
|
||||
# Output is structured for the OpenClaw agent to parse and act on.
|
||||
# The agent reads SKILL.md for detailed instructions on thread management.
|
||||
#
|
||||
# Flow:
|
||||
# 1. Run log-splitter to extract new unauthorized entries from gateway log
|
||||
# 2. Check thread index status (does the agent need to refresh it?)
|
||||
# 3. Run check to categorize IPs as new or returning
|
||||
# 4. Format output with ACTION markers the agent will parse
|
||||
#
|
||||
# Exit: always 0. Empty output = nothing to do (agent replies NO_REPLY)
|
||||
set -e
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
# --- Step 1: Split unauthorized entries from gateway log ---
|
||||
SPLIT_RESULT=$("$SCRIPT_DIR/log-splitter.sh" 2>/dev/null) || true
|
||||
|
||||
# --- Step 2: Check thread index ---
|
||||
INDEX_STATUS=$("$SCRIPT_DIR/index-threads.sh" needs-refresh 2>/dev/null) || INDEX_STATUS="missing"
|
||||
|
||||
# --- Step 3: Check for new/returning unauthorized connections ---
|
||||
CHECK_RESULT=$("$SCRIPT_DIR/check.sh" 2>/dev/null) || CHECK_RESULT='{}'
|
||||
|
||||
NEW_COUNT=$(echo "$CHECK_RESULT" | jq '.new_ips | length' 2>/dev/null || echo 0)
|
||||
RETURNING_COUNT=$(echo "$CHECK_RESULT" | jq '.returning_ips | length' 2>/dev/null || echo 0)
|
||||
TOTAL=$(echo "$CHECK_RESULT" | jq '.total_events' 2>/dev/null || echo 0)
|
||||
|
||||
# Nothing to report
|
||||
if [ "$NEW_COUNT" -eq 0 ] && [ "$RETURNING_COUNT" -eq 0 ] && [ "$INDEX_STATUS" = "fresh" ]; then
|
||||
exit 0
|
||||
fi
|
||||
|
||||
OUTPUT=""
|
||||
|
||||
# --- Thread index refresh needed ---
|
||||
if [ "$INDEX_STATUS" != "fresh" ]; then
|
||||
OUTPUT+="ACTION:INDEX_THREADS
|
||||
The thread index is ${INDEX_STATUS}. Before processing connections, refresh the thread index.
|
||||
See SKILL.md section \"Thread Index Management\" for instructions.
|
||||
---END_ACTION---
|
||||
"
|
||||
fi
|
||||
|
||||
# --- New IPs: agent should create threads ---
|
||||
if [ "$NEW_COUNT" -gt 0 ]; then
|
||||
OUTPUT+="
|
||||
🚨 UNAUTHORIZED CONNECTIONS — ${NEW_COUNT} NEW IP(s) DETECTED
|
||||
============================================================
|
||||
"
|
||||
# Emit each new IP as an ACTION block
|
||||
for i in $(seq 0 $((NEW_COUNT - 1))); do
|
||||
IP=$(echo "$CHECK_RESULT" | jq -r ".new_ips[$i].ip")
|
||||
FIRST_SEEN=$(echo "$CHECK_RESULT" | jq -r ".new_ips[$i].first_seen")
|
||||
ATTEMPTS=$(echo "$CHECK_RESULT" | jq -r ".new_ips[$i].attempts")
|
||||
REASON=$(echo "$CHECK_RESULT" | jq -r ".new_ips[$i].reason")
|
||||
ORIGIN=$(echo "$CHECK_RESULT" | jq -r ".new_ips[$i].origin")
|
||||
UA=$(echo "$CHECK_RESULT" | jq -r ".new_ips[$i].user_agent")
|
||||
REMOTE=$(echo "$CHECK_RESULT" | jq -r ".new_ips[$i].remote")
|
||||
|
||||
OUTPUT+="
|
||||
ACTION:NEW_THREAD
|
||||
IP:${IP}
|
||||
---
|
||||
🚨 **NEW UNAUTHORIZED CONNECTION**
|
||||
|
||||
**IP:** \`${IP}\`
|
||||
**First Seen:** ${FIRST_SEEN}
|
||||
**Attempts:** ${ATTEMPTS}
|
||||
**Reason:** ${REASON}
|
||||
**Origin:** ${ORIGIN}
|
||||
**User Agent:** ${UA}
|
||||
**Remote (proxy):** ${REMOTE}
|
||||
|
||||
_New IP — thread created by security monitor._
|
||||
---END_ACTION---
|
||||
"
|
||||
done
|
||||
fi
|
||||
|
||||
# --- Returning IPs: agent should update existing threads ---
|
||||
if [ "$RETURNING_COUNT" -gt 0 ]; then
|
||||
OUTPUT+="
|
||||
⚠️ RETURNING CONNECTIONS — ${RETURNING_COUNT} KNOWN IP(s)
|
||||
==========================================================
|
||||
"
|
||||
for i in $(seq 0 $((RETURNING_COUNT - 1))); do
|
||||
IP=$(echo "$CHECK_RESULT" | jq -r ".returning_ips[$i].ip")
|
||||
LATEST=$(echo "$CHECK_RESULT" | jq -r ".returning_ips[$i].latest")
|
||||
NEW_ATTEMPTS=$(echo "$CHECK_RESULT" | jq -r ".returning_ips[$i].new_attempts")
|
||||
TOTAL_ATTEMPTS=$(echo "$CHECK_RESULT" | jq -r ".returning_ips[$i].total_attempts")
|
||||
REASON=$(echo "$CHECK_RESULT" | jq -r ".returning_ips[$i].reason")
|
||||
ORIGIN=$(echo "$CHECK_RESULT" | jq -r ".returning_ips[$i].origin")
|
||||
UA=$(echo "$CHECK_RESULT" | jq -r ".returning_ips[$i].user_agent")
|
||||
REMOTE=$(echo "$CHECK_RESULT" | jq -r ".returning_ips[$i].remote")
|
||||
|
||||
# Look up existing thread
|
||||
THREAD_INFO=$("$SCRIPT_DIR/index-threads.sh" lookup "$IP" 2>/dev/null) || THREAD_INFO=""
|
||||
SESSION_KEY=""
|
||||
if [ -n "$THREAD_INFO" ]; then
|
||||
SESSION_KEY=$(echo "$THREAD_INFO" | jq -r '.session_key // empty')
|
||||
fi
|
||||
|
||||
OUTPUT+="
|
||||
ACTION:UPDATE_THREAD
|
||||
IP:${IP}
|
||||
SESSION_KEY:${SESSION_KEY}
|
||||
---
|
||||
⚠️ **RETURNING UNAUTHORIZED CONNECTION**
|
||||
|
||||
**IP:** \`${IP}\`
|
||||
**Latest Attempt:** ${LATEST}
|
||||
**New Attempts (this period):** ${NEW_ATTEMPTS}
|
||||
**Total Attempts (all time):** ${TOTAL_ATTEMPTS}
|
||||
**Reason:** ${REASON}
|
||||
**Origin:** ${ORIGIN}
|
||||
**User Agent:** ${UA}
|
||||
**Remote (proxy):** ${REMOTE}
|
||||
|
||||
_Recurring access attempt — updated by security monitor._
|
||||
---END_ACTION---
|
||||
"
|
||||
done
|
||||
fi
|
||||
|
||||
echo -e "$OUTPUT"
|
||||
123
workspace-security/monitor-unauthorized/scripts/index-threads.sh
Executable file
123
workspace-security/monitor-unauthorized/scripts/index-threads.sh
Executable file
@@ -0,0 +1,123 @@
|
||||
#!/bin/bash
|
||||
# Thread index manager for monitor-unauthorized
|
||||
#
|
||||
# Maintains a local JSON index of Discord threads so the cron-wrapper
|
||||
# can determine whether to create a new thread or update an existing one.
|
||||
#
|
||||
# The index is populated BY THE AGENT (not by this script) because only
|
||||
# the agent has access to OpenClaw session/thread listing tools.
|
||||
#
|
||||
# This script provides helper operations:
|
||||
# bash index-threads.sh lookup <ip> — find thread for an IP (exit 0 = found)
|
||||
# bash index-threads.sh status — check if index exists and is fresh
|
||||
# bash index-threads.sh record <ip> <session_key> — add/update an entry
|
||||
# bash index-threads.sh needs-refresh — exit 0 if index is missing/stale
|
||||
#
|
||||
# Index location: STATE_DIR/thread-index.json
|
||||
# Format:
|
||||
# {
|
||||
# "indexed_at": "2026-02-16T12:00:00Z",
|
||||
# "channel_id": "1471181304782389381",
|
||||
# "threads": {
|
||||
# "1.2.3.4": {
|
||||
# "session_key": "agent:security:discord:channel:1471181304782389381:thread:🚨 1.2.3.4 — unauthorized gateway access",
|
||||
# "thread_name": "🚨 1.2.3.4 — unauthorized gateway access",
|
||||
# "first_indexed": "2026-02-10T08:00:00Z",
|
||||
# "last_updated": "2026-02-16T12:00:00Z",
|
||||
# "update_count": 3
|
||||
# }
|
||||
# }
|
||||
# }
|
||||
|
||||
set -e
|
||||
|
||||
STATE_DIR="/home/node/.openclaw/workspace-security/memory"
|
||||
INDEX_FILE="$STATE_DIR/thread-index.json"
|
||||
CHANNEL_ID="1471181304782389381"
|
||||
MAX_AGE=86400 # 24 hours before considered stale
|
||||
|
||||
mkdir -p "$STATE_DIR"
|
||||
|
||||
# Initialize empty index if missing
|
||||
init_index() {
|
||||
if [ ! -f "$INDEX_FILE" ] || [ ! -s "$INDEX_FILE" ]; then
|
||||
jq -n --arg ts "$(date -u +%Y-%m-%dT%H:%M:%SZ)" --arg ch "$CHANNEL_ID" \
|
||||
'{indexed_at: $ts, channel_id: $ch, threads: {}}' > "$INDEX_FILE"
|
||||
fi
|
||||
}
|
||||
|
||||
# Check if index needs refresh (missing, empty, or stale)
|
||||
needs_refresh() {
|
||||
if [ ! -f "$INDEX_FILE" ] || [ ! -s "$INDEX_FILE" ]; then
|
||||
echo "missing"
|
||||
return 0
|
||||
fi
|
||||
local age=$(( $(date +%s) - $(stat -c%Y "$INDEX_FILE" 2>/dev/null || echo 0) ))
|
||||
if [ "$age" -gt "$MAX_AGE" ]; then
|
||||
echo "stale"
|
||||
return 0
|
||||
fi
|
||||
echo "fresh"
|
||||
return 1
|
||||
}
|
||||
|
||||
# Look up a thread by IP
|
||||
lookup() {
|
||||
local ip="$1"
|
||||
init_index
|
||||
local result
|
||||
result=$(jq -e --arg ip "$ip" '.threads[$ip] // empty' "$INDEX_FILE" 2>/dev/null)
|
||||
if [ -n "$result" ]; then
|
||||
echo "$result"
|
||||
return 0
|
||||
fi
|
||||
return 1
|
||||
}
|
||||
|
||||
# Record a thread entry for an IP
|
||||
record() {
|
||||
local ip="$1"
|
||||
local session_key="$2"
|
||||
local thread_name="${3:-🚨 $ip — unauthorized gateway access}"
|
||||
local now
|
||||
now=$(date -u +%Y-%m-%dT%H:%M:%SZ)
|
||||
|
||||
init_index
|
||||
|
||||
local tmp
|
||||
tmp=$(mktemp)
|
||||
jq --arg ip "$ip" \
|
||||
--arg sk "$session_key" \
|
||||
--arg tn "$thread_name" \
|
||||
--arg now "$now" \
|
||||
'
|
||||
.threads[$ip] = (
|
||||
(.threads[$ip] // {first_indexed: $now, update_count: 0}) |
|
||||
.session_key = $sk |
|
||||
.thread_name = $tn |
|
||||
.last_updated = $now |
|
||||
.update_count = (.update_count + 1)
|
||||
)
|
||||
' "$INDEX_FILE" > "$tmp" && mv "$tmp" "$INDEX_FILE"
|
||||
echo '{"ok":true}'
|
||||
}
|
||||
|
||||
# Print index status
|
||||
status() {
|
||||
init_index
|
||||
local count
|
||||
count=$(jq '.threads | length' "$INDEX_FILE")
|
||||
local indexed_at
|
||||
indexed_at=$(jq -r '.indexed_at' "$INDEX_FILE")
|
||||
local age=$(( $(date +%s) - $(stat -c%Y "$INDEX_FILE" 2>/dev/null || echo 0) ))
|
||||
echo "{\"entries\":$count,\"indexed_at\":\"$indexed_at\",\"age_seconds\":$age,\"stale\":$([ $age -gt $MAX_AGE ] && echo true || echo false)}"
|
||||
}
|
||||
|
||||
# Dispatch
|
||||
case "${1:-status}" in
|
||||
lookup) lookup "$2" ;;
|
||||
record) record "$2" "$3" "$4" ;;
|
||||
status) status ;;
|
||||
needs-refresh) needs_refresh ;;
|
||||
*) echo "Usage: $0 {lookup|record|status|needs-refresh} [args...]" >&2; exit 1 ;;
|
||||
esac
|
||||
76
workspace-security/monitor-unauthorized/scripts/log-splitter.sh
Executable file
76
workspace-security/monitor-unauthorized/scripts/log-splitter.sh
Executable file
@@ -0,0 +1,76 @@
|
||||
#!/bin/bash
|
||||
# Log splitter for monitor-unauthorized
|
||||
# Extracts unauthorized WebSocket connection entries from the gateway log
|
||||
# into a dedicated log file for efficient incremental processing.
|
||||
#
|
||||
# Usage:
|
||||
# bash log-splitter.sh — extract recent entries (batch mode)
|
||||
# bash log-splitter.sh --full — re-extract from entire log (rebuild)
|
||||
#
|
||||
# Filters for log lines containing:
|
||||
# - "forwardedFor" AND ("unauthorized" OR "pairing-required")
|
||||
#
|
||||
# Output: /tmp/openclaw/unauthorized-connections.log
|
||||
# Each line is a valid JSON object extracted from the gateway log.
|
||||
|
||||
set -e
|
||||
|
||||
GATEWAY_LOG="/tmp/openclaw/openclaw.log"
|
||||
UNAUTH_LOG="/tmp/openclaw/unauthorized-connections.log"
|
||||
STATE_DIR="/home/node/.openclaw/workspace-security/memory"
|
||||
OFFSET_FILE="$STATE_DIR/unauth-splitter-offset"
|
||||
|
||||
mkdir -p "$STATE_DIR"
|
||||
touch "$UNAUTH_LOG"
|
||||
|
||||
FULL_MODE=false
|
||||
[ "${1:-}" = "--full" ] && FULL_MODE=true
|
||||
|
||||
if [ ! -f "$GATEWAY_LOG" ]; then
|
||||
echo '{"error":"Gateway log not found","log":"'"$GATEWAY_LOG"'"}' >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
FILE_SIZE=$(stat -c%s "$GATEWAY_LOG")
|
||||
|
||||
# Determine where to start reading
|
||||
LAST_OFFSET=0
|
||||
if [ -f "$OFFSET_FILE" ] && [ "$FULL_MODE" != "true" ]; then
|
||||
LAST_OFFSET=$(cat "$OFFSET_FILE")
|
||||
fi
|
||||
|
||||
# If file shrank (log rotation), reset
|
||||
if [ "$LAST_OFFSET" -gt "$FILE_SIZE" ]; then
|
||||
LAST_OFFSET=0
|
||||
fi
|
||||
|
||||
BYTES_NEW=$((FILE_SIZE - LAST_OFFSET))
|
||||
|
||||
if [ "$BYTES_NEW" -le 0 ]; then
|
||||
echo "$FILE_SIZE" > "$OFFSET_FILE"
|
||||
echo '{"new_lines":0,"total_lines":'$(wc -l < "$UNAUTH_LOG" | tr -d ' ')'}'
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Extract unauthorized connection lines from new bytes
|
||||
# Filter: must have forwardedFor AND be unauthorized/pairing-required
|
||||
NEW_LINES=0
|
||||
TMPFILE=$(mktemp)
|
||||
trap "rm -f $TMPFILE" EXIT
|
||||
|
||||
tail -c +"$((LAST_OFFSET + 1))" "$GATEWAY_LOG" \
|
||||
| grep '"forwardedFor"' \
|
||||
| grep -E '"unauthorized"|"pairing-required"' \
|
||||
> "$TMPFILE" 2>/dev/null || true
|
||||
|
||||
NEW_LINES=$(wc -l < "$TMPFILE" | tr -d ' ')
|
||||
|
||||
if [ "$NEW_LINES" -gt 0 ]; then
|
||||
cat "$TMPFILE" >> "$UNAUTH_LOG"
|
||||
fi
|
||||
|
||||
# Save new offset
|
||||
echo "$FILE_SIZE" > "$OFFSET_FILE"
|
||||
|
||||
TOTAL_LINES=$(wc -l < "$UNAUTH_LOG" | tr -d ' ')
|
||||
echo "{\"new_lines\":$NEW_LINES,\"total_lines\":$TOTAL_LINES}"
|
||||
38
workspace-security/vt-monitor/SKILL.md
Normal file
38
workspace-security/vt-monitor/SKILL.md
Normal file
@@ -0,0 +1,38 @@
|
||||
---
|
||||
name: vt-monitor
|
||||
description: >-
|
||||
Monitor VT-Sentinel activity from gateway logs. Parses scan events, uploads,
|
||||
verdicts, quarantines, blocks, and failures. Returns structured JSON of all
|
||||
VT-Sentinel activity since last check.
|
||||
metadata:
|
||||
openclaw:
|
||||
emoji: "🛡️"
|
||||
---
|
||||
|
||||
# VT-Sentinel Monitor
|
||||
|
||||
Parses the OpenClaw gateway log for all VT-Sentinel activity and returns structured reports.
|
||||
|
||||
## Available Tools
|
||||
|
||||
### `vt_monitor_check` — Check Recent Activity
|
||||
Returns all VT-Sentinel events since last check (or last N minutes).
|
||||
|
||||
```
|
||||
vt_monitor_check [minutes]
|
||||
```
|
||||
|
||||
Arguments:
|
||||
- `minutes` (optional, default: 60) — How far back to look
|
||||
|
||||
Output: JSON with categorized events (scans, uploads, verdicts, blocks, quarantines, failures).
|
||||
|
||||
### `vt_monitor_tail` — Live Tail
|
||||
Returns the last N VT-Sentinel log entries.
|
||||
|
||||
```
|
||||
vt_monitor_tail [count]
|
||||
```
|
||||
|
||||
Arguments:
|
||||
- `count` (optional, default: 50) — Number of recent entries
|
||||
111
workspace-security/vt-monitor/scripts/check.sh
Executable file
111
workspace-security/vt-monitor/scripts/check.sh
Executable file
@@ -0,0 +1,111 @@
|
||||
#!/bin/bash
|
||||
# Check VT-Sentinel activity — incremental (byte-offset tracking)
|
||||
# Only reads NEW log lines since last check
|
||||
set -e
|
||||
|
||||
LOG_FILE="/tmp/openclaw/vt-sentinel.log"
|
||||
STATE_DIR="/home/node/.openclaw/workspace-security/memory"
|
||||
OFFSET_FILE="$STATE_DIR/vt-log-offset"
|
||||
mkdir -p "$STATE_DIR"
|
||||
|
||||
if [ ! -f "$LOG_FILE" ]; then
|
||||
echo '{"error":"Gateway log not found"}'
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Current file size
|
||||
FILE_SIZE=$(stat -c%s "$LOG_FILE")
|
||||
|
||||
# Last read offset (0 = first run, reads last 1MB as bootstrap)
|
||||
LAST_OFFSET=0
|
||||
if [ -f "$OFFSET_FILE" ]; then
|
||||
LAST_OFFSET=$(cat "$OFFSET_FILE")
|
||||
fi
|
||||
|
||||
# If file shrank (log rotation), reset
|
||||
if [ "$LAST_OFFSET" -gt "$FILE_SIZE" ]; then
|
||||
LAST_OFFSET=0
|
||||
fi
|
||||
|
||||
# Dedicated log is small — no need for bootstrap limit
|
||||
SKIP=$LAST_OFFSET
|
||||
|
||||
BYTES_NEW=$((FILE_SIZE - SKIP))
|
||||
|
||||
# Nothing new
|
||||
if [ "$BYTES_NEW" -le 0 ]; then
|
||||
echo "$FILE_SIZE" > "$OFFSET_FILE"
|
||||
echo '{"totalEvents":0,"alerts":false,"summary":{"uploads":0,"upload_complete":0,"upload_failed":0,"cache_hits":0,"verdicts":{"clean":0,"malicious":0,"suspicious":0},"quarantined":0,"blocked":0},"events":[]}'
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Read only new bytes, grep VT-Sentinel
|
||||
TMPFILE=$(mktemp)
|
||||
trap "rm -f $TMPFILE" EXIT
|
||||
|
||||
tail -c +"$((SKIP + 1))" "$LOG_FILE" | grep '"subsystem.*gateway"' | grep '\[VT-Sentinel\]' | while IFS= read -r line; do
|
||||
TIMESTAMP=$(echo "$line" | grep -oP '"date":"\K[^"]*' | head -1)
|
||||
MESSAGE=$(echo "$line" | grep -oP '\[VT-Sentinel\] \K[^"\\]*' | head -1)
|
||||
[ -z "$TIMESTAMP" ] || [ -z "$MESSAGE" ] && continue
|
||||
|
||||
CATEGORY="info"
|
||||
case "$MESSAGE" in
|
||||
*"Unknown"*"uploading"*) CATEGORY="upload" ;;
|
||||
*"Uploaded for analysis"*) CATEGORY="upload_complete" ;;
|
||||
*"Upload failed"*) CATEGORY="upload_failed" ;;
|
||||
*"Cache hit"*) CATEGORY="cache_hit" ;;
|
||||
*"MALICIOUS"*) CATEGORY="verdict_malicious" ;;
|
||||
*"SUSPICIOUS"*) CATEGORY="verdict_suspicious" ;;
|
||||
*"clean"*|*"BENIGN"*) CATEGORY="verdict_clean" ;;
|
||||
*"quarantin"*) CATEGORY="quarantine" ;;
|
||||
*"BLOCKED"*|*"blocked"*) CATEGORY="blocked" ;;
|
||||
*"Watching:"*) CATEGORY="config" ;;
|
||||
*"Plugin loaded"*|*"Service stopped"*|*"Auto-"*|*"Registered"*|*"Using"*) CATEGORY="lifecycle" ;;
|
||||
esac
|
||||
|
||||
jq -cn --arg ts "$TIMESTAMP" --arg msg "$MESSAGE" --arg cat "$CATEGORY" \
|
||||
'{"timestamp":$ts,"message":$msg,"category":$cat}'
|
||||
done > "$TMPFILE"
|
||||
|
||||
# Save new offset
|
||||
echo "$FILE_SIZE" > "$OFFSET_FILE"
|
||||
|
||||
TOTAL=$(wc -l < "$TMPFILE" | tr -d ' ')
|
||||
[ -z "$TOTAL" ] && TOTAL=0
|
||||
|
||||
if [ "$TOTAL" -eq 0 ]; then
|
||||
echo '{"totalEvents":0,"alerts":false,"summary":{"uploads":0,"upload_complete":0,"upload_failed":0,"cache_hits":0,"verdicts":{"clean":0,"malicious":0,"suspicious":0},"quarantined":0,"blocked":0},"events":[]}'
|
||||
exit 0
|
||||
fi
|
||||
|
||||
EVENTS_JSON=$(jq -s '.' "$TMPFILE")
|
||||
|
||||
count_cat() {
|
||||
local c
|
||||
c=$(grep -c "\"category\":\"$1\"" "$TMPFILE" 2>/dev/null || true)
|
||||
echo "${c:-0}"
|
||||
}
|
||||
|
||||
jq -n \
|
||||
--argjson events "$EVENTS_JSON" \
|
||||
--argjson total "$TOTAL" \
|
||||
--argjson uploads "$(count_cat upload)" \
|
||||
--argjson upload_complete "$(count_cat upload_complete)" \
|
||||
--argjson upload_failed "$(count_cat upload_failed)" \
|
||||
--argjson cache_hits "$(count_cat cache_hit)" \
|
||||
--argjson clean "$(count_cat verdict_clean)" \
|
||||
--argjson malicious "$(count_cat verdict_malicious)" \
|
||||
--argjson suspicious "$(count_cat verdict_suspicious)" \
|
||||
--argjson quarantined "$(count_cat quarantine)" \
|
||||
--argjson blocked "$(count_cat blocked)" \
|
||||
'{
|
||||
totalEvents: $total,
|
||||
summary: {
|
||||
uploads: $uploads, upload_complete: $upload_complete, upload_failed: $upload_failed,
|
||||
cache_hits: $cache_hits,
|
||||
verdicts: {clean: $clean, malicious: $malicious, suspicious: $suspicious},
|
||||
quarantined: $quarantined, blocked: $blocked
|
||||
},
|
||||
alerts: ($malicious > 0 or $suspicious > 0 or $quarantined > 0 or $blocked > 0),
|
||||
events: $events
|
||||
}'
|
||||
136
workspace-security/vt-monitor/scripts/cron-wrapper.sh
Executable file
136
workspace-security/vt-monitor/scripts/cron-wrapper.sh
Executable file
@@ -0,0 +1,136 @@
|
||||
#!/bin/bash
|
||||
# VT-Sentinel monitoring cron wrapper (incremental)
|
||||
# Checks both VT-Sentinel file activity AND plugin updates
|
||||
set -e
|
||||
|
||||
SCRIPT_DIR="$(dirname "$0")"
|
||||
LOG_FILE="/tmp/openclaw/openclaw.log"
|
||||
STATE_DIR="/home/node/.openclaw/workspace-security/memory"
|
||||
SEEN_FILE="$STATE_DIR/seen-plugin-updates.txt"
|
||||
mkdir -p "$STATE_DIR"
|
||||
touch "$SEEN_FILE"
|
||||
|
||||
OUTPUT=""
|
||||
|
||||
# --- VT-Sentinel activity ---
|
||||
REPORT=$("$SCRIPT_DIR/check.sh" 2>/dev/null)
|
||||
TOTAL=$(echo "$REPORT" | jq '.totalEvents')
|
||||
ALERTS=$(echo "$REPORT" | jq '.alerts')
|
||||
ACTION_EVENTS=$(echo "$REPORT" | jq '[.summary.uploads, .summary.upload_complete, .summary.upload_failed, .summary.cache_hits, .summary.verdicts.clean, .summary.verdicts.malicious, .summary.verdicts.suspicious, .summary.quarantined, .summary.blocked] | add')
|
||||
|
||||
if [ "$ACTION_EVENTS" -gt 0 ]; then
|
||||
OUTPUT+="🛡️ VT-Sentinel Activity Report\n===============================\n\n"
|
||||
OUTPUT+="📊 Summary: ${ACTION_EVENTS} file events\n"
|
||||
OUTPUT+=$(echo "$REPORT" | jq -r '
|
||||
.summary |
|
||||
(if .uploads > 0 then " ⬆️ Uploads initiated: \(.uploads)" else empty end),
|
||||
(if .upload_complete > 0 then " ✅ Uploads completed: \(.upload_complete)" else empty end),
|
||||
(if .upload_failed > 0 then " ❌ Upload failures: \(.upload_failed)" else empty end),
|
||||
(if .cache_hits > 0 then " 💾 Cache hits: \(.cache_hits)" else empty end),
|
||||
(if .verdicts.clean > 0 then " ✅ Clean verdicts: \(.verdicts.clean)" else empty end),
|
||||
(if .verdicts.malicious > 0 then " 🚨 MALICIOUS: \(.verdicts.malicious)" else empty end),
|
||||
(if .verdicts.suspicious > 0 then " ⚠️ SUSPICIOUS: \(.verdicts.suspicious)" else empty end),
|
||||
(if .quarantined > 0 then " 📦 Quarantined: \(.quarantined)" else empty end),
|
||||
(if .blocked > 0 then " 🚫 Blocked: \(.blocked)" else empty end)
|
||||
')
|
||||
OUTPUT+="\n\n📋 Event Details:\n"
|
||||
OUTPUT+=$(echo "$REPORT" | jq -r '.events[] | select(.category != "lifecycle" and .category != "config") | " [\(.timestamp)] \(.category): \(.message)"')
|
||||
|
||||
if [ "$ALERTS" = "true" ]; then
|
||||
OUTPUT+="\n\n⚠️ SECURITY ALERT: Review malicious/suspicious/blocked events above!"
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- VT scan thread creation for new uploads ---
|
||||
PENDING_FILE="$STATE_DIR/pending-scans.json"
|
||||
[ ! -f "$PENDING_FILE" ] || [ ! -s "$PENDING_FILE" ] && echo '[]' > "$PENDING_FILE"
|
||||
SEEN_SCANS="$STATE_DIR/seen-scan-hashes.txt"
|
||||
touch "$SEEN_SCANS"
|
||||
|
||||
UPLOADS=$(echo "$REPORT" | jq -r '[.events[] | select(.category == "upload")] | .[]' 2>/dev/null)
|
||||
COMPLETES=$(echo "$REPORT" | jq -r '[.events[] | select(.category == "upload_complete")] | .[].message' 2>/dev/null)
|
||||
|
||||
if [ -n "$UPLOADS" ]; then
|
||||
# Extract filenames + risk categories from upload events
|
||||
echo "$REPORT" | jq -r '.events[] | select(.category == "upload") | .message' | while IFS= read -r msg; do
|
||||
RISK_CAT=$(echo "$msg" | grep -oP 'Unknown \K[A-Z_]+' || echo "UNKNOWN")
|
||||
FNAME=$(echo "$msg" | grep -oP 'file \K[^,]+' || echo "unknown")
|
||||
[ -z "$FNAME" ] || [ "$FNAME" = "unknown" ] && continue
|
||||
|
||||
# Check if already tracked
|
||||
grep -qF "$FNAME" "$SEEN_SCANS" 2>/dev/null && continue
|
||||
|
||||
# Find corresponding transaction ID from upload_complete events
|
||||
HASH=""
|
||||
if [ -n "$COMPLETES" ]; then
|
||||
# Get first unmatched transaction ID, decode to extract hash
|
||||
TXN_ID=$(echo "$COMPLETES" | grep -oP '\(\K[A-Za-z0-9+/=]+(?=\))' | head -1)
|
||||
if [ -n "$TXN_ID" ]; then
|
||||
HASH=$(echo "$TXN_ID" | base64 -d 2>/dev/null | cut -d: -f1)
|
||||
fi
|
||||
fi
|
||||
|
||||
# Create thread
|
||||
RESULT=$("$SCRIPT_DIR/scan-thread.sh" "$FNAME" "$RISK_CAT" "${HASH:-pending}" 2>/dev/null)
|
||||
THREAD_ID=$(echo "$RESULT" | jq -r '.threadId // empty')
|
||||
|
||||
if [ -n "$THREAD_ID" ] && [ -n "$HASH" ]; then
|
||||
# Add to pending scans
|
||||
PENDING=$(cat "$PENDING_FILE")
|
||||
PENDING=$(echo "$PENDING" | jq --arg h "$HASH" --arg f "$FNAME" --arg t "$THREAD_ID" --arg r "$RISK_CAT" \
|
||||
'. += [{"hash":$h,"filename":$f,"threadId":$t,"riskCategory":$r}]')
|
||||
echo "$PENDING" > "$PENDING_FILE"
|
||||
OUTPUT+="\n🛡️ Created scan thread: [$RISK_CAT] $FNAME (hash: ${HASH:0:12}...)\n"
|
||||
fi
|
||||
|
||||
echo "$FNAME" >> "$SEEN_SCANS"
|
||||
done
|
||||
fi
|
||||
|
||||
# --- Follow up on pending scans ---
|
||||
PENDING_COUNT=$(jq 'length' "$PENDING_FILE" 2>/dev/null || echo 0)
|
||||
if [ "$PENDING_COUNT" -gt 0 ]; then
|
||||
FOLLOWUP=$("$SCRIPT_DIR/followup.sh" 2>/dev/null)
|
||||
if [ -n "$FOLLOWUP" ]; then
|
||||
OUTPUT+="\n$FOLLOWUP\n"
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- Plugin update check (creates Discord forum threads) ---
|
||||
# Filter: only plugins subsystem entries, strip ANSI + multibyte artifacts, deduplicate
|
||||
PLUGIN_UPDATES=$(tail -c 5000000 "$LOG_FILE" 2>/dev/null \
|
||||
| grep '"subsystem.*plugins"' \
|
||||
| grep -oP 'Update available: [^"\\]+' \
|
||||
| sed 's/\x1b\[[0-9;]*m//g; s/\[3[0-9]m//g' \
|
||||
| tr -d '\r' \
|
||||
| sort -u)
|
||||
if [ -n "$PLUGIN_UPDATES" ]; then
|
||||
while IFS= read -r line; do
|
||||
[ -z "$line" ] && continue
|
||||
grep -qF "$line" "$SEEN_FILE" 2>/dev/null && continue
|
||||
|
||||
# Parse: "Update available: 0.4.0 → 0.6.0. Run: openclaw plugins install plugin-name"
|
||||
OLD_VER=$(echo "$line" | grep -oP 'Update available: \K[0-9.]+')
|
||||
NEW_VER=$(echo "$line" | grep -oP '→ \K[0-9.]+')
|
||||
INSTALL_CMD=$(echo "$line" | grep -oP 'Run: \K.*')
|
||||
PLUGIN_NAME=$(echo "$INSTALL_CMD" | grep -oP 'install \K\S+')
|
||||
|
||||
if [ -n "$PLUGIN_NAME" ] && [ -n "$OLD_VER" ] && [ -n "$NEW_VER" ]; then
|
||||
RESULT=$("$SCRIPT_DIR/plugin-update-thread.sh" "$PLUGIN_NAME" "$OLD_VER" "$NEW_VER" "$INSTALL_CMD" 2>/dev/null)
|
||||
if echo "$RESULT" | jq -e '.ok == true' >/dev/null 2>&1; then
|
||||
OUTPUT+="\n📦 Created thread for plugin update: ${PLUGIN_NAME} ${OLD_VER} → ${NEW_VER}\n"
|
||||
else
|
||||
OUTPUT+="\n📦 Plugin update: ${PLUGIN_NAME} ${OLD_VER} → ${NEW_VER} (thread creation failed)\n"
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "$line" >> "$SEEN_FILE"
|
||||
done <<< "$PLUGIN_UPDATES"
|
||||
fi
|
||||
|
||||
# --- Output ---
|
||||
if [ -z "$OUTPUT" ]; then
|
||||
echo "NO_REPLY"
|
||||
else
|
||||
echo -e "$OUTPUT"
|
||||
fi
|
||||
106
workspace-security/vt-monitor/scripts/followup.sh
Executable file
106
workspace-security/vt-monitor/scripts/followup.sh
Executable file
@@ -0,0 +1,106 @@
|
||||
#!/bin/bash
|
||||
# Follow up on pending VT scans — poll VT API, update Discord threads
|
||||
# Usage: followup.sh
|
||||
set -e
|
||||
|
||||
STATE_DIR="/home/node/.openclaw/workspace-security/memory"
|
||||
PENDING_FILE="$STATE_DIR/pending-scans.json"
|
||||
CHANNEL_ID="1470849667737714851"
|
||||
|
||||
# Initialize if missing
|
||||
if [ ! -f "$PENDING_FILE" ] || [ ! -s "$PENDING_FILE" ]; then
|
||||
echo '[]' > "$PENDING_FILE"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
PENDING=$(cat "$PENDING_FILE")
|
||||
COUNT=$(echo "$PENDING" | jq 'length')
|
||||
[ "$COUNT" -eq 0 ] && exit 0
|
||||
|
||||
TOKEN=$(printenv DISCORD_BOT_TOKEN)
|
||||
VT_KEY=$(printenv VIRUSTOTAL_API_KEY)
|
||||
|
||||
if [ -z "$TOKEN" ] || [ -z "$VT_KEY" ]; then
|
||||
echo "Missing DISCORD_BOT_TOKEN or VIRUSTOTAL_API_KEY" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
UPDATED="[]"
|
||||
RESOLVED=0
|
||||
|
||||
for i in $(seq 0 $((COUNT - 1))); do
|
||||
ENTRY=$(echo "$PENDING" | jq ".[$i]")
|
||||
HASH=$(echo "$ENTRY" | jq -r '.hash')
|
||||
THREAD_ID=$(echo "$ENTRY" | jq -r '.threadId')
|
||||
FILENAME=$(echo "$ENTRY" | jq -r '.filename')
|
||||
RISK_CAT=$(echo "$ENTRY" | jq -r '.riskCategory')
|
||||
|
||||
# Query VT API
|
||||
VT_RESULT=$(curl -s --max-time 10 -H "x-apikey: $VT_KEY" \
|
||||
"https://www.virustotal.com/api/v3/files/$HASH" 2>/dev/null)
|
||||
|
||||
STATS=$(echo "$VT_RESULT" | jq '.data.attributes.last_analysis_stats // empty' 2>/dev/null)
|
||||
|
||||
if [ -z "$STATS" ] || [ "$STATS" = "null" ]; then
|
||||
# Still pending or not found — keep in queue
|
||||
UPDATED=$(echo "$UPDATED" | jq --argjson entry "$ENTRY" '. += [$entry]')
|
||||
continue
|
||||
fi
|
||||
|
||||
# Extract verdict
|
||||
MALICIOUS=$(echo "$STATS" | jq '.malicious // 0')
|
||||
SUSPICIOUS=$(echo "$STATS" | jq '.suspicious // 0')
|
||||
UNDETECTED=$(echo "$STATS" | jq '.undetected // 0')
|
||||
TOTAL=$((MALICIOUS + SUSPICIOUS + UNDETECTED))
|
||||
TYPE_DESC=$(echo "$VT_RESULT" | jq -r '.data.attributes.type_description // "Unknown"')
|
||||
SHA256=$(echo "$VT_RESULT" | jq -r '.data.attributes.sha256 // "unknown"')
|
||||
VT_LINK="https://www.virustotal.com/gui/file/$SHA256"
|
||||
|
||||
# Determine verdict
|
||||
if [ "$MALICIOUS" -gt 0 ]; then
|
||||
VERDICT="🚨 MALICIOUS"
|
||||
EMOJI="🚨"
|
||||
VERDICT_SHORT="MALICIOUS ($MALICIOUS/$TOTAL)"
|
||||
elif [ "$SUSPICIOUS" -gt 0 ]; then
|
||||
VERDICT="⚠️ SUSPICIOUS"
|
||||
EMOJI="⚠️"
|
||||
VERDICT_SHORT="SUSPICIOUS ($SUSPICIOUS/$TOTAL)"
|
||||
else
|
||||
VERDICT="✅ CLEAN"
|
||||
EMOJI="✅"
|
||||
VERDICT_SHORT="CLEAN (0/$TOTAL)"
|
||||
fi
|
||||
|
||||
# Post verdict to thread
|
||||
MSG=$(printf '%s **Analysis Complete — %s**\n\n**File:** `%s`\n**Type:** %s\n**SHA-256:** `%s`\n\n**Results:**\n• Malicious: %s\n• Suspicious: %s\n• Undetected: %s engines\n\n**VT Link:** %s\n\n**Verdict:** %s' \
|
||||
"$EMOJI" "$VERDICT_SHORT" "$FILENAME" "$TYPE_DESC" "$SHA256" \
|
||||
"$MALICIOUS" "$SUSPICIOUS" "$UNDETECTED" "$VT_LINK" "$VERDICT")
|
||||
|
||||
curl -s -X POST \
|
||||
-H "Authorization: Bot $TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "$(jq -n --arg content "$MSG" '{content: $content}')" \
|
||||
"https://discord.com/api/v10/channels/$THREAD_ID/messages" > /dev/null
|
||||
|
||||
# Update thread title
|
||||
NEW_TITLE=$(printf '[%s] %s — %s' "$RISK_CAT" "$FILENAME" "$VERDICT")
|
||||
curl -s -X PATCH \
|
||||
-H "Authorization: Bot $TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "$(jq -n --arg name "$NEW_TITLE" '{name: $name}')" \
|
||||
"https://discord.com/api/v10/channels/$THREAD_ID" > /dev/null
|
||||
|
||||
RESOLVED=$((RESOLVED + 1))
|
||||
echo "✅ Resolved: $FILENAME → $VERDICT_SHORT"
|
||||
|
||||
# VT rate limit: 4 req/min on free tier, be conservative
|
||||
sleep 1
|
||||
done
|
||||
|
||||
# Save remaining pending scans
|
||||
echo "$UPDATED" > "$PENDING_FILE"
|
||||
|
||||
if [ "$RESOLVED" -gt 0 ]; then
|
||||
REMAINING=$(echo "$UPDATED" | jq 'length')
|
||||
echo "Resolved $RESOLVED scan(s), $REMAINING still pending"
|
||||
fi
|
||||
51
workspace-security/vt-monitor/scripts/log-splitter.sh
Executable file
51
workspace-security/vt-monitor/scripts/log-splitter.sh
Executable file
@@ -0,0 +1,51 @@
|
||||
#!/bin/bash
|
||||
# VT-Sentinel log splitter — extracts sentinel entries into dedicated log
|
||||
# Usage: bash log-splitter.sh [start|stop|status]
|
||||
set -eo pipefail
|
||||
|
||||
GATEWAY_LOG="/tmp/openclaw/openclaw.log"
|
||||
VT_LOG="/tmp/openclaw/vt-sentinel.log"
|
||||
PID_FILE="/tmp/openclaw/vt-log-splitter.pid"
|
||||
PATTERN="VT-Sentinel\|vt-sentinel\|vtsentinel"
|
||||
|
||||
start() {
|
||||
# Kill existing
|
||||
stop 2>/dev/null || true
|
||||
|
||||
# Start tail from current end of file
|
||||
nohup tail -F "$GATEWAY_LOG" 2>/dev/null \
|
||||
| grep --line-buffered -i "$PATTERN" \
|
||||
>> "$VT_LOG" 2>/dev/null &
|
||||
echo $! > "$PID_FILE"
|
||||
echo "Started (PID $(cat $PID_FILE)), writing to $VT_LOG"
|
||||
}
|
||||
|
||||
stop() {
|
||||
if [ -f "$PID_FILE" ]; then
|
||||
PID=$(cat "$PID_FILE")
|
||||
# Kill the tail pipeline (parent + children)
|
||||
pkill -P "$PID" 2>/dev/null || true
|
||||
kill "$PID" 2>/dev/null || true
|
||||
rm -f "$PID_FILE"
|
||||
echo "Stopped"
|
||||
else
|
||||
echo "Not running"
|
||||
fi
|
||||
}
|
||||
|
||||
status() {
|
||||
if [ -f "$PID_FILE" ] && kill -0 "$(cat $PID_FILE)" 2>/dev/null; then
|
||||
echo "Running (PID $(cat $PID_FILE))"
|
||||
[ -f "$VT_LOG" ] && echo "Log size: $(du -h "$VT_LOG" | cut -f1), $(wc -l < "$VT_LOG") lines"
|
||||
else
|
||||
echo "Not running"
|
||||
rm -f "$PID_FILE" 2>/dev/null
|
||||
fi
|
||||
}
|
||||
|
||||
case "${1:-start}" in
|
||||
start) start ;;
|
||||
stop) stop ;;
|
||||
status) status ;;
|
||||
*) echo "Usage: $0 [start|stop|status]" ;;
|
||||
esac
|
||||
41
workspace-security/vt-monitor/scripts/plugin-update-thread.sh
Executable file
41
workspace-security/vt-monitor/scripts/plugin-update-thread.sh
Executable file
@@ -0,0 +1,41 @@
|
||||
#!/bin/bash
|
||||
# Create Discord forum thread for a plugin update notification
|
||||
# Usage: plugin-update-thread.sh "plugin-name" "old_version" "new_version" "install_cmd"
|
||||
set -e
|
||||
|
||||
PLUGIN="$1"
|
||||
OLD_VER="$2"
|
||||
NEW_VER="$3"
|
||||
INSTALL_CMD="$4"
|
||||
CHANNEL_ID="1470849667737714851"
|
||||
|
||||
TOKEN=$(printenv DISCORD_BOT_TOKEN)
|
||||
if [ -z "$TOKEN" ]; then
|
||||
echo '{"error":"DISCORD_BOT_TOKEN not set"}'
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Build content with real newlines using printf
|
||||
CONTENT=$(printf '📦 **Plugin Update Available**\n\n**Plugin:** `%s`\n**Current:** %s\n**Available:** %s\n\n**Install:**\n```\n%s\n```\n\n---\n*Detected by VT-Sentinel Monitor*' \
|
||||
"$PLUGIN" "$OLD_VER" "$NEW_VER" "$INSTALL_CMD")
|
||||
|
||||
THREAD_NAME=$(printf '📦 %s — %s → %s' "$PLUGIN" "$OLD_VER" "$NEW_VER")
|
||||
|
||||
# Use jq to properly encode the content with real newlines
|
||||
PAYLOAD=$(jq -n \
|
||||
--arg name "$THREAD_NAME" \
|
||||
--arg content "$CONTENT" \
|
||||
'{name: $name, message: {content: $content}, auto_archive_duration: 1440}')
|
||||
|
||||
RESULT=$(curl -s -X POST \
|
||||
-H "Authorization: Bot $TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "$PAYLOAD" \
|
||||
"https://discord.com/api/v10/channels/${CHANNEL_ID}/threads")
|
||||
|
||||
THREAD_ID=$(echo "$RESULT" | jq -r '.id // empty')
|
||||
if [ -n "$THREAD_ID" ]; then
|
||||
echo "{\"ok\":true,\"threadId\":\"$THREAD_ID\",\"plugin\":\"$PLUGIN\"}"
|
||||
else
|
||||
echo "{\"ok\":false,\"error\":$(echo "$RESULT" | jq -c '.')}"
|
||||
fi
|
||||
35
workspace-security/vt-monitor/scripts/scan-thread.sh
Executable file
35
workspace-security/vt-monitor/scripts/scan-thread.sh
Executable file
@@ -0,0 +1,35 @@
|
||||
#!/bin/bash
|
||||
# Create a Discord forum thread for a VT file scan event
|
||||
# Usage: scan-thread.sh "filename" "risk_category" ["hash"]
|
||||
set -e
|
||||
|
||||
FILENAME="$1"
|
||||
RISK_CAT="$2"
|
||||
HASH="${3:-unknown}"
|
||||
CHANNEL_ID="1470849667737714851"
|
||||
|
||||
TOKEN=$(printenv DISCORD_BOT_TOKEN)
|
||||
[ -z "$TOKEN" ] && echo '{"ok":false,"error":"no token"}' && exit 1
|
||||
|
||||
CONTENT=$(printf '🛡️ **VT-Sentinel File Scan**\n\n**File:** `%s`\n**Category:** %s\n**Status:** ⏳ PENDING — uploaded to VirusTotal for analysis\n**Hash:** `%s`\n\n---\n*Will update when verdict is available.*' \
|
||||
"$FILENAME" "$RISK_CAT" "$HASH")
|
||||
|
||||
THREAD_NAME=$(printf '[%s] %s — ⏳ PENDING' "$RISK_CAT" "$FILENAME")
|
||||
|
||||
PAYLOAD=$(jq -n \
|
||||
--arg name "$THREAD_NAME" \
|
||||
--arg content "$CONTENT" \
|
||||
'{name: $name, message: {content: $content}, auto_archive_duration: 1440}')
|
||||
|
||||
RESULT=$(curl -s -X POST \
|
||||
-H "Authorization: Bot $TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "$PAYLOAD" \
|
||||
"https://discord.com/api/v10/channels/${CHANNEL_ID}/threads")
|
||||
|
||||
THREAD_ID=$(echo "$RESULT" | jq -r '.id // empty')
|
||||
if [ -n "$THREAD_ID" ]; then
|
||||
echo "{\"ok\":true,\"threadId\":\"$THREAD_ID\"}"
|
||||
else
|
||||
echo "{\"ok\":false,\"error\":$(echo "$RESULT" | jq -c '.')}"
|
||||
fi
|
||||
20
workspace-security/vt-monitor/scripts/tail.sh
Executable file
20
workspace-security/vt-monitor/scripts/tail.sh
Executable file
@@ -0,0 +1,20 @@
|
||||
#!/bin/bash
|
||||
# Tail recent VT-Sentinel log entries (last N from recent log tail)
|
||||
# Usage: tail.sh [count]
|
||||
set -e
|
||||
|
||||
COUNT="${1:-50}"
|
||||
LOG_FILE="/tmp/openclaw/openclaw.log"
|
||||
|
||||
if [ ! -f "$LOG_FILE" ]; then
|
||||
echo '{"error":"Gateway log not found"}'
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Read last 5MB, grep VT-Sentinel, take last N
|
||||
tail -c 5000000 "$LOG_FILE" | grep '"subsystem.*gateway"' | grep '\[VT-Sentinel\]' | tail -n "$COUNT" | while IFS= read -r line; do
|
||||
TIMESTAMP=$(echo "$line" | grep -oP '"date":"\K[^"]*' | head -1)
|
||||
MESSAGE=$(echo "$line" | grep -oP '\[VT-Sentinel\] \K[^"\\]*' | head -1)
|
||||
[ -n "$TIMESTAMP" ] && [ -n "$MESSAGE" ] && \
|
||||
jq -cn --arg ts "$TIMESTAMP" --arg msg "$MESSAGE" '{"timestamp":$ts,"message":$msg}'
|
||||
done | jq -s '{count:length, entries:.}'
|
||||
67
workspace/capmetro-monitor/SKILL.md
Normal file
67
workspace/capmetro-monitor/SKILL.md
Normal file
@@ -0,0 +1,67 @@
|
||||
---
|
||||
name: capmetro-monitor
|
||||
description: Monitor CapMetro (Austin, TX) service changes for specific routes. Checks tri-annual service change pages for Route 5 (Bus) and Route 500 (MetroRail), translates transit operator language into plain English summaries. Use for weekly monitoring of commute-relevant transit updates.
|
||||
---
|
||||
|
||||
# CapMetro Service Change Monitor
|
||||
|
||||
Weekly monitoring of Austin transit route changes with plain-English summaries.
|
||||
|
||||
## What It Does
|
||||
|
||||
1. Checks CapMetro service change pages (tri-annual: Jan, Jun, Aug)
|
||||
2. Filters for Route 5 (Bus) and Route 500 (MetroRail)
|
||||
3. Detects new changes since last check
|
||||
4. Returns structured JSON for processing
|
||||
|
||||
## Monitored Routes
|
||||
|
||||
- **Route 5** - Woodrow/East 12th (Bus)
|
||||
- **Route 500** - MetroRail (Red Line)
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
bash skills/capmetro-monitor/scripts/check-changes.sh
|
||||
```
|
||||
|
||||
**Output when nothing new:**
|
||||
```json
|
||||
{"hasNew":false}
|
||||
```
|
||||
|
||||
**Output with new changes:**
|
||||
```json
|
||||
{
|
||||
"hasNew": true,
|
||||
"newChanges": [
|
||||
{
|
||||
"url": "https://www.capmetro.org/servicechange/june-2026",
|
||||
"title": "June 2026 Proposed Service Changes",
|
||||
"id": "https://www.capmetro.org/servicechange/june-2026"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Integration
|
||||
|
||||
Designed for weekly cron job that:
|
||||
1. Runs check script
|
||||
2. If `hasNew: true`, fetch full details and summarize in plain English
|
||||
3. Translate transit terminology (timepoint, alignment, turnaround) for clarity
|
||||
|
||||
## State Tracking
|
||||
|
||||
State stored in `memory/capmetro-check-state.json`:
|
||||
```json
|
||||
{
|
||||
"lastCheck": "2026-02-04T17:30:00Z",
|
||||
"seenChanges": ["url1", "url2"]
|
||||
}
|
||||
```
|
||||
|
||||
## Requirements
|
||||
|
||||
- `curl` for web requests
|
||||
- `jq` for JSON processing
|
||||
63
workspace/capmetro-monitor/scripts/check-changes.sh
Executable file
63
workspace/capmetro-monitor/scripts/check-changes.sh
Executable file
@@ -0,0 +1,63 @@
|
||||
#!/bin/bash
|
||||
# Check CapMetro service changes for Route 5 and Route 500
|
||||
# Returns JSON with new changes since last check
|
||||
|
||||
set -e
|
||||
|
||||
STATE_FILE="${STATE_FILE:-memory/capmetro-check-state.json}"
|
||||
WORKSPACE="${WORKSPACE:-/home/node/.openclaw/workspace}"
|
||||
cd "$WORKSPACE"
|
||||
|
||||
# Initialize state if missing
|
||||
if [ ! -f "$STATE_FILE" ]; then
|
||||
mkdir -p "$(dirname "$STATE_FILE")"
|
||||
echo '{"lastCheck":"1970-01-01T00:00:00Z","seenChanges":[]}' > "$STATE_FILE"
|
||||
fi
|
||||
|
||||
NOW=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
|
||||
|
||||
# Fetch current service changes page
|
||||
CHANGES=$(curl -s "https://www.capmetro.org/servicechange" | \
|
||||
grep -oP 'href="/servicechange/[^"]+' | \
|
||||
sed 's/href="//' | \
|
||||
sort -u)
|
||||
|
||||
# Check each change period for Route 5 or Route 500
|
||||
RELEVANT_CHANGES='[]'
|
||||
|
||||
for change_url in $CHANGES; do
|
||||
FULL_URL="https://www.capmetro.org$change_url"
|
||||
CONTENT=$(curl -s "$FULL_URL")
|
||||
|
||||
# Check if Route 5 or Route 500 mentioned
|
||||
if echo "$CONTENT" | grep -qiE "(Route 5[^0-9]|Route 500)"; then
|
||||
TITLE=$(echo "$CONTENT" | grep -oP '<title>\K[^<]+' | head -1)
|
||||
RELEVANT_CHANGES=$(echo "$RELEVANT_CHANGES" | jq --arg url "$FULL_URL" --arg title "$TITLE" \
|
||||
'. += [{"url":$url, "title":$title, "id":$url}]')
|
||||
fi
|
||||
done
|
||||
|
||||
# Load seen changes
|
||||
SEEN=$(jq -r '.seenChanges // []' "$STATE_FILE")
|
||||
|
||||
# Find new changes
|
||||
NEW_CHANGES=$(echo "$RELEVANT_CHANGES" | jq --argjson seen "$SEEN" '[
|
||||
.[] | select(.id as $id | $seen | index($id) | not)
|
||||
]')
|
||||
|
||||
NEW_COUNT=$(echo "$NEW_CHANGES" | jq 'length')
|
||||
|
||||
# Update state
|
||||
ALL_IDS=$(echo "$RELEVANT_CHANGES" | jq -r '[.[].id]')
|
||||
jq -n \
|
||||
--arg now "$NOW" \
|
||||
--argjson ids "$ALL_IDS" \
|
||||
'{lastCheck:$now, seenChanges:$ids}' > "$STATE_FILE"
|
||||
|
||||
# Output
|
||||
if [ "$NEW_COUNT" -eq 0 ]; then
|
||||
echo '{"hasNew":false}'
|
||||
exit 0
|
||||
fi
|
||||
|
||||
jq -n --argjson changes "$NEW_CHANGES" '{hasNew:true, newChanges:$changes}'
|
||||
182
workspace/capmetro-monitor/scripts/monitor-route5.js
Normal file
182
workspace/capmetro-monitor/scripts/monitor-route5.js
Normal file
@@ -0,0 +1,182 @@
|
||||
// Route 5 bus monitor — parses GTFS-RT feed for real-time vehicle positions
|
||||
// Usage: node monitor-route5.js
|
||||
const https = require('https');
|
||||
const fs = require('fs');
|
||||
const protobuf = require('/tmp/gtfs-rt/node_modules/protobufjs');
|
||||
|
||||
const FEED_URL = 'https://data.texas.gov/download/i5qp-g5fd/application/octet-stream';
|
||||
const ROUTE_ID = '5';
|
||||
const FIRST_STOP = '5854'; // Anderson/Northcross
|
||||
const USER_STOP = '964'; // Woodrow/Choquette
|
||||
const TRAVEL_SECS = 446; // 7 min 26 sec from first stop to user stop
|
||||
|
||||
// GTFS-RT proto definition (minimal, inline)
|
||||
const PROTO = `
|
||||
syntax = "proto2";
|
||||
package transit_realtime;
|
||||
|
||||
message FeedMessage {
|
||||
required FeedHeader header = 1;
|
||||
repeated FeedEntity entity = 2;
|
||||
}
|
||||
message FeedHeader {
|
||||
required string gtfs_realtime_version = 1;
|
||||
optional uint64 timestamp = 2;
|
||||
}
|
||||
message FeedEntity {
|
||||
required string id = 1;
|
||||
optional TripUpdate trip_update = 3;
|
||||
optional VehiclePosition vehicle = 4;
|
||||
optional Alert alert = 5;
|
||||
}
|
||||
message TripUpdate {
|
||||
optional TripDescriptor trip = 1;
|
||||
optional VehicleDescriptor vehicle = 3;
|
||||
repeated StopTimeUpdate stop_time_update = 2;
|
||||
optional uint64 timestamp = 4;
|
||||
message StopTimeUpdate {
|
||||
optional uint32 stop_sequence = 1;
|
||||
optional string stop_id = 4;
|
||||
optional StopTimeEvent arrival = 2;
|
||||
optional StopTimeEvent departure = 3;
|
||||
enum ScheduleRelationship { SCHEDULED = 0; SKIPPED = 1; NO_DATA = 2; }
|
||||
optional ScheduleRelationship schedule_relationship = 5;
|
||||
}
|
||||
}
|
||||
message StopTimeEvent {
|
||||
optional int32 delay = 1;
|
||||
optional int64 time = 2;
|
||||
optional int32 uncertainty = 3;
|
||||
}
|
||||
message VehiclePosition {
|
||||
optional TripDescriptor trip = 1;
|
||||
optional VehicleDescriptor vehicle = 8;
|
||||
optional Position position = 2;
|
||||
optional uint32 current_stop_sequence = 3;
|
||||
optional string stop_id = 7;
|
||||
enum VehicleStopStatus { INCOMING_AT = 0; STOPPED_AT = 1; IN_TRANSIT_TO = 2; }
|
||||
optional VehicleStopStatus current_status = 4;
|
||||
optional uint64 timestamp = 5;
|
||||
enum CongestionLevel { UNKNOWN = 0; RUNNING_SMOOTHLY = 1; STOP_AND_GO = 2; CONGESTION = 3; SEVERE_CONGESTION = 4; }
|
||||
optional CongestionLevel congestion_level = 6;
|
||||
enum OccupancyStatus { EMPTY = 0; MANY_SEATS = 1; FEW_SEATS = 2; STANDING_ROOM = 3; CRUSHED = 4; FULL = 5; NOT_ACCEPTING = 6; }
|
||||
optional OccupancyStatus occupancy_status = 9;
|
||||
}
|
||||
message TripDescriptor {
|
||||
optional string trip_id = 1;
|
||||
optional string route_id = 5;
|
||||
optional uint32 direction_id = 6;
|
||||
optional string start_time = 2;
|
||||
optional string start_date = 3;
|
||||
enum ScheduleRelationship { SCHEDULED = 0; ADDED = 1; UNSCHEDULED = 2; CANCELED = 3; }
|
||||
optional ScheduleRelationship schedule_relationship = 4;
|
||||
}
|
||||
message VehicleDescriptor {
|
||||
optional string id = 1;
|
||||
optional string label = 2;
|
||||
optional string license_plate = 3;
|
||||
}
|
||||
message Position {
|
||||
required float latitude = 1;
|
||||
required float longitude = 2;
|
||||
optional float bearing = 3;
|
||||
optional double odometer = 4;
|
||||
optional float speed = 5;
|
||||
}
|
||||
message Alert {
|
||||
repeated TimeRange active_period = 1;
|
||||
repeated EntitySelector informed_entity = 5;
|
||||
optional TranslatedString header_text = 10;
|
||||
optional TranslatedString description_text = 11;
|
||||
}
|
||||
message TimeRange { optional uint64 start = 1; optional uint64 end = 2; }
|
||||
message EntitySelector { optional string agency_id = 1; optional string route_id = 3; optional TripDescriptor trip = 4; optional string stop_id = 6; }
|
||||
message TranslatedString { repeated Translation translation = 1; message Translation { required string text = 1; optional string language = 2; } }
|
||||
`;
|
||||
|
||||
function fetch(url) {
|
||||
return new Promise((resolve, reject) => {
|
||||
https.get(url, (res) => {
|
||||
const chunks = [];
|
||||
res.on('data', c => chunks.push(c));
|
||||
res.on('end', () => resolve(Buffer.concat(chunks)));
|
||||
res.on('error', reject);
|
||||
}).on('error', reject);
|
||||
});
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const root = protobuf.parse(PROTO, { keepCase: true }).root;
|
||||
const FeedMessage = root.lookupType('transit_realtime.FeedMessage');
|
||||
|
||||
const buf = await fetch(FEED_URL);
|
||||
const feed = FeedMessage.decode(buf);
|
||||
|
||||
const now = Math.floor(Date.now() / 1000);
|
||||
const cst = new Date((now - 6*3600) * 1000); // CST offset
|
||||
const timeStr = cst.toISOString().replace('T', ' ').substring(0, 19) + ' CST';
|
||||
|
||||
// Filter Route 5 vehicles
|
||||
const vehicles = feed.entity
|
||||
.filter(e => e.vehicle && e.vehicle.trip && e.vehicle.trip.route_id === ROUTE_ID)
|
||||
.map(e => {
|
||||
const v = e.vehicle;
|
||||
const age = now - (v.timestamp?.low || v.timestamp || 0);
|
||||
const status = ['INCOMING_AT', 'STOPPED_AT', 'IN_TRANSIT_TO'][v.current_status] || 'UNKNOWN';
|
||||
return {
|
||||
vehicleId: v.vehicle?.label || v.vehicle?.id || 'unknown',
|
||||
tripId: v.trip?.trip_id,
|
||||
directionId: v.trip?.direction_id,
|
||||
stopId: v.stop_id,
|
||||
stopSequence: v.current_stop_sequence,
|
||||
status,
|
||||
lat: v.position?.latitude,
|
||||
lon: v.position?.longitude,
|
||||
speed: v.position?.speed,
|
||||
bearing: v.position?.bearing,
|
||||
ageSec: age,
|
||||
timestamp: v.timestamp
|
||||
};
|
||||
});
|
||||
|
||||
// Filter Route 5 trip updates
|
||||
const tripUpdates = feed.entity
|
||||
.filter(e => e.trip_update && e.trip_update.trip && e.trip_update.trip.route_id === ROUTE_ID)
|
||||
.map(e => {
|
||||
const tu = e.trip_update;
|
||||
const userStopUpdate = tu.stop_time_update?.find(s => s.stop_id === USER_STOP);
|
||||
const firstStopUpdate = tu.stop_time_update?.find(s => s.stop_id === FIRST_STOP);
|
||||
return {
|
||||
tripId: tu.trip?.trip_id,
|
||||
directionId: tu.trip?.direction_id,
|
||||
vehicleId: tu.vehicle?.label || tu.vehicle?.id,
|
||||
userStopDelay: userStopUpdate?.departure?.delay || userStopUpdate?.arrival?.delay || null,
|
||||
userStopTime: userStopUpdate?.arrival?.time || userStopUpdate?.departure?.time || null,
|
||||
firstStopDelay: firstStopUpdate?.departure?.delay || null,
|
||||
firstStopTime: firstStopUpdate?.departure?.time || null,
|
||||
totalStops: tu.stop_time_update?.length || 0
|
||||
};
|
||||
});
|
||||
|
||||
// Eastbound (direction 0) only
|
||||
const ebVehicles = vehicles.filter(v => v.directionId === 0);
|
||||
const ebUpdates = tripUpdates.filter(t => t.directionId === 0);
|
||||
|
||||
const result = {
|
||||
timestamp: timeStr,
|
||||
feedTimestamp: feed.header?.timestamp?.toString(),
|
||||
route5_eastbound: {
|
||||
activeVehicles: ebVehicles.length,
|
||||
vehicles: ebVehicles,
|
||||
tripUpdates: ebUpdates.filter(t => t.userStopDelay !== null || t.userStopTime !== null)
|
||||
},
|
||||
route5_all: {
|
||||
totalVehicles: vehicles.length,
|
||||
totalTripUpdates: tripUpdates.length
|
||||
}
|
||||
};
|
||||
|
||||
console.log(JSON.stringify(result, null, 2));
|
||||
}
|
||||
|
||||
main().catch(e => console.error(JSON.stringify({ error: e.message })));
|
||||
89
workspace/capmetro-monitor/scripts/monitor.sh
Executable file
89
workspace/capmetro-monitor/scripts/monitor.sh
Executable file
@@ -0,0 +1,89 @@
|
||||
#!/bin/bash
|
||||
# Route 5 smart monitor — determines direction by time of day, launches background watcher
|
||||
# Usage: bash monitor.sh [channel_id]
|
||||
# Before 11 AM CST → Eastbound (morning commute)
|
||||
# After 11 AM CST → Westbound (evening commute)
|
||||
set -eo pipefail
|
||||
|
||||
CHANNEL="${1:-1467247377743347953}"
|
||||
TU_URL="https://data.texas.gov/download/mqtr-wwpy/application%2Fjson"
|
||||
|
||||
STATE_DIR="/home/node/.openclaw/workspace/skills/capmetro-monitor/memory"
|
||||
mkdir -p "$STATE_DIR"
|
||||
|
||||
NOW=$(date +%s)
|
||||
CST_HOUR=$(TZ=America/Chicago date +%H)
|
||||
|
||||
if [ "$CST_HOUR" -lt 11 ]; then
|
||||
DIRECTION=0
|
||||
DIR_NAME="Eastbound"
|
||||
FIRST_STOP="5854"
|
||||
FIRST_STOP_NAME="Anderson/Northcross"
|
||||
USER_STOP="964"
|
||||
USER_STOP_NAME="Woodrow/Choquette"
|
||||
TRAVEL_FIRST_TO_USER=446 # 7m26s
|
||||
WALK_LEAD=0 # already near stop
|
||||
else
|
||||
DIRECTION=1
|
||||
DIR_NAME="Westbound"
|
||||
FIRST_STOP="4606"
|
||||
FIRST_STOP_NAME="Tannehill/Webberville"
|
||||
USER_STOP="5499"
|
||||
USER_STOP_NAME="6th/West"
|
||||
TRAVEL_FIRST_TO_USER=2384 # 39m44s
|
||||
WALK_LEAD=900 # 15 min walk from office
|
||||
HOME_STOP="1072"
|
||||
HOME_STOP_NAME="Woodrow/Dwyce"
|
||||
TRAVEL_USER_TO_HOME=1344 # 22m24s
|
||||
fi
|
||||
|
||||
# Find next departure from first stop
|
||||
TU=$(curl -sL --max-time 10 "$TU_URL" 2>/dev/null)
|
||||
NEXT=$(echo "$TU" | jq --arg dir "$DIRECTION" --arg fs "$FIRST_STOP" --arg now "$NOW" '
|
||||
[.entity[] |
|
||||
select(.tripUpdate.trip.routeId == "5" and (.tripUpdate.trip.directionId | tostring) == $dir) |
|
||||
{
|
||||
tripId: .tripUpdate.trip.tripId,
|
||||
depart: ([.tripUpdate.stopTimeUpdate[] | select(.stopId == $fs) | (.departure.time // .arrival.time)] | .[0])
|
||||
}
|
||||
] | [.[] | select(.depart != null and (.depart | tonumber) > ($now | tonumber))]
|
||||
| sort_by(.depart | tonumber) | .[0]' 2>/dev/null)
|
||||
|
||||
TRIP_ID=$(echo "$NEXT" | jq -r '.tripId // empty')
|
||||
DEPART_TS=$(echo "$NEXT" | jq -r '.depart // empty')
|
||||
|
||||
if [ -z "$TRIP_ID" ] || [ -z "$DEPART_TS" ]; then
|
||||
echo '{"ok":false,"error":"No upcoming Route 5 departures found"}'
|
||||
exit 1
|
||||
fi
|
||||
|
||||
DEPART_CST=$(TZ=America/Chicago date -d "@$DEPART_TS" +"%I:%M %p" 2>/dev/null)
|
||||
MINS_AWAY=$(( (DEPART_TS - NOW) / 60 ))
|
||||
|
||||
# Export config for the watcher
|
||||
export DIRECTION DIR_NAME FIRST_STOP FIRST_STOP_NAME USER_STOP USER_STOP_NAME
|
||||
export TRAVEL_FIRST_TO_USER WALK_LEAD TRIP_ID DEPART_TS CHANNEL
|
||||
export HOME_STOP HOME_STOP_NAME TRAVEL_USER_TO_HOME
|
||||
|
||||
echo "{\"ok\":true,\"direction\":\"$DIR_NAME\",\"tripId\":\"$TRIP_ID\",\"firstStopDepart\":\"$DEPART_CST\",\"minsUntilDepart\":$MINS_AWAY,\"userStop\":\"$USER_STOP_NAME\"}"
|
||||
|
||||
# Kill any existing watcher
|
||||
PID_FILE="$STATE_DIR/watcher.pid"
|
||||
if [ -f "$PID_FILE" ]; then
|
||||
OLD_PID=$(cat "$PID_FILE" 2>/dev/null)
|
||||
if [ -n "$OLD_PID" ] && kill -0 "$OLD_PID" 2>/dev/null; then
|
||||
kill "$OLD_PID" 2>/dev/null || true
|
||||
fi
|
||||
rm -f "$PID_FILE"
|
||||
fi
|
||||
|
||||
# Dynamic timeout: time until departure + 5 min buffer for delays
|
||||
WAIT_SECS=$(( DEPART_TS - NOW + 300 ))
|
||||
[ "$WAIT_SECS" -lt 900 ] && WAIT_SECS=900 # minimum 15 min
|
||||
|
||||
# Calculate MAX_POLLS for the watcher (poll every 20s)
|
||||
export MAX_POLLS=$(( WAIT_SECS / 20 ))
|
||||
|
||||
SCRIPT_DIR="$(dirname "$0")"
|
||||
nohup timeout "$WAIT_SECS" bash "$SCRIPT_DIR/watch-departure-v2.sh" > /dev/null 2>&1 &
|
||||
echo $! > "$PID_FILE"
|
||||
58
workspace/capmetro-monitor/scripts/route5-status.sh
Executable file
58
workspace/capmetro-monitor/scripts/route5-status.sh
Executable file
@@ -0,0 +1,58 @@
|
||||
#!/bin/bash
|
||||
# Route 5 on-demand monitor — checks real-time bus status
|
||||
# Usage: bash route5-status.sh
|
||||
set -eo pipefail
|
||||
|
||||
VP_URL="https://data.texas.gov/download/cuc7-ywmd/application%2Fjson"
|
||||
TU_URL="https://data.texas.gov/download/mqtr-wwpy/application%2Fjson"
|
||||
|
||||
# Fetch both feeds in parallel
|
||||
VP=$(curl -sL --max-time 10 "$VP_URL") &
|
||||
TU=$(curl -sL --max-time 10 "$TU_URL") &
|
||||
VP=$(curl -sL --max-time 10 "$VP_URL")
|
||||
TU=$(curl -sL --max-time 10 "$TU_URL")
|
||||
|
||||
NOW=$(date +%s)
|
||||
|
||||
# Route 5 eastbound vehicles
|
||||
VEHICLES=$(echo "$VP" | jq -c '[.entity[] | select(.vehicle.trip.routeId == "5" and .vehicle.trip.directionId == 0) | {
|
||||
vehicleId: .vehicle.vehicle.label,
|
||||
tripId: .vehicle.trip.tripId,
|
||||
stopId: .vehicle.stopId,
|
||||
status: .vehicle.currentStatus,
|
||||
lat: .vehicle.position.latitude,
|
||||
lon: .vehicle.position.longitude,
|
||||
speed: .vehicle.position.speed
|
||||
}]')
|
||||
|
||||
# Route 5 eastbound trip updates
|
||||
TRIPS=$(echo "$TU" | jq -c --arg now "$NOW" '[.entity[] | select(.tripUpdate.trip.routeId == "5" and .tripUpdate.trip.directionId == 0) | {
|
||||
tripId: .tripUpdate.trip.tripId,
|
||||
firstStopDepart: ([.tripUpdate.stopTimeUpdate[] | select(.stopId == "5854") | (.departure.time // .arrival.time)] | .[0]),
|
||||
userStopArrive: ([.tripUpdate.stopTimeUpdate[] | select(.stopId == "964") | (.arrival.time // .departure.time)] | .[0]),
|
||||
userStopDelay: ([.tripUpdate.stopTimeUpdate[] | select(.stopId == "964") | (.arrival.delay // .departure.delay)] | .[0])
|
||||
}] | [.[] | select(.firstStopDepart != null)] | sort_by(.firstStopDepart)')
|
||||
|
||||
# Format output
|
||||
jq -n \
|
||||
--argjson vehicles "$VEHICLES" \
|
||||
--argjson trips "$TRIPS" \
|
||||
--arg now "$NOW" '{
|
||||
timestampUTC: ($now | tonumber | todate),
|
||||
timestampCST: (($now | tonumber - 21600) | todate),
|
||||
route: "5 - Woodrow/Lamar",
|
||||
direction: "Eastbound → Downtown",
|
||||
firstStop: "Anderson/Northcross (5854)",
|
||||
userStop: "Woodrow/Choquette (964)",
|
||||
scheduledTravel: "7m 26s",
|
||||
activeVehicles: ($vehicles | length),
|
||||
vehicles: $vehicles,
|
||||
nextDepartures: [$trips[] | {
|
||||
tripId,
|
||||
firstStopDepart: (if .firstStopDepart then (.firstStopDepart | tonumber | todate) else null end),
|
||||
userStopArrive: (if .userStopArrive then (.userStopArrive | tonumber | todate) else null end),
|
||||
delaySec: .userStopDelay,
|
||||
minsUntilDepart: (if .firstStopDepart then (((.firstStopDepart | tonumber) - ($now | tonumber)) / 60 | floor) else null end),
|
||||
minsUntilArrive: (if .userStopArrive then (((.userStopArrive | tonumber) - ($now | tonumber)) / 60 | floor) else null end)
|
||||
}]
|
||||
}'
|
||||
84
workspace/capmetro-monitor/scripts/watch-departure-v2.sh
Executable file
84
workspace/capmetro-monitor/scripts/watch-departure-v2.sh
Executable file
@@ -0,0 +1,84 @@
|
||||
#!/bin/bash
|
||||
# Route 5 departure watcher v2 — direction-aware, runs in background
|
||||
# Called by monitor.sh with env vars set
|
||||
set -eo pipefail
|
||||
|
||||
VP_URL="https://data.texas.gov/download/cuc7-ywmd/application%2Fjson"
|
||||
TOKEN=$(printenv DISCORD_BOT_TOKEN)
|
||||
MAX_POLLS=${MAX_POLLS:-40}
|
||||
POLL_INTERVAL=20
|
||||
PID_FILE="/home/node/.openclaw/workspace/skills/capmetro-monitor/memory/watcher.pid"
|
||||
|
||||
# Clean up PID file on exit (success, timeout, or signal)
|
||||
cleanup() { rm -f "$PID_FILE" 2>/dev/null; }
|
||||
trap cleanup EXIT INT TERM
|
||||
|
||||
for i in $(seq 1 $MAX_POLLS); do
|
||||
VP=$(curl -sL --max-time 8 "$VP_URL" 2>/dev/null)
|
||||
|
||||
VEHICLE=$(echo "$VP" | jq -c ".entity[] | select(.vehicle.trip.tripId == \"$TRIP_ID\")" 2>/dev/null)
|
||||
|
||||
if [ -z "$VEHICLE" ]; then
|
||||
sleep $POLL_INTERVAL
|
||||
continue
|
||||
fi
|
||||
|
||||
STOP_ID=$(echo "$VEHICLE" | jq -r '.vehicle.stopId')
|
||||
STATUS=$(echo "$VEHICLE" | jq -r '.vehicle.currentStatus')
|
||||
VEH_ID=$(echo "$VEHICLE" | jq -r '.vehicle.vehicle.label')
|
||||
|
||||
# Bus has left the first stop
|
||||
if [ "$STOP_ID" != "$FIRST_STOP" ] || { [ "$STATUS" = "IN_TRANSIT_TO" ] && [ "$STOP_ID" != "$FIRST_STOP" ]; }; then
|
||||
ACTUAL_TS=$(date +%s)
|
||||
ACTUAL_CST=$(TZ=America/Chicago date +"%I:%M %p")
|
||||
DELAY=$((ACTUAL_TS - DEPART_TS))
|
||||
DELAY_MIN=$((DELAY / 60))
|
||||
|
||||
if [ "$DELAY_MIN" -le 0 ]; then
|
||||
STATUS_ICON="🟢"
|
||||
STATUS_TEXT="On time"
|
||||
elif [ "$DELAY_MIN" -le 2 ]; then
|
||||
STATUS_ICON="🟡"
|
||||
STATUS_TEXT="~${DELAY_MIN}min late"
|
||||
else
|
||||
STATUS_ICON="🔴"
|
||||
STATUS_TEXT="${DELAY_MIN}min late"
|
||||
fi
|
||||
|
||||
# Calculate ETAs
|
||||
ETA_USER=$((ACTUAL_TS + TRAVEL_FIRST_TO_USER))
|
||||
ETA_USER_CST=$(TZ=America/Chicago date -d "@$ETA_USER" +"%I:%M %p")
|
||||
|
||||
if [ "$DIRECTION" = "0" ]; then
|
||||
# EASTBOUND: simple alert
|
||||
MSG=$(printf '🚌 **Route 5 %s Departed!**\nBus %s left %s at %s\n%s %s\n📍 ETA at %s: **%s**' \
|
||||
"$DIR_NAME" "$VEH_ID" "$FIRST_STOP_NAME" "$ACTUAL_CST" \
|
||||
"$STATUS_ICON" "$STATUS_TEXT" "$USER_STOP_NAME" "$ETA_USER_CST")
|
||||
else
|
||||
# WESTBOUND: include leave-office time and home ETA
|
||||
LEAVE_TS=$((ETA_USER - WALK_LEAD))
|
||||
LEAVE_CST=$(TZ=America/Chicago date -d "@$LEAVE_TS" +"%I:%M %p")
|
||||
ETA_HOME=$((ACTUAL_TS + TRAVEL_FIRST_TO_USER + TRAVEL_USER_TO_HOME))
|
||||
ETA_HOME_CST=$(TZ=America/Chicago date -d "@$ETA_HOME" +"%I:%M %p")
|
||||
|
||||
MSG=$(printf '🚌 **Route 5 %s Departed!**\nBus %s left %s at %s\n%s %s\n\n🚶 **Leave office by %s** (15 min walk)\n📍 Bus arrives %s: **%s**\n🏠 Home (%s): **%s**' \
|
||||
"$DIR_NAME" "$VEH_ID" "$FIRST_STOP_NAME" "$ACTUAL_CST" \
|
||||
"$STATUS_ICON" "$STATUS_TEXT" \
|
||||
"$LEAVE_CST" "$USER_STOP_NAME" "$ETA_USER_CST" \
|
||||
"$HOME_STOP_NAME" "$ETA_HOME_CST")
|
||||
fi
|
||||
|
||||
curl -s -X POST -H "Authorization: Bot $TOKEN" -H "Content-Type: application/json" \
|
||||
-d "$(jq -n --arg c "$MSG" '{content: $c}')" \
|
||||
"https://discord.com/api/v10/channels/$CHANNEL/messages" > /dev/null
|
||||
exit 0
|
||||
fi
|
||||
|
||||
sleep $POLL_INTERVAL
|
||||
done
|
||||
|
||||
# Timeout
|
||||
SCHED_CST=$(TZ=America/Chicago date -d "@$DEPART_TS" +"%I:%M %p" 2>/dev/null)
|
||||
curl -s -X POST -H "Authorization: Bot $TOKEN" -H "Content-Type: application/json" \
|
||||
-d "{\"content\":\"⚠️ Route 5 $DIR_NAME watcher timed out — could not confirm $SCHED_CST departure.\"}" \
|
||||
"https://discord.com/api/v10/channels/$CHANNEL/messages" > /dev/null
|
||||
103
workspace/capmetro-monitor/scripts/watch-departure.sh
Executable file
103
workspace/capmetro-monitor/scripts/watch-departure.sh
Executable file
@@ -0,0 +1,103 @@
|
||||
#!/bin/bash
|
||||
# Route 5 departure watcher — runs in background, posts to Discord when bus departs
|
||||
# Usage: bash watch-departure.sh <scheduled_time_UTC> [channel_id]
|
||||
# Example: bash watch-departure.sh "2026-02-12T13:30:00Z" 1467247377743347953
|
||||
set -eo pipefail
|
||||
|
||||
SCHED_DEPART="$1"
|
||||
CHANNEL="${2:-1467247377743347953}" # Default: DM channel
|
||||
VP_URL="https://data.texas.gov/download/cuc7-ywmd/application%2Fjson"
|
||||
FIRST_STOP="5854"
|
||||
USER_STOP="964"
|
||||
TOKEN=$(printenv DISCORD_BOT_TOKEN)
|
||||
MAX_POLLS=40 # ~13 minutes max watch time
|
||||
POLL_INTERVAL=20 # seconds between polls
|
||||
|
||||
# Find the trip matching this scheduled departure
|
||||
find_trip() {
|
||||
local TU=$(curl -sL --max-time 10 "https://data.texas.gov/download/mqtr-wwpy/application%2Fjson" 2>/dev/null)
|
||||
echo "$TU" | jq -r --arg sched "$SCHED_DEPART" '.entity[] |
|
||||
select(.tripUpdate.trip.routeId == "5" and .tripUpdate.trip.directionId == 0) |
|
||||
select([.tripUpdate.stopTimeUpdate[] | select(.stopId == "5854") |
|
||||
((.departure.time // .arrival.time) | tonumber)] | .[0] == ($sched | sub("Z$";"") | strptime("%Y-%m-%dT%H:%M:%S") | mktime)) |
|
||||
.tripUpdate.trip.tripId' 2>/dev/null | head -1
|
||||
}
|
||||
|
||||
# Convert ISO to epoch
|
||||
sched_epoch() {
|
||||
date -d "$SCHED_DEPART" +%s 2>/dev/null || date -u -d "${SCHED_DEPART%Z}" +%s 2>/dev/null
|
||||
}
|
||||
|
||||
SCHED_TS=$(sched_epoch)
|
||||
SCHED_CST=$(TZ=America/Chicago date -d "@$SCHED_TS" +"%I:%M %p" 2>/dev/null)
|
||||
|
||||
# Find the trip ID
|
||||
TRIP_ID=$(find_trip)
|
||||
if [ -z "$TRIP_ID" ]; then
|
||||
# Fallback: find closest eastbound trip
|
||||
TU=$(curl -sL --max-time 10 "https://data.texas.gov/download/mqtr-wwpy/application%2Fjson" 2>/dev/null)
|
||||
TRIP_ID=$(echo "$TU" | jq -r --arg ts "$SCHED_TS" '[.entity[] |
|
||||
select(.tripUpdate.trip.routeId == "5" and .tripUpdate.trip.directionId == 0) |
|
||||
{tripId: .tripUpdate.trip.tripId, depart: ([.tripUpdate.stopTimeUpdate[] | select(.stopId == "5854") | (.departure.time // .arrival.time)] | .[0] | tonumber)}] |
|
||||
sort_by((. .depart - ($ts | tonumber)) | fabs) | .[0].tripId' 2>/dev/null)
|
||||
fi
|
||||
|
||||
if [ -z "$TRIP_ID" ]; then
|
||||
# Post error
|
||||
curl -s -X POST -H "Authorization: Bot $TOKEN" -H "Content-Type: application/json" \
|
||||
-d "{\"content\":\"⚠️ Could not find Route 5 trip for $SCHED_CST departure.\"}" \
|
||||
"https://discord.com/api/v10/channels/$CHANNEL/messages" > /dev/null
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Poll until departure
|
||||
for i in $(seq 1 $MAX_POLLS); do
|
||||
VP=$(curl -sL --max-time 8 "https://data.texas.gov/download/cuc7-ywmd/application%2Fjson" 2>/dev/null)
|
||||
|
||||
VEHICLE=$(echo "$VP" | jq -c ".entity[] | select(.vehicle.trip.tripId == \"$TRIP_ID\")" 2>/dev/null)
|
||||
|
||||
if [ -z "$VEHICLE" ]; then
|
||||
sleep $POLL_INTERVAL
|
||||
continue
|
||||
fi
|
||||
|
||||
STOP_ID=$(echo "$VEHICLE" | jq -r '.vehicle.stopId')
|
||||
STATUS=$(echo "$VEHICLE" | jq -r '.vehicle.currentStatus')
|
||||
SPEED=$(echo "$VEHICLE" | jq -r '.vehicle.position.speed')
|
||||
VEH_ID=$(echo "$VEHICLE" | jq -r '.vehicle.vehicle.label')
|
||||
|
||||
# Bus has left the first stop
|
||||
if [ "$STOP_ID" != "$FIRST_STOP" ] || ([ "$STATUS" = "IN_TRANSIT_TO" ] && [ "$STOP_ID" != "$FIRST_STOP" ]); then
|
||||
DEPART_TS=$(date +%s)
|
||||
DEPART_CST=$(TZ=America/Chicago date +"%I:%M:%S %p")
|
||||
DELAY=$((DEPART_TS - SCHED_TS))
|
||||
DELAY_MIN=$((DELAY / 60))
|
||||
|
||||
# Calculate ETA at user stop (7m26s from first stop)
|
||||
ETA_TS=$((DEPART_TS + 446))
|
||||
ETA_CST=$(TZ=America/Chicago date -d "@$ETA_TS" +"%I:%M %p" 2>/dev/null)
|
||||
|
||||
if [ "$DELAY_MIN" -le 0 ]; then
|
||||
STATUS_MSG="🟢 On time"
|
||||
elif [ "$DELAY_MIN" -le 2 ]; then
|
||||
STATUS_MSG="🟡 ~${DELAY_MIN}min late"
|
||||
else
|
||||
STATUS_MSG="🔴 ${DELAY_MIN}min late"
|
||||
fi
|
||||
|
||||
MSG="🚌 **Route 5 Departed!**\nBus ${VEH_ID} left Anderson/Northcross at ${DEPART_CST}\nScheduled: ${SCHED_CST} | ${STATUS_MSG}\n📍 ETA at Woodrow/Choquette: **${ETA_CST}**"
|
||||
|
||||
CONTENT=$(printf "$MSG")
|
||||
curl -s -X POST -H "Authorization: Bot $TOKEN" -H "Content-Type: application/json" \
|
||||
-d "$(jq -n --arg c "$CONTENT" '{content: $c}')" \
|
||||
"https://discord.com/api/v10/channels/$CHANNEL/messages" > /dev/null
|
||||
exit 0
|
||||
fi
|
||||
|
||||
sleep $POLL_INTERVAL
|
||||
done
|
||||
|
||||
# Timeout — bus never departed (or we missed it)
|
||||
curl -s -X POST -H "Authorization: Bot $TOKEN" -H "Content-Type: application/json" \
|
||||
-d "{\"content\":\"⚠️ Route 5 watcher timed out — could not confirm $SCHED_CST departure from Anderson/Northcross.\"}" \
|
||||
"https://discord.com/api/v10/channels/$CHANNEL/messages" > /dev/null
|
||||
127
workspace/github-notifications/SKILL.md
Normal file
127
workspace/github-notifications/SKILL.md
Normal file
@@ -0,0 +1,127 @@
|
||||
---
|
||||
name: github-notifications
|
||||
description: Check GitHub notifications for PR activity and major releases. Filters for PRs where user is mentioned/author, and major releases (v*.0.0) plus all Mirrowel/LLM-API-Key-Proxy dev builds. Tracks state to avoid duplicate alerts. Use for periodic GitHub notification checking via cron jobs or manual checks.
|
||||
---
|
||||
|
||||
# GitHub Notifications Checker
|
||||
|
||||
Efficiently check GitHub notifications with smart filtering and state tracking.
|
||||
|
||||
## What It Does
|
||||
|
||||
1. **Fetches notifications** via GitHub CLI (`gh api`)
|
||||
2. **Filters intelligently:**
|
||||
- PRs where you're mentioned, author, or review requested
|
||||
- Major releases (v*.0.0 format)
|
||||
- ALL releases from `Mirrowel/LLM-API-Key-Proxy` (including dev builds)
|
||||
- Excludes: rc/pre/beta/alpha/nightly releases
|
||||
3. **Tracks state** to avoid duplicate notifications
|
||||
4. **Returns JSON** for easy parsing
|
||||
|
||||
## Usage
|
||||
|
||||
### Basic Check
|
||||
|
||||
```bash
|
||||
bash skills/github-notifications/scripts/check.sh
|
||||
```
|
||||
|
||||
**Output when nothing new:**
|
||||
```json
|
||||
{"hasNew":false}
|
||||
```
|
||||
|
||||
**Output with new activity:**
|
||||
```json
|
||||
{
|
||||
"hasNew": true,
|
||||
"newPRs": [
|
||||
{
|
||||
"repo": "openclaw/openclaw",
|
||||
"title": "feat: Add cron silent mode",
|
||||
"url": "https://api.github.com/repos/openclaw/openclaw/pulls/1234",
|
||||
"updated": "2026-02-03T14:30:00Z",
|
||||
"reason": "mention",
|
||||
"id": "openclaw/openclaw#feat: Add cron silent mode"
|
||||
}
|
||||
],
|
||||
"newReleases": [
|
||||
{
|
||||
"repo": "some/repo",
|
||||
"title": "v2.0.0",
|
||||
"updated": "2026-02-03T12:00:00Z",
|
||||
"id": "some/repo@v2.0.0"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
- `STATE_FILE` - Path to state tracking file (default: `memory/github-check-state.json`)
|
||||
- `WORKSPACE` - Workspace directory (default: `/home/node/.openclaw/workspace`)
|
||||
|
||||
### State Tracking
|
||||
|
||||
State is stored in `memory/github-check-state.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"lastCheck": "2026-02-03T14:00:00Z",
|
||||
"seenPRs": ["repo#PR Title", ...],
|
||||
"seenReleases": ["repo@v1.0.0", ...]
|
||||
}
|
||||
```
|
||||
|
||||
## Integration with Cron
|
||||
|
||||
This skill is designed to work with OpenClaw cron jobs. The script handles all filtering and state management, only calling the LLM when there's actual content to summarize.
|
||||
|
||||
**Recommended cron setup:**
|
||||
|
||||
1. Script runs periodically (every 4 hours)
|
||||
2. If `hasNew: false`, script exits silently - no LLM call, no message
|
||||
3. If `hasNew: true`, cron job can format the summary and deliver it
|
||||
|
||||
This approach:
|
||||
- ✅ Saves tokens (no LLM call when nothing new)
|
||||
- ✅ Handles errors gracefully (GitHub API failures logged)
|
||||
- ✅ Avoids duplicate notifications (state tracking)
|
||||
- ✅ Faster execution (no LLM parsing)
|
||||
|
||||
## Error Handling
|
||||
|
||||
If GitHub API fails, returns:
|
||||
```json
|
||||
{
|
||||
"error": "GitHub API failed",
|
||||
"details": "..."
|
||||
}
|
||||
```
|
||||
|
||||
Check for `.error` field in output to detect failures.
|
||||
|
||||
## Auto-Dismiss Low-Value Notifications
|
||||
|
||||
```bash
|
||||
# Dry run (see what would be dismissed)
|
||||
DRY_RUN=true bash skills/github-notifications/scripts/auto-dismiss.sh
|
||||
|
||||
# Actually dismiss
|
||||
bash skills/github-notifications/scripts/auto-dismiss.sh
|
||||
```
|
||||
|
||||
**Auto-dismisses:**
|
||||
- Title matches: nightly, preview, checkpoint, pre-release, canary, alpha, beta, snapshot
|
||||
- Releases with empty release notes
|
||||
|
||||
**Output:**
|
||||
```json
|
||||
{"dismissed":3,"checked":12}
|
||||
```
|
||||
|
||||
## Requirements
|
||||
|
||||
- `gh` CLI authenticated
|
||||
- `jq` for JSON parsing
|
||||
- GitHub token with `notifications` scope
|
||||
74
workspace/github-notifications/scripts/auto-dismiss.sh
Executable file
74
workspace/github-notifications/scripts/auto-dismiss.sh
Executable file
@@ -0,0 +1,74 @@
|
||||
#!/bin/bash
|
||||
# Auto-dismiss GitHub notifications matching certain patterns
|
||||
# - Nightlies, previews, checkpoints, rc, etc (by title pattern)
|
||||
# - Releases with no release notes
|
||||
# Exempts specified repos from auto-dismiss
|
||||
#
|
||||
# Dismiss = PATCH (read) + DELETE thread + DELETE subscription
|
||||
|
||||
set -e
|
||||
|
||||
DRY_RUN="${DRY_RUN:-false}"
|
||||
|
||||
# Repos exempt from auto-dismiss (always show these)
|
||||
EXEMPT_REPOS="Mirrowel/LLM-API-Key-Proxy|b3nw/LLM-API-Key-Proxy|pedramamini/Maestro"
|
||||
|
||||
# Patterns to auto-dismiss (case-insensitive)
|
||||
DISMISS_PATTERNS="nightly|preview|checkpoint|pre-release|canary|alpha|beta|snapshot|-rc\.|rc[0-9]"
|
||||
|
||||
# Get all unread notifications
|
||||
NOTIFICATIONS=$(gh api /notifications 2>/dev/null || echo "[]")
|
||||
|
||||
if [ "$NOTIFICATIONS" = "[]" ] || [ -z "$NOTIFICATIONS" ]; then
|
||||
echo '{"dismissed":0,"checked":0}'
|
||||
exit 0
|
||||
fi
|
||||
|
||||
DISMISSED=0
|
||||
TOTAL=$(echo "$NOTIFICATIONS" | jq 'length')
|
||||
|
||||
echo "$NOTIFICATIONS" | jq -c '.[]' | while read -r notif; do
|
||||
ID=$(echo "$notif" | jq -r '.id')
|
||||
TITLE=$(echo "$notif" | jq -r '.subject.title')
|
||||
TYPE=$(echo "$notif" | jq -r '.subject.type')
|
||||
URL=$(echo "$notif" | jq -r '.subject.url')
|
||||
REPO=$(echo "$notif" | jq -r '.repository.full_name')
|
||||
|
||||
# Skip exempt repos
|
||||
if echo "$REPO" | grep -qiE "$EXEMPT_REPOS"; then
|
||||
continue
|
||||
fi
|
||||
|
||||
SHOULD_DISMISS=false
|
||||
REASON=""
|
||||
|
||||
# Check title patterns
|
||||
if echo "$TITLE" | grep -qiE "$DISMISS_PATTERNS"; then
|
||||
SHOULD_DISMISS=true
|
||||
REASON="title_pattern"
|
||||
fi
|
||||
|
||||
# Check releases with no notes
|
||||
if [ "$TYPE" = "Release" ] && [ "$SHOULD_DISMISS" = "false" ]; then
|
||||
RELEASE_BODY=$(gh api "$URL" --jq '.body // ""' 2>/dev/null || echo "")
|
||||
if [ -z "$RELEASE_BODY" ] || [ "$RELEASE_BODY" = "null" ]; then
|
||||
SHOULD_DISMISS=true
|
||||
REASON="empty_release_notes"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "$SHOULD_DISMISS" = "true" ]; then
|
||||
if [ "$DRY_RUN" = "true" ]; then
|
||||
echo "Would dismiss: [$REPO] $TITLE ($REASON)" >&2
|
||||
else
|
||||
# Full dismiss: mark read + delete thread + delete subscription
|
||||
gh api -X PATCH "/notifications/threads/$ID" 2>/dev/null || true
|
||||
gh api -X DELETE "/notifications/threads/$ID" 2>/dev/null || true
|
||||
gh api -X DELETE "/notifications/threads/$ID/subscription" 2>/dev/null || true
|
||||
echo "Dismissed: [$REPO] $TITLE ($REASON)" >&2
|
||||
fi
|
||||
DISMISSED=$((DISMISSED + 1))
|
||||
fi
|
||||
done
|
||||
|
||||
echo "{\"dismissed\":$DISMISSED,\"checked\":$TOTAL}"
|
||||
104
workspace/github-notifications/scripts/check.sh
Executable file
104
workspace/github-notifications/scripts/check.sh
Executable file
@@ -0,0 +1,104 @@
|
||||
#!/bin/bash
|
||||
# GitHub Notifications Checker
|
||||
# Filters PRs and releases, tracks state, returns JSON summary
|
||||
|
||||
set -e
|
||||
|
||||
STATE_FILE="${STATE_FILE:-memory/github-check-state.json}"
|
||||
WORKSPACE="${WORKSPACE:-/home/node/.openclaw/workspace}"
|
||||
cd "$WORKSPACE"
|
||||
|
||||
# Initialize state file if missing
|
||||
if [ ! -f "$STATE_FILE" ]; then
|
||||
mkdir -p "$(dirname "$STATE_FILE")"
|
||||
echo '{"lastCheck":"1970-01-01T00:00:00Z","seenPRs":[],"seenReleases":[]}' > "$STATE_FILE"
|
||||
fi
|
||||
|
||||
# Load last check time
|
||||
LAST_CHECK=$(jq -r '.lastCheck' "$STATE_FILE")
|
||||
NOW=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
|
||||
|
||||
# Fetch notifications (PRs only)
|
||||
PR_DATA=$(gh api 'notifications?all=true&per_page=100' 2>&1)
|
||||
|
||||
if [ $? -ne 0 ]; then
|
||||
echo '{"error":"GitHub API failed","details":"'"${PR_DATA//\"/\\\"}"'"}' | jq .
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Filter PRs where user is mentioned or author
|
||||
FILTERED_PRS=$(echo "$PR_DATA" | jq -r '[
|
||||
.[] |
|
||||
select(.subject.type == "PullRequest") |
|
||||
select(.reason == "mention" or .reason == "author" or .reason == "review_requested") |
|
||||
{
|
||||
repo: .repository.full_name,
|
||||
title: .subject.title,
|
||||
url: .subject.url,
|
||||
updated: .updated_at,
|
||||
reason: .reason,
|
||||
id: (.repository.full_name + "#" + .subject.title)
|
||||
}
|
||||
]')
|
||||
|
||||
# Filter releases
|
||||
RELEASE_DATA=$(echo "$PR_DATA" | jq -r '[
|
||||
.[] |
|
||||
select(.subject.type == "Release") |
|
||||
{
|
||||
repo: .repository.full_name,
|
||||
title: .subject.title,
|
||||
updated: .updated_at,
|
||||
id: (.repository.full_name + "@" + .subject.title)
|
||||
}
|
||||
]')
|
||||
|
||||
# Filter major releases (v*.0.0) + ALL Mirrowel/LLM-API-Key-Proxy releases
|
||||
FILTERED_RELEASES=$(echo "$RELEASE_DATA" | jq -r '[
|
||||
.[] |
|
||||
select(
|
||||
(.repo == "Mirrowel/LLM-API-Key-Proxy") or
|
||||
(.title | test("^v[0-9]+\\.0\\.0"))
|
||||
) |
|
||||
select(.title | test("(rc|pre|beta|alpha|nightly)") | not)
|
||||
]')
|
||||
|
||||
# Load seen items
|
||||
SEEN_PRS=$(jq -r '.seenPRs // []' "$STATE_FILE")
|
||||
SEEN_RELEASES=$(jq -r '.seenReleases // []' "$STATE_FILE")
|
||||
|
||||
# Find new items
|
||||
NEW_PRS=$(echo "$FILTERED_PRS" | jq --argjson seen "$SEEN_PRS" '[
|
||||
.[] | select(.id as $id | $seen | index($id) | not)
|
||||
]')
|
||||
|
||||
NEW_RELEASES=$(echo "$FILTERED_RELEASES" | jq --argjson seen "$SEEN_RELEASES" '[
|
||||
.[] | select(.id as $id | $seen | index($id) | not)
|
||||
]')
|
||||
|
||||
# Count new items
|
||||
NEW_PR_COUNT=$(echo "$NEW_PRS" | jq 'length')
|
||||
NEW_RELEASE_COUNT=$(echo "$NEW_RELEASES" | jq 'length')
|
||||
|
||||
# Update state
|
||||
ALL_PR_IDS=$(echo "$FILTERED_PRS" | jq -r '[.[].id]')
|
||||
ALL_RELEASE_IDS=$(echo "$FILTERED_RELEASES" | jq -r '[.[].id]')
|
||||
|
||||
jq -n \
|
||||
--arg now "$NOW" \
|
||||
--argjson prIds "$ALL_PR_IDS" \
|
||||
--argjson relIds "$ALL_RELEASE_IDS" \
|
||||
'{lastCheck:$now, seenPRs:$prIds, seenReleases:$relIds}' \
|
||||
> "$STATE_FILE"
|
||||
|
||||
# Output result
|
||||
if [ "$NEW_PR_COUNT" -eq 0 ] && [ "$NEW_RELEASE_COUNT" -eq 0 ]; then
|
||||
echo '{"hasNew":false}'
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Return new items
|
||||
jq -n \
|
||||
--argjson prs "$NEW_PRS" \
|
||||
--argjson releases "$NEW_RELEASES" \
|
||||
'{hasNew:true, newPRs:$prs, newReleases:$releases}'
|
||||
55
workspace/github-notifications/scripts/cron-wrapper.sh
Executable file
55
workspace/github-notifications/scripts/cron-wrapper.sh
Executable file
@@ -0,0 +1,55 @@
|
||||
#!/bin/bash
|
||||
# Cron wrapper for GitHub notifications
|
||||
# 1. Auto-dismisses low-value notifications (nightlies, previews, empty releases)
|
||||
# 2. Checks remaining notifications and formats for human consumption
|
||||
|
||||
set -e
|
||||
|
||||
WORKSPACE="${WORKSPACE:-/home/node/.openclaw/workspace}"
|
||||
cd "$WORKSPACE"
|
||||
|
||||
# First: auto-dismiss low-value notifications
|
||||
bash skills/github-notifications/scripts/auto-dismiss.sh >/dev/null 2>&1 || true
|
||||
|
||||
# Then: run the checker
|
||||
RESULT=$(bash skills/github-notifications/scripts/check.sh)
|
||||
|
||||
# Check for errors
|
||||
if echo "$RESULT" | jq -e '.error' > /dev/null 2>&1; then
|
||||
ERROR_MSG=$(echo "$RESULT" | jq -r '.error')
|
||||
ERROR_DETAILS=$(echo "$RESULT" | jq -r '.details')
|
||||
echo "❌ **GitHub Check Failed**"
|
||||
echo ""
|
||||
echo "Error: $ERROR_MSG"
|
||||
echo "\`\`\`"
|
||||
echo "$ERROR_DETAILS" | head -20
|
||||
echo "\`\`\`"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Check if there's new activity
|
||||
HAS_NEW=$(echo "$RESULT" | jq -r '.hasNew')
|
||||
|
||||
if [ "$HAS_NEW" != "true" ]; then
|
||||
# Nothing new - stay completely silent (no output = no message)
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Format and output the summary
|
||||
echo "🔔 **GitHub Activity Update**"
|
||||
echo ""
|
||||
|
||||
# Process PRs
|
||||
PR_COUNT=$(echo "$RESULT" | jq '.newPRs | length')
|
||||
if [ "$PR_COUNT" -gt 0 ]; then
|
||||
echo "**Pull Requests ($PR_COUNT new):**"
|
||||
echo "$RESULT" | jq -r '.newPRs[] | "- **\(.repo)** #\(.title)\n Updated: \(.updated) | Reason: \(.reason)"'
|
||||
echo ""
|
||||
fi
|
||||
|
||||
# Process Releases
|
||||
RELEASE_COUNT=$(echo "$RESULT" | jq '.newReleases | length')
|
||||
if [ "$RELEASE_COUNT" -gt 0 ]; then
|
||||
echo "**Releases ($RELEASE_COUNT new):**"
|
||||
echo "$RESULT" | jq -r '.newReleases[] | "- **\(.repo)** `\(.title)`\n Released: \(.updated)"'
|
||||
fi
|
||||
86
workspace/model-selector/SKILL.md
Normal file
86
workspace/model-selector/SKILL.md
Normal file
@@ -0,0 +1,86 @@
|
||||
---
|
||||
name: model-selector
|
||||
description: Safely change an agent's primary and fallback models by validating IDs against the live LLM proxy model list. Use for model switches, fallback chain updates, and model-availability troubleshooting.
|
||||
---
|
||||
|
||||
# Model Selector
|
||||
|
||||
## Core Rules
|
||||
|
||||
1. Validate model IDs against `/v1/models` before proposing changes.
|
||||
2. Keep at least 2 fallback models.
|
||||
3. Do not remove a primary model without setting a replacement.
|
||||
4. Use exact IDs from the model catalog; do not guess.
|
||||
5. Prefer provider diversity in fallbacks.
|
||||
6. Get explicit user approval before writing config.
|
||||
7. Treat `/model` as temporary; it creates per-session overrides.
|
||||
8. After backend default changes, clear session pins and reset active sessions.
|
||||
|
||||
## Workflow
|
||||
|
||||
### 1) Fetch Available Models
|
||||
|
||||
```bash
|
||||
bash {baseDir}/scripts/list-models.sh
|
||||
bash {baseDir}/scripts/list-models.sh --providers
|
||||
```
|
||||
|
||||
### 2) Validate Candidate IDs
|
||||
|
||||
```bash
|
||||
bash {baseDir}/scripts/validate-model.sh "nvidia_nim/moonshotai/kimi-k2.5"
|
||||
```
|
||||
|
||||
### 3) Inspect Current Configuration
|
||||
|
||||
```bash
|
||||
bash {baseDir}/scripts/show-current.sh
|
||||
```
|
||||
|
||||
### 4) Apply Backend Model Changes
|
||||
|
||||
```bash
|
||||
# Primary only
|
||||
bash {baseDir}/scripts/update-model.sh --primary "nanogpt/moonshotai/kimi-k2.5"
|
||||
|
||||
# Fallbacks only
|
||||
bash {baseDir}/scripts/update-model.sh --fallbacks "nvidia_nim/moonshotai/kimi-k2.5,chutes/zai-org/GLM-5-TEE"
|
||||
|
||||
# Primary + fallbacks
|
||||
bash {baseDir}/scripts/update-model.sh \
|
||||
--primary "nanogpt/moonshotai/kimi-k2.5" \
|
||||
--fallbacks "nvidia_nim/moonshotai/kimi-k2.5,chutes/zai-org/GLM-5-TEE"
|
||||
```
|
||||
|
||||
### 5) Required Rollout Sequence (Do Not Skip)
|
||||
|
||||
1. Clear per-session model pins so defaults can apply.
|
||||
2. Restart gateway so in-memory runtime state reloads config.
|
||||
3. In active channels/threads, run `/reset` (or `/new`) before testing.
|
||||
|
||||
Use pin cleanup helper:
|
||||
|
||||
```bash
|
||||
# Clear all session model pins for an agent
|
||||
bash {baseDir}/scripts/clear-session-model-pins.sh --agent home
|
||||
|
||||
# Clear only one channel session family
|
||||
bash {baseDir}/scripts/clear-session-model-pins.sh --agent home --channel 1470162839284224184
|
||||
```
|
||||
|
||||
## Model ID Format
|
||||
|
||||
- Catalog ID format: `<provider>/<model-path>`
|
||||
- Config reference format: `llm-proxy/<catalog-id>`
|
||||
|
||||
Examples:
|
||||
- `nanogpt/moonshotai/kimi-k2.5` -> `llm-proxy/nanogpt/moonshotai/kimi-k2.5`
|
||||
- `nvidia_nim/moonshotai/kimi-k2.5` -> `llm-proxy/nvidia_nim/moonshotai/kimi-k2.5`
|
||||
|
||||
For `/model` inside a session, use catalog IDs (without `llm-proxy/`).
|
||||
|
||||
## Troubleshooting Quick Checks
|
||||
|
||||
1. Model missing: rerun `list-models.sh` and validate exact ID.
|
||||
2. Old model still used: clear session pins + restart gateway + `/reset`.
|
||||
3. Unexpected fallbacks: confirm fallback chain order in `show-current.sh`.
|
||||
97
workspace/model-selector/scripts/clear-session-model-pins.sh
Executable file
97
workspace/model-selector/scripts/clear-session-model-pins.sh
Executable file
@@ -0,0 +1,97 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
clear-session-model-pins.sh --agent <agent-id> [--channel <channel-id>] [--sessions-file <path>]
|
||||
|
||||
Examples:
|
||||
clear-session-model-pins.sh --agent home
|
||||
clear-session-model-pins.sh --agent home --channel 1470162839284224184
|
||||
|
||||
Notes:
|
||||
- Removes per-session "model" keys so agent defaults apply again.
|
||||
- By default targets: /home/node/.openclaw/agents/<agent>/sessions/sessions.json
|
||||
EOF
|
||||
}
|
||||
|
||||
AGENT_ID=""
|
||||
CHANNEL_ID=""
|
||||
SESSIONS_FILE=""
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--agent)
|
||||
AGENT_ID="${2:-}"
|
||||
shift 2
|
||||
;;
|
||||
--channel)
|
||||
CHANNEL_ID="${2:-}"
|
||||
shift 2
|
||||
;;
|
||||
--sessions-file)
|
||||
SESSIONS_FILE="${2:-}"
|
||||
shift 2
|
||||
;;
|
||||
-h|--help)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
echo "Unknown argument: $1" >&2
|
||||
usage >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [[ -z "$AGENT_ID" ]]; then
|
||||
echo "--agent is required" >&2
|
||||
usage >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ -z "$SESSIONS_FILE" ]]; then
|
||||
SESSIONS_FILE="/home/node/.openclaw/agents/${AGENT_ID}/sessions/sessions.json"
|
||||
fi
|
||||
|
||||
if [[ ! -f "$SESSIONS_FILE" ]]; then
|
||||
echo "sessions file not found: $SESSIONS_FILE" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
python3 - <<PY
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
path = Path(${SESSIONS_FILE@Q})
|
||||
channel = ${CHANNEL_ID@Q}
|
||||
|
||||
with path.open() as f:
|
||||
data = json.load(f)
|
||||
|
||||
removed = 0
|
||||
scanned = 0
|
||||
|
||||
for key, value in data.items():
|
||||
if not isinstance(value, dict):
|
||||
continue
|
||||
scanned += 1
|
||||
|
||||
if channel:
|
||||
if f"channel:{channel}" not in key:
|
||||
continue
|
||||
|
||||
if "model" in value:
|
||||
del value["model"]
|
||||
removed += 1
|
||||
|
||||
with path.open("w") as f:
|
||||
json.dump(data, f, indent=2)
|
||||
f.write("\n")
|
||||
|
||||
print(f"scanned={scanned}")
|
||||
print(f"removed_model_pins={removed}")
|
||||
print(f"sessions_file={path}")
|
||||
PY
|
||||
74
workspace/model-selector/scripts/list-models.sh
Executable file
74
workspace/model-selector/scripts/list-models.sh
Executable file
@@ -0,0 +1,74 @@
|
||||
#!/usr/bin/env bash
|
||||
# list-models.sh — Query the LLM proxy /v1/models endpoint
|
||||
# Usage:
|
||||
# list-models.sh # List all model IDs (sorted)
|
||||
# list-models.sh --providers # List unique provider names
|
||||
# list-models.sh --json # Raw JSON response
|
||||
set -euo pipefail
|
||||
|
||||
# Resolve proxy URL and API key from environment or defaults
|
||||
PROXY_URL="${LLM_PROXY_URL:-https://llm-proxy.ext.ben.io/v1}"
|
||||
PROXY_KEY="${PROXY_API_KEY:-${LLM_PROXY_API_KEY:-}}"
|
||||
|
||||
if [[ -z "$PROXY_KEY" ]]; then
|
||||
# Try to read from openclaw config
|
||||
for cfg_path in \
|
||||
"${OPENCLAW_STATE_DIR:-$HOME/.openclaw/state}/openclaw.json" \
|
||||
"${OPENCLAW_CONFIG_DIR:-$HOME/.openclaw/config}/openclaw.json"; do
|
||||
if [[ -f "$cfg_path" ]]; then
|
||||
# Extract apiKey from llm-proxy provider config (handles JSON5 comments)
|
||||
key=$(grep -A5 '"llm-proxy"' "$cfg_path" | grep '"apiKey"' | head -1 | sed 's/.*"apiKey"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/' || true)
|
||||
if [[ -n "$key" && "$key" != *'${'* ]]; then
|
||||
PROXY_KEY="$key"
|
||||
break
|
||||
fi
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
if [[ -z "$PROXY_KEY" ]]; then
|
||||
echo "ERROR: No API key found. Set PROXY_API_KEY or LLM_PROXY_API_KEY environment variable." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Strip trailing /v1 from PROXY_URL if present, then always append /v1/models
|
||||
# This prevents double /v1/v1/ when LLM_PROXY_URL already includes /v1
|
||||
PROXY_BASE="${PROXY_URL%/v1}"
|
||||
PROXY_BASE="${PROXY_BASE%/}"
|
||||
|
||||
response=$(curl -s -f -H "Authorization: Bearer $PROXY_KEY" "${PROXY_BASE}/v1/models" 2>&1) || {
|
||||
echo "ERROR: Failed to query ${PROXY_BASE}/v1/models" >&2
|
||||
echo "$response" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
case "${1:-}" in
|
||||
--providers)
|
||||
echo "$response" | python3 -c "
|
||||
import sys, json
|
||||
data = json.load(sys.stdin)
|
||||
providers = sorted(set(m['id'].split('/')[0] for m in data.get('data', [])))
|
||||
for p in providers:
|
||||
print(p)
|
||||
"
|
||||
;;
|
||||
--json)
|
||||
echo "$response"
|
||||
;;
|
||||
--count)
|
||||
echo "$response" | python3 -c "
|
||||
import sys, json
|
||||
data = json.load(sys.stdin)
|
||||
print(len(data.get('data', [])))
|
||||
"
|
||||
;;
|
||||
*)
|
||||
echo "$response" | python3 -c "
|
||||
import sys, json
|
||||
data = json.load(sys.stdin)
|
||||
models = sorted(m['id'] for m in data.get('data', []))
|
||||
for m in models:
|
||||
print(m)
|
||||
"
|
||||
;;
|
||||
esac
|
||||
97
workspace/model-selector/scripts/show-current.sh
Executable file
97
workspace/model-selector/scripts/show-current.sh
Executable file
@@ -0,0 +1,97 @@
|
||||
#!/usr/bin/env bash
|
||||
# show-current.sh — Display the current model configuration from openclaw state
|
||||
# Usage: show-current.sh
|
||||
set -euo pipefail
|
||||
|
||||
# Find the state file
|
||||
STATE_FILE="${OPENCLAW_STATE_DIR:-$HOME/.openclaw/state}/openclaw.json"
|
||||
|
||||
if [[ ! -f "$STATE_FILE" ]]; then
|
||||
# Try alternative locations
|
||||
for alt in \
|
||||
"/opt/openclaw/state/openclaw.json" \
|
||||
"${OPENCLAW_CONFIG_DIR:-$HOME/.openclaw/config}/openclaw.json"; do
|
||||
if [[ -f "$alt" ]]; then
|
||||
STATE_FILE="$alt"
|
||||
break
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
if [[ ! -f "$STATE_FILE" ]]; then
|
||||
echo "ERROR: Cannot find openclaw.json state file" >&2
|
||||
echo "Searched:" >&2
|
||||
echo " ${OPENCLAW_STATE_DIR:-$HOME/.openclaw/state}/openclaw.json" >&2
|
||||
echo " /opt/openclaw/state/openclaw.json" >&2
|
||||
echo " ${OPENCLAW_CONFIG_DIR:-$HOME/.openclaw/config}/openclaw.json" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "📁 Config file: $STATE_FILE"
|
||||
echo ""
|
||||
|
||||
python3 -c "
|
||||
import json, sys, re
|
||||
|
||||
# Read file and strip JSON5 comments for parsing
|
||||
with open('$STATE_FILE', 'r') as f:
|
||||
content = f.read()
|
||||
|
||||
# Strip single-line comments (// ...) but not inside strings
|
||||
lines = content.split('\n')
|
||||
cleaned = []
|
||||
for line in lines:
|
||||
stripped = line.rstrip()
|
||||
s = stripped.lstrip()
|
||||
if s.startswith('//'):
|
||||
continue
|
||||
in_string = False
|
||||
result = []
|
||||
i = 0
|
||||
while i < len(stripped):
|
||||
c = stripped[i]
|
||||
if c == '\"' and (i == 0 or stripped[i-1] != '\\\\'):
|
||||
in_string = not in_string
|
||||
elif c == '/' and i + 1 < len(stripped) and stripped[i+1] == '/' and not in_string:
|
||||
break
|
||||
result.append(c)
|
||||
i += 1
|
||||
cleaned.append(''.join(result))
|
||||
|
||||
# Remove trailing commas (JSON5)
|
||||
json_str = '\n'.join(cleaned)
|
||||
json_str = re.sub(r',\s*([}\]])', r'\1', json_str)
|
||||
|
||||
try:
|
||||
cfg = json.loads(json_str)
|
||||
except json.JSONDecodeError:
|
||||
try:
|
||||
cfg = json.loads(content)
|
||||
except json.JSONDecodeError as e:
|
||||
print(f'ERROR: Failed to parse config: {e}', file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
agents = cfg.get('agents', {})
|
||||
defaults = agents.get('defaults', {})
|
||||
model = defaults.get('model', {})
|
||||
|
||||
if isinstance(model, str):
|
||||
print(f'🎯 Primary: {model}')
|
||||
print(f'⛓️ Fallbacks: (none configured)')
|
||||
else:
|
||||
primary = model.get('primary', '(not set)')
|
||||
fallbacks = model.get('fallbacks', [])
|
||||
print(f'🎯 Primary: {primary}')
|
||||
print(f'⛓️ Fallbacks ({len(fallbacks)}):')
|
||||
for i, fb in enumerate(fallbacks, 1):
|
||||
print(f' {i}. {fb}')
|
||||
|
||||
# Check for per-agent model overrides
|
||||
agent_list = agents.get('list', [])
|
||||
overrides = [(a.get('id', '?'), a.get('model', '')) for a in agent_list if 'model' in a]
|
||||
if overrides:
|
||||
print()
|
||||
print('⚠️ Per-agent model overrides:')
|
||||
for aid, amodel in overrides:
|
||||
print(f' {aid}: {amodel}')
|
||||
" 2>&1
|
||||
216
workspace/model-selector/scripts/update-model.sh
Executable file
216
workspace/model-selector/scripts/update-model.sh
Executable file
@@ -0,0 +1,216 @@
|
||||
#!/usr/bin/env bash
|
||||
# update-model.sh — Update model configuration in openclaw state file
|
||||
# Usage:
|
||||
# update-model.sh --primary <model-id>
|
||||
# update-model.sh --fallbacks <model1,model2,model3>
|
||||
# update-model.sh --primary <model-id> --fallbacks <model1,model2>
|
||||
#
|
||||
# All model IDs are validated against /v1/models before writing.
|
||||
# A backup of the current config is created before any changes.
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
PRIMARY=""
|
||||
FALLBACKS=""
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--primary)
|
||||
PRIMARY="$2"
|
||||
shift 2
|
||||
;;
|
||||
--fallbacks)
|
||||
FALLBACKS="$2"
|
||||
shift 2
|
||||
;;
|
||||
--help|-h)
|
||||
echo "Usage: update-model.sh [--primary <model-id>] [--fallbacks <model1,model2,...>]"
|
||||
echo ""
|
||||
echo "Options:"
|
||||
echo " --primary Set the primary model (will be prefixed with llm-proxy/)"
|
||||
echo " --fallbacks Comma-separated list of fallback models (min 2 required)"
|
||||
echo ""
|
||||
echo "All model IDs are validated against /v1/models before writing."
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
echo "ERROR: Unknown argument: $1" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [[ -z "$PRIMARY" && -z "$FALLBACKS" ]]; then
|
||||
echo "ERROR: Must specify --primary and/or --fallbacks" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Find the state file
|
||||
STATE_FILE="${OPENCLAW_STATE_DIR:-$HOME/.openclaw/state}/openclaw.json"
|
||||
if [[ ! -f "$STATE_FILE" ]]; then
|
||||
for alt in \
|
||||
"/opt/openclaw/state/openclaw.json" \
|
||||
"${OPENCLAW_CONFIG_DIR:-$HOME/.openclaw/config}/openclaw.json"; do
|
||||
if [[ -f "$alt" ]]; then
|
||||
STATE_FILE="$alt"
|
||||
break
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
if [[ ! -f "$STATE_FILE" ]]; then
|
||||
echo "ERROR: Cannot find openclaw.json state file" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "📁 Config file: $STATE_FILE"
|
||||
|
||||
# Validate all model IDs first
|
||||
echo ""
|
||||
echo "🔍 Validating model IDs against /v1/models..."
|
||||
VALIDATION_FAILED=0
|
||||
|
||||
if [[ -n "$PRIMARY" ]]; then
|
||||
# Strip llm-proxy/ prefix for validation
|
||||
PRIMARY_CLEAN="${PRIMARY#llm-proxy/}"
|
||||
if ! "$SCRIPT_DIR/validate-model.sh" "$PRIMARY_CLEAN" 2>&1; then
|
||||
VALIDATION_FAILED=1
|
||||
fi
|
||||
fi
|
||||
|
||||
if [[ -n "$FALLBACKS" ]]; then
|
||||
IFS=',' read -ra FB_ARRAY <<< "$FALLBACKS"
|
||||
|
||||
if [[ ${#FB_ARRAY[@]} -lt 2 ]]; then
|
||||
echo "❌ ERROR: Minimum 2 fallback models required (got ${#FB_ARRAY[@]})" >&2
|
||||
VALIDATION_FAILED=1
|
||||
fi
|
||||
|
||||
for fb in "${FB_ARRAY[@]}"; do
|
||||
fb_clean="${fb#llm-proxy/}"
|
||||
fb_clean="$(echo "$fb_clean" | xargs)" # trim whitespace
|
||||
if ! "$SCRIPT_DIR/validate-model.sh" "$fb_clean" 2>&1; then
|
||||
VALIDATION_FAILED=1
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
if [[ $VALIDATION_FAILED -ne 0 ]]; then
|
||||
echo ""
|
||||
echo "❌ Validation failed. No changes made." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Create backup
|
||||
BACKUP="${STATE_FILE}.backup.$(date +%Y%m%d-%H%M%S)"
|
||||
cp "$STATE_FILE" "$BACKUP"
|
||||
echo ""
|
||||
echo "💾 Backup saved: $BACKUP"
|
||||
|
||||
# Apply changes using Python for safe JSON manipulation
|
||||
python3 -c "
|
||||
import json, sys, re
|
||||
|
||||
state_file = '$STATE_FILE'
|
||||
primary = '${PRIMARY}'.strip() or None
|
||||
fallbacks_raw = '${FALLBACKS}'.strip() or None
|
||||
|
||||
# Read and parse (handle JSON5 comments)
|
||||
with open(state_file, 'r') as f:
|
||||
content = f.read()
|
||||
|
||||
# Strip comments for parsing
|
||||
lines = content.split('\n')
|
||||
cleaned = []
|
||||
for line in lines:
|
||||
s = line.lstrip()
|
||||
if s.startswith('//'):
|
||||
continue
|
||||
in_string = False
|
||||
result = []
|
||||
i = 0
|
||||
while i < len(line):
|
||||
c = line[i]
|
||||
if c == '\"' and (i == 0 or line[i-1] != '\\\\'):
|
||||
in_string = not in_string
|
||||
elif c == '/' and i + 1 < len(line) and line[i+1] == '/' and not in_string:
|
||||
break
|
||||
result.append(c)
|
||||
i += 1
|
||||
cleaned.append(''.join(result))
|
||||
|
||||
# Remove trailing commas before } or ] (JSON5 feature)
|
||||
json_str = '\n'.join(cleaned)
|
||||
json_str = re.sub(r',\s*([}\]])', r'\1', json_str)
|
||||
|
||||
try:
|
||||
cfg = json.loads(json_str)
|
||||
except json.JSONDecodeError:
|
||||
try:
|
||||
cfg = json.loads(content)
|
||||
except json.JSONDecodeError as e:
|
||||
print(f'ERROR: Failed to parse config: {e}', file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
# Ensure path exists
|
||||
if 'agents' not in cfg:
|
||||
cfg['agents'] = {}
|
||||
if 'defaults' not in cfg['agents']:
|
||||
cfg['agents']['defaults'] = {}
|
||||
if 'model' not in cfg['agents']['defaults']:
|
||||
cfg['agents']['defaults']['model'] = {}
|
||||
|
||||
model = cfg['agents']['defaults']['model']
|
||||
if isinstance(model, str):
|
||||
model = {'primary': model}
|
||||
cfg['agents']['defaults']['model'] = model
|
||||
|
||||
old_primary = model.get('primary', '(none)')
|
||||
old_fallbacks = model.get('fallbacks', [])
|
||||
|
||||
# Apply primary
|
||||
if primary:
|
||||
# Ensure llm-proxy/ prefix
|
||||
if not primary.startswith('llm-proxy/'):
|
||||
primary = f'llm-proxy/{primary}'
|
||||
model['primary'] = primary
|
||||
|
||||
# Apply fallbacks
|
||||
if fallbacks_raw:
|
||||
fbs = [fb.strip() for fb in fallbacks_raw.split(',') if fb.strip()]
|
||||
fbs = [f'llm-proxy/{fb}' if not fb.startswith('llm-proxy/') else fb for fb in fbs]
|
||||
model['fallbacks'] = fbs
|
||||
|
||||
# Remove per-agent model overrides that match the old primary
|
||||
# (they were likely set by the same drift that caused the issue)
|
||||
agent_list = cfg.get('agents', {}).get('list', [])
|
||||
removed_overrides = []
|
||||
for agent in agent_list:
|
||||
if 'model' in agent:
|
||||
removed_overrides.append((agent.get('id', '?'), agent['model']))
|
||||
del agent['model']
|
||||
|
||||
# Write back
|
||||
with open(state_file, 'w') as f:
|
||||
json.dump(cfg, f, indent=2)
|
||||
f.write('\n')
|
||||
|
||||
# Print summary
|
||||
print()
|
||||
print('✅ Configuration updated:')
|
||||
print()
|
||||
print(f' Primary: {old_primary} → {model.get(\"primary\", \"(none)\")}')
|
||||
print(f' Fallbacks:')
|
||||
for i, fb in enumerate(model.get('fallbacks', []), 1):
|
||||
old_marker = '' if fb in old_fallbacks else ' (new)'
|
||||
print(f' {i}. {fb}{old_marker}')
|
||||
if removed_overrides:
|
||||
print()
|
||||
print(' 🧹 Cleared per-agent model overrides:')
|
||||
for aid, amodel in removed_overrides:
|
||||
print(f' {aid}: {amodel} → (uses default)')
|
||||
" 2>&1
|
||||
|
||||
echo ""
|
||||
echo "Done. Restart OpenClaw for changes to take effect."
|
||||
39
workspace/model-selector/scripts/validate-model.sh
Executable file
39
workspace/model-selector/scripts/validate-model.sh
Executable file
@@ -0,0 +1,39 @@
|
||||
#!/usr/bin/env bash
|
||||
# validate-model.sh — Validate that a model ID exists in the LLM proxy
|
||||
# Usage: validate-model.sh <model-id>
|
||||
# Exit 0 if valid, 1 if not found
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
if [[ $# -lt 1 ]]; then
|
||||
echo "Usage: validate-model.sh <model-id>" >&2
|
||||
echo "Example: validate-model.sh nanogpt/deepseek-chat" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
MODEL_ID="$1"
|
||||
|
||||
# Strip llm-proxy/ prefix if present (user might pass the openclaw.json format)
|
||||
MODEL_ID="${MODEL_ID#llm-proxy/}"
|
||||
|
||||
# Get the live model list
|
||||
available=$("$SCRIPT_DIR/list-models.sh" 2>/dev/null) || {
|
||||
echo "ERROR: Could not fetch model list from LLM proxy" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
if echo "$available" | grep -qxF "$MODEL_ID"; then
|
||||
echo "✅ Model '$MODEL_ID' is available"
|
||||
exit 0
|
||||
else
|
||||
echo "❌ Model '$MODEL_ID' NOT found in /v1/models" >&2
|
||||
# Suggest close matches
|
||||
partial=$(echo "$available" | grep -i "$(echo "$MODEL_ID" | sed 's|.*/||')" | head -5)
|
||||
if [[ -n "$partial" ]]; then
|
||||
echo "" >&2
|
||||
echo "Did you mean one of these?" >&2
|
||||
echo "$partial" | sed 's/^/ /' >&2
|
||||
fi
|
||||
exit 1
|
||||
fi
|
||||
Reference in New Issue
Block a user