Implement LLM-powered rule violation analyzer #5

New Issue

b3nw · 2026-01-05T21:33:25-06:00

b3nw commented

2026-01-05 21:33:25 -06:00

Overview

Use Claude API to analyze posts for forum rule violations.

Tasks

Create src/eve_mod/analyzer.py
Configure Anthropic client for llm-proxy.ext.ben.io
Load API key from environment (LLM_PROXY_KEY)
Build system prompt with full rule definitions
Analyze posts in batches of 5 for efficiency
Parse structured JSON responses
Handle LLM errors and retries

Analysis Schema

@dataclass
class Violation:
    rule_id: str
    rule_name: str
    severity: str  # high, medium, low
    confidence: float  # 0.0-1.0
    explanation: str
    quote: str  # relevant excerpt

@dataclass
class PostAnalysis:
    post: UserPost
    violations: list[Violation]
    clean: bool

LLM Configuration

Endpoint: https://llm-proxy.ext.ben.io
Model: claude-sonnet-4-5-20250514 (or similar)
Auth: Bearer token from env

Acceptance Criteria

Can analyze batch of posts
Returns structured violation data
Handles rate limits and errors

## Overview Use Claude API to analyze posts for forum rule violations. ## Tasks - [ ] Create `src/eve_mod/analyzer.py` - [ ] Configure Anthropic client for `llm-proxy.ext.ben.io` - [ ] Load API key from environment (`LLM_PROXY_KEY`) - [ ] Build system prompt with full rule definitions - [ ] Analyze posts in batches of 5 for efficiency - [ ] Parse structured JSON responses - [ ] Handle LLM errors and retries ## Analysis Schema ```python @dataclass class Violation: rule_id: str rule_name: str severity: str # high, medium, low confidence: float # 0.0-1.0 explanation: str quote: str # relevant excerpt @dataclass class PostAnalysis: post: UserPost violations: list[Violation] clean: bool ``` ## LLM Configuration - Endpoint: `https://llm-proxy.ext.ben.io` - Model: `claude-sonnet-4-5-20250514` (or similar) - Auth: Bearer token from env ## Acceptance Criteria - Can analyze batch of posts - Returns structured violation data - Handles rate limits and errors

b3nw closed this issue

2026-01-05 21:41:12 -06:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: b3nw/eve-forum-moderator#5