b3nw/eve-wiki-content
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial wiki structure

Browse Source
This commit is contained in:
b3nw
2026-04-19 03:57:03 +00:00
commit 31c3083ff4
11 changed files with 1041 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
docs/validation/quality-control.md Normal file
View File

@@ -0,0 +1,60 @@
# Quality Control & Operations Protocol
This document defines the automated decision-making logic for content validation and the system-wide response to operational failures.
## 1. Validation Scoring Formula (Blocker #6)
Agent E (Validation) and Agent F (Numerical) calculate a combined **Confidence Score (0-100%)**. A score of **95% or higher** is required for auto-publication to the `main` branch.
### 1.1 Weighted Categories
| Category | Weight | Must Pass? | Criteria |
|----------|--------|------------|----------|
| **Numerical (ESI)** | 40% | **YES** | Stats (HP, Slots, PG/CPU) must match ESI ±0%. |
| **Structural** | 20% | **YES** | Valid YAML, required fields present, correct URL path. |
| **Relational** | 20% | NO | Internal links resolve, TypeIDs are valid. |
| **Semantic** | 20% | NO | Prose description matches structured data intent. |
### 1.2 The "Must Pass" Rule
If any **"Must Pass"** category fails (score < 100% in that category), the total Confidence Score is immediately capped at **0%**, regardless of other categories. This prevents LLM hallucinations from overriding official game data.
### 1.3 Scoring Logic
`Total Score = (ESI * 0.4) + (Struct * 0.2) + (Relat * 0.2) + (Semant * 0.2)`
---
## 2. Failure Handling Matrix (Blocker #8)
The system distinguishes between transient infrastructure issues and content quality failures.
### 2.1 Tiered Response Matrix
| Error Type | Tier | Initial Action | Max Retries | Escalation |
|------------|------|----------------|-------------|------------|
| **API Timeout / 5xx** | 1 | Exponential Backoff (1m, 5m, 15m) | 3 | Tier 3 Alert |
| **Validation Fail (70-94%)** | 2 | Regeneration with Error Feedback | 2 | Tier 3 Alert |
| **Validation Fail (<70%)** | 2 | Immediate Rejection | 1 | Tier 3 Alert |
| **Auth Failure / 401** | 3 | Halt Pipeline | 0 | **Critical System Alert** |
| **Git Conflict** | 3 | Halt Pipeline | 0 | **Critical System Alert** |
### 2.2 Feedback Loop (Tier 2)
When a page fails validation with a score of 70-94%, Agent E sends a "Correction Request" back to the source agent (A, B, or C) containing:
1. The specific field that failed.
2. The expected value (from ESI or Schema).
3. The current value produced.
### 2.3 Critical System Alerting
Tier 3 failures trigger a webhook notification to the system administrator. The LangGraph state is persisted as a "Suspended" checkpoint, allowing for manual inspection and resume via LangSmith.
---
## 3. Rate Limiting Policy (Blocker #11)
To ensure stability and prevent IP bans, the following global limits are enforced at the transport layer:
| Target | Rate Limit | Burst |
|--------|------------|-------|
| **ESI API** | 20 req / sec | 50 |
| **MediaWiki API** | 2 req / sec | 5 |
| **Wiki.js API** | 5 req / sec | 10 |
| **LLM APIs** | Per Provider Tier | N/A |