--- name: learned_lessons_ledger description: Keep a confidence-scored lessons ledger for an agent. Captures operational patterns with an initial confidence score, updates scores on confirmed hits and misses, prunes stale entries, and makes reuse a hard step before acting. Triggers when an agent keeps re-learning the same mistakes, needs to audit its own heuristics, or wants experience to compound across runs. metadata: { "includeInPrompt": true } --- # Learned Lessons Ledger ## Purpose An agent that never writes down what it learns will re-learn it forever. This skill runs a personal lessons ledger: a small, scored, pruned file of patterns the agent actually uses. The difference from a tips file is the scoring. Every entry carries a confidence score that moves with evidence, and a prune loop that deletes what stops being true. The ledger compounds: single incidents merge into broader rules as evidence accumulates. ## Workflow ### 1. Capture (after anything worth learning) When a task finishes, ask one question: did something here happen before, or will it happen again? If yes, write an entry. Five fields, no more: - **Pattern**: the one-sentence rule ("check render coverage before publishing"). - **Evidence**: the incident that taught it (date, what happened, what it cost). - **Context**: when it applies and, more importantly, when it does not. - **Confidence**: a number in [0, 1]. New entries start at 0.5, never higher. No exceptions. - **Falsifier**: what would prove it wrong ("one clean deploy without the check"). One incident can still land in the ledger. The 0.5 starting score marks it as unverified, and the falsifier keeps it honest. This mirrors the SRE postmortem habit of writing the record even while the root cause is fresh, then letting the record earn its authority over time. ### 2. Reuse (before any non-trivial task) Read the ledger before starting work, not after finishing it. Scan for entries whose context matches the task. Follow entries with confidence above 0.7; treat 0.5 to 0.7 entries as hypotheses worth testing. If an entry fired and changed the outcome, close the loop in step 3. ### 3. Score (update on hits and misses) A hit: the pattern predicted something correctly or prevented a repeat. A miss: the pattern fired and was wrong, or it stayed silent when it should have fired. Update with a fixed rule so scores stay comparable: - Hit: `c = c + (1 - c) * 0.2` (approaches 1 without ever reaching it) - Miss: `c = c * 0.6` (a miss hurts more than a hit helps) This is deliberately asymmetric. Trust is hard to build and easy to lose, and the math should reflect that. Raw confidence without this calibration is exactly what goes wrong with uncalibrated judge models: confident and wrong beats nothing at all for being dangerous. ### 4. Prune (monthly, or every ~50 tasks) Delete anything that is dead weight. An entry is pruned when any of these hold: - Last touched more than 60 days ago AND confidence below 0.4 - Three consecutive misses - Contradicted by two higher-confidence entries covering the same context - Its context no longer exists (tool removed, process changed, world moved on) Pruning is the discipline that separates a ledger from a junk drawer. Memory that never forgets degrades: stale entries get quoted at the wrong moment and cost more than the entries that were never captured. Archive pruned entries to a dated file rather than deleting them outright, so a recurring environment can revive them. ### 5. Compound (merge entries into rules) When three or more related entries all sit above 0.6, write a single meta-entry that subsumes them and link the originals as evidence. Example: entries about "verify the commit after committing," "snapshot files before destructive git operations," and "check rendered state after posting" compound into "never trust a claimed effect, verify it on the actual surface." Meta-entries score independently; if the meta-entry misses, demote it back to its children. ## Output Contract Every ledger entry, in markdown or JSON, contains exactly: `pattern`, `evidence`, `context`, `confidence` (float 0..1), `last_touched` (date), `hits`, `misses`, `falsifier`. Optional: `tags`, `supersedes` (links to merged entry ids). The monthly prune produces a short report: entries pruned with reasons, entries promoted above 0.7, new meta-entries proposed. ## Operating Rules 1. Never start an entry above 0.5 confidence. One incident is a hypothesis, not a law. 2. Blameless entries only. Record what happened and what the system allowed, never who failed. A ledger used for blame stops getting honest entries. 3. Every entry has a falsifier. If you cannot say what would prove it wrong, it is a belief, not a lesson. 4. Quote entries by id when reusing them, so hits can be credited to the right pattern. 5. One ledger per agent, one file per month, plain markdown or the JSON format below. Keep the file the agent actually opens small: archive everything else. 6. Prune on schedule, not on feeling. An entry you are fond of but that never fires is decoration. 7. Reuse comes before capture. A ledger nobody reads is a journal, not a tool. ## Tooling `bin/ledger.py` implements the loop as a CLI over a JSON ledger file: ``` ledger.py add --pattern "..." --evidence "..." --context "..." --falsifier "..." ledger.py hit # confirmed correct ledger.py miss # confirmed wrong ledger.py list --min 0.7 # entries worth trusting ledger.py prune --days 60 --min 0.4 --archive archive.json ledger.py merge --pattern "..." # compound into a meta-entry ``` LEDGER env var selects the file; defaults to `./ledger.json`. JSON ids are `L-001`, `L-002`, and so on. All update math follows the rules in this file.