---
name: "regex_mastery"
description: "Practical regular expressions for agents: core syntax, the grep/Python/sed dialect differences that actually bite, and tested patterns for timestamps, UUIDs, IPs, emails, and log parsing. Trigger when extracting data from logs or text, or rewriting text with patterns."
---

## Purpose
Give agents a working regex toolkit: the core syntax, which dialect they're in (this is where most bugs live), and copy-paste patterns that were actually tested. Every pattern below was verified against real input before inclusion.

## Workflow
1. **Identify the dialect** from the Dialects table — BRE vs ERE vs PCRE is the #1 source of silent failures.
2. **Draft the pattern** using the core syntax and the tested pattern library.
3. **Test before trusting**: run it against real input (`grep -o`, a quick `python3 -c`, or `sed` on a sample line) and eyeball the matches. A pattern that "looks right" and one that works are different things.
4. **Anchor and bound it**: add `^$` or `\b` guards so it matches what you mean, not a substring of something else.
5. **For extraction**, prefer Python `re` with named groups — the result is self-documenting and easy to act on.

## Output Contract
- **Extraction** returns a list of matches (or named-group dicts), one entry per match in order; verify count and content against the source text before acting on them.
- **Rewriting** returns the transformed text; diff it against the original to confirm only intended regions changed.
- **No match is a signal**: before concluding "not present," re-run with a looser pattern to rule out a dialect or anchoring bug.

## Core syntax (dialect-neutral)
- Character classes: `[abc]`, `[^abc]`, `[0-9]`, `[a-zA-Z]`; shorthands `\d \w \s` (+ `\D \W \S`); `.` = any char except newline (escape it: `\.`).
- Quantifiers: `*` (0+), `+` (1+), `?` (0/1), `{n}`, `{n,}`, `{n,m}` — all **greedy** by default; add `?` for lazy: `<.*>` grabs `<a><b>`, `<.*?>` grabs `<a>` then `<b>`.
- Anchors: `^` start, `$` end, `\b` word boundary. `re.findall(r'^\d+$', '1\n2', re.M)` matches each line.
- Groups/captures: `(...)` captures; `(?:...)` non-capturing; `(?P<name>...)` named; `\1` backreferences the capture.
- Alternation: `cat|dog` — scope it with groups when needed: `\b(cat|dog)s?\b`.
- Flags: `re.DOTALL` lets `.` cross newlines; `re.MULTILINE` makes `^$` per-line.

## Dialects (verified on GNU grep 3.11 / GNU sed)
| Tool | Default dialect | What works |
|---|---|---|
| `grep` | BRE: `( ) { }` are literal unless escaped as `\( \) \{ \}`; `+` and `?` are literal | Use BRE escapes, or... |
| `grep -E` | ERE: `+ ? () \| {}` work unescaped; no `\d \w \b` | Default to `-E` for ad-hoc searching |
| `grep -P` | PCRE: `\d \w \s \b`, lookarounds | Works here, but treat as non-portable (absent on BSD/macOS grep) — default to `-E` |
| `sed` | BRE by default; `-E` enables ERE | `-E` is portable (both GNU `-E` and `-r` work); replacement uses `\1` backreferences and `&` for the whole match |
| Python `re` | PCRE-like | Always use raw strings: `r'\d+'`. Default to `re.search` (finds anywhere); `re.match` only matches at the string start — a classic silent bug. |

Tested examples:
```bash
echo "abcabc" | grep -E "(abc)\1"            # ERE backreference
echo "2026-09-17" | sed -E 's/([0-9]{4})-([0-9]{2})-([0-9]{2})/\3\/\2\/\1/'  # -> 17/09/2026
```

## Tested pattern library
- Timestamps (ISO-ish): `\d{4}-\d{2}-\d{2}[T ]\d{2}:\d{2}:\d{2}(?:Z|[+-]\d{2}:?\d{2})?`
- UUID: `\b[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}\b`
- IPv4: `\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b` (matches octet-shaped text; `999.1.1.1` passes — validate numerically if strictness matters)
- IPv6: `(?<![0-9A-Fa-f:])(?:(?:[0-9A-Fa-f]{1,4}:){7}[0-9A-Fa-f]{1,4}|(?:[0-9A-Fa-f]{1,4}:){1,6}(?::[0-9A-Fa-f]{1,4}){1,}|(?:[0-9A-Fa-f]{1,4}:){1,7}:|:(?::[0-9A-Fa-f]{1,4}){1,7})(?![0-9A-Fa-f:])`
- Email (practical, not strict): `[\w.+-]+@[\w-]+\.[\w.-]+`

**Why strict email regex is a trap:** full RFC validation is thousands of chars and still wrong at the edges (quoted locals, IP literals, new TLDs). Default to the simple pattern above plus a real verification step (send, or a lookup) — a regex can only tell you it *looks* like an email.

## Log-parsing example
```python
import re
line = '[2026-09-17 02:54:59] ERROR worker-3: connection refused (10.0.0.5:8080)'
m = re.search(r'\[(?P<ts>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})\]\s+(?P<level>[A-Z]+)\s+(?P<msg>.*)', line)
m.groupdict()
# {'ts': '2026-09-17 02:54:59', 'level': 'ERROR', 'msg': 'worker-3: connection refused (10.0.0.5:8080)'}
```

## Operating Rules
1. Never invent regex features — if unsure a construct exists in your dialect, test it first.
2. Default to `grep -E` over plain `grep` and over `grep -P`; default to raw strings in Python; default to `re.search` over `re.match`.
3. Escape literal dots in IPs/domains (`\.`) — unescaped, `10.0.0.5` also matches `10x0y0z5`.
4. Watch greedy `.*` spanning more than intended; reach for `.*?` or a negated class (`[^]]*`) in delimited text.
5. When a pattern silently matches nothing, first suspect the dialect (BRE vs ERE), then the anchors — not the data.
6. Soften portability claims: what works on GNU grep/sed here may not hold on BSD/macOS — verify on the target machine.