rss_feeds v1.0.0

name: "rss_feeds" description: "Publish a valid RSS 2.0 or Atom 1.0 feed an agent can maintain (required elements, guid strategy, content:encoded, autodiscovery), and poll feeds reliably with conditional GET (ETag/Last-Modified), guid-based dedupe, polite intervals, and malformed-feed handling. Use when an agent needs to syndicate its own updates or watch someone else's feed for new items without spamming or re-notifying." metadata: includeInPrompt: true

Purpose

Two directions, one file. Publish: emit a minimal valid RSS 2.0 (or Atom 1.0) feed with stable guids, full HTML bodies, and an autodiscovery tag. Consume: poll feeds with conditional GET, dedupe on guid (never title), and report only genuinely new items. A poller that re-notifies on re-seen items is a bug, not a feature.

Workflow

A. Publish

1. Copy the generator below into make_feed.py, edit CHANNEL and ITEMS (newest first), run python3 make_feed.py > feed.xml. Requires stdlib only.
2. Validate the output before shipping: python3 -c "import xml.etree.ElementTree as ET; ET.parse('feed.xml')" must succeed, and feedparser.parse('feed.xml').bozo must be False with the expected entry count (pip install feedparser).
3. Serve feed.xml over HTTPS with Content-Type: application/rss+xml (Atom: application/atom+xml) and a stable URL — moving the URL breaks every subscriber.
4. Add the autodiscovery tag inside the site's HTML <head>: <link rel="alternate" type="application/rss+xml" title="My Agent Feed" href="https://example.com/feed.xml">
5. On each new item: prepend the item, keep the guid scheme identical, regenerate, done. Never edit a published item's guid.

python
#!/usr/bin/env python3
"""Minimal valid RSS 2.0 generator. Edit CHANNEL/ITEMS, run: python3 make_feed.py > feed.xml"""
import xml.etree.ElementTree as ET
from email.utils import format_datetime
from datetime import datetime, timezone

CHANNEL = {"title": "My Agent Feed", "link": "https://example.com/",
           "description": "Updates from my agent", "feed_url": "https://example.com/feed.xml"}
ITEMS = [  # newest first; guid = stable, unique, never reused (permalink URL or tag: URI)
    {"title": "First post", "link": "https://example.com/posts/1",
     "guid": "https://example.com/posts/1",
     "pub": datetime(2026, 9, 21, 12, 0, tzinfo=timezone.utc),
     "summary": "Short summary.", "html": "<p>Full <b>HTML</b> body.</p>"},
]

rss = ET.Element("rss", version="2.0")
rss.set("xmlns:atom", "http://www.w3.org/2005/Atom")
rss.set("xmlns:content", "http://purl.org/rss/1.0/modules/content/")
ch = ET.SubElement(rss, "channel")
ET.SubElement(ch, "title").text = CHANNEL["title"]
ET.SubElement(ch, "link").text = CHANNEL["link"]
ET.SubElement(ch, "description").text = CHANNEL["description"]
ET.SubElement(ch, "language").text = "en-us"
ET.SubElement(ch, "{http://www.w3.org/2005/Atom}link",
              href=CHANNEL["feed_url"], rel="self", type="application/rss+xml")
for it in ITEMS:
    i = ET.SubElement(ch, "item")
    ET.SubElement(i, "title").text = it["title"]
    ET.SubElement(i, "link").text = it["link"]
    ET.SubElement(i, "guid", isPermaLink="true").text = it["guid"]
    ET.SubElement(i, "pubDate").text = format_datetime(it["pub"], usegmt=True)
    ET.SubElement(i, "description").text = it["summary"]
    ET.SubElement(i, "{http://purl.org/rss/1.0/modules/content/}encoded").text = it["html"]
print('<?xml version="1.0" encoding="UTF-8"?>')
print(ET.tostring(rss, encoding="unicode"))

Atom 1.0 element map (same data, different tags): channel→feed, item→entry, guid→id, pubDate (RFC 822, e.g. Mon, 21 Sep 2026 12:00:00 GMT)→updated (RFC 3339, e.g. 2026-09-21T12:00:00Z), content:encoded→<content type="html">, self-link→<link href="..." rel="self"/>. Absolute URLs everywhere.

B. Consume / poll

1. Save the poller below as poll.py (pip install feedparser first). Run: python3 poll.py FEED_URL. State lives in ~/.rss_poller_state.json.
2. First run stores the ETag and Last-Modified response headers plus every seen guid. Later runs pass them back as If-None-Match / If-Modified-Since.
3. A 304 Not Modified means zero new items — exit quietly, do not re-scan, do not re-report.
4. On 200, compute new items as entry.id not in seen_guids, report those, then add their guids to the seen set. entry.id is the guid; never dedupe on title or link.
5. If feedparser sets bozo and yields no entries, log the exception and skip the run — never crash, never invent missing fields.

python
#!/usr/bin/env python3
"""RSS/Atom poller: conditional GET + guid dedupe. Run: python3 poll.py FEED_URL"""
import json, os, sys
import feedparser

URL = sys.argv[1]
STATE = os.path.expanduser("~/.rss_poller_state.json")

def load():
    try: return json.load(open(STATE))
    except (FileNotFoundError, json.JSONDecodeError): return {}

def main():
    st = load().get(URL, {})
    d = feedparser.parse(URL, etag=st.get("etag"), modified=st.get("modified"),
                         agent="rss_feeds-skill-poller/1.0")
    if getattr(d, "status", 200) == 304:
        print("304 not modified — nothing new"); return
    if d.bozo and not d.entries:
        print("WARN: unparseable feed:", repr(d.bozo_exception)[:120]); return
    seen = set(st.get("seen", []))
    new = [e for e in d.entries if e.get("id") not in seen]
    for e in new:
        print("NEW", e.get("id"), "|", e.get("title"), "|", e.get("link"))
        seen.add(e.get("id"))
    if d.get("etag"): st["etag"] = d.etag
    if d.get("modified"): st["modified"] = d.modified
    st["seen"] = sorted(seen)[-5000:]
    all_st = load(); all_st[URL] = st
    json.dump(all_st, open(STATE, "w"))
    if not new: print("no new items")

main()

Output Contract

• Publishing: (1) the feed XML written to the agreed path/URL; (2) validation evidence — xml.etree parse OK and feedparser bozo=False with entry count; (3) the autodiscovery <link> tag placed in the site <head> (or a report of where it must go).
• Polling: (1) state file path reported on first run; (2) per run, return ONLY items whose guid is new — each as guid | title | link | published/updated; (3) on 304 or unparseable feed, return a one-line status and emit nothing else. A run that re-emits an already-seen guid fails the contract.

Operating Rules

1. Dedupe on guid (entry.id), never title or link. Titles get corrected, links get redirected; guids are the identity.
2. Never re-notify on a re-seen guid — even if its content or timestamp changed — unless the task explicitly asks for update alerts. Report new items only.
3. Always send stored validators (If-None-Match / If-Modified-Since) and honor 304 as "nothing changed". A feed may send only one of ETag/Last-Modified; store and send whichever you got.
4. Poll politely by default: prefer the publisher's own interval (ttl, sy:updatePeriod/sy:updateFrequency, Cache-Control) and check its robots.txt/terms before going faster; default to hourly unless the source clearly updates faster and permits it. Back off on 429 and honor Retry-After; one request per feed per interval, no parallel fetches of the same feed.
5. Guids are permanent. A guid identifies one item forever; never reuse a guid for different content, never change a published guid. isPermaLink="true" only when the guid is the item's real URL.
6. Dates in the right format: RSS pubDate/lastBuildDate = RFC 822 (Mon, 21 Sep 2026 12:00:00 GMT); Atom updated/published = RFC 3339 (2026-09-21T12:00:00Z). email.utils.format_datetime(dt, usegmt=True) produces valid RFC 822.
7. Absolute URLs in link and guid. Relative URLs silently break subscribers and dedupe.
8. Identify your poller with a custom User-Agent (the script's agent= kwarg) so publishers can contact you instead of blocking you.
9. Malformed feeds degrade, never crash: on bozo with entries, use what parsed and flag the warning; on total failure, skip the run. Try the feed's declared encoding before assuming UTF-8; chardet is a fallback, not a first resort.
10. Validate before you publish, every time: well-formed XML plus feedparser bozo=False. A feed that only parses in your head is not a feed.