agent_capability_profiles v1.0.0

name: agent_capability_profiles description: "Write and check a structured capability profile for an agent: skills with levels, tools with access, stated limits, evidence for each claim, and a freshness date, so a routing, delegation, or selection decision can rest on checked claims. Use when an agent must declare what it can do, or when profiles must be compared before work is assigned." metadata: { "includeInPrompt": true }

Agent Capability Profiles

Purpose

Build a capability profile: a single document that declares what an agent can do, at what level, with which tools and access, within which limits, and with evidence for each claim. A reviewer uses the profile to route a task, delegate work, or compare agents for a selection decision. A profile with claims and no evidence is advertising copy. This skill turns the declaration into a checked record with a date, a version, and a pass or fail result from a fixed check routine.

Scope:

• agent-persona-design covers character and voice. This skill covers capability claims: skills, tools, access, limits, evidence, and freshness. A profile says nothing about tone or personality.
• subagent-orchestration covers delegation mechanics: splitting work, running delegated tasks, and reconciling results. This skill covers the declaration that is read before delegation starts, and the check that decides whether a profile is fit to route from.
• agent-observability covers run telemetry: traces, token counts, and failure records from work in progress or work finished. This skill covers a standing declaration made outside any single run. Run records can serve as evidence inside a profile, and a profile never replaces run records.
• This skill does not cover testing an agent, scoring a run, granting access, or building a registry. The deliverable is a profile document plus a check routine. Any storage, index, or directory that holds profiles is a separate choice, recorded where the profile is kept.
• Tool claims in this skill were checked on 2026-10-15 by running the inlined check script with Python 3.12.3, which is the version python3 --version returned on the drafting machine. The script uses only the Python standard library (argparse, json, sys, and datetime), so no package version is quoted and no install step is needed. Every number in the Output Contract and in the expected output block comes from a real run of that script against its fixture.

Workflow

1. State the decision the profile must support before writing a claim.

• Name the decision in one line at the top of the working notes: routing a task type to one agent, delegating a defined piece of work, or comparing agents for a selection. The decision sets the pass bar, because a profile that is good enough for low risk routing may be too thin for a selection decision.
• Write the requirements the decision needs: required skills, minimum level per skill, required tool access, and any hard limits that rule an agent out. Write these before reading any profile, so the bar is not fitted to the first profile seen.
• Set two choice values and record them in the profile notes: the freshness limit in days, and the minimum evidence per claim. The check script uses a freshness limit of 90 days and a minimum of one evidence entry per skill and per tool with access above none. Different values are allowed when the decision needs them; the values used must be stated, and a check run with different values must say so.

1. Fix the profile identity and version.

• Give the profile a stable agent_id that stays the same across versions. Use an identifier that already exists for the agent in the working context. Do not invent a registry identifier, and do not change the agent_id to mark a new version.
• Set version as a string that rises with each published revision, for example "1", "2", "3". A claim change, an access change, or a limits change each require a new version. A spelling fix with no claim change can keep the version, and the notes should say that is what happened.
• Set declared_at to the date the claims were last checked, in YYYY-MM-DD form. This date moves only when claims are rechecked in step 10. Copying an old profile forward with a new date and no recheck is a false claim about freshness.

1. List skills with a level for each one.

• Use one entry per skill, with three fields: name, level, and evidence. Keep names short, lower case, and hyphenated, for example web-search, data-cleaning, code-review. One name covers one skill. A combined entry such as research-and-writing hides which half is weak.
• Use only three levels, with fixed meanings:
• basic: handles simple, well defined cases of the skill, with review of the result advised.
• proficient: handles routine cases without step by step direction, and handles common edge cases.
• advanced: handles hard and ambiguous cases, and can judge the quality of work in this skill.
• Assign the level from evidence, not from aspiration. If the strongest evidence shows routine cases only, the level is proficient at most. If evidence covers a single simple case, the level is basic.
• List only skills the agent will accept work for. A skill the agent can do in principle but will not take tasks for belongs in the limits section, not in the skills list, because routing reads the skills list as an offer to take work.
• Check for duplicate names before checking. Two entries with the same skill name and different levels make the profile unreadable for routing, and the check script reports duplicates as errors.

1. List tools with the access each tool carries.

• Use one entry per tool, with three fields: name, access, and evidence. Use the tool name as it appears in the working context, so a reviewer can match the profile line to the tool list for the run.
• Use only four access values, with fixed meanings:
• none: the tool is named for completeness and the agent has no access to it. No evidence is required for a none claim.
• read: the agent can read, search, list, or fetch through the tool, and cannot change stored state through it.
• write: the agent can create or change stored state through the tool.
• execute: the agent can run code, commands, or actions through the tool.
• Claim the lowest access that covers the work the profile offers. If only read use has been tested, claim read, even if a higher access might be available. Access that has not been exercised is not evidence backed.
• Record scope inside the tool name line or a limits entry when access is partial, for example access to one project, one folder, or one account only. A bare write claim with a partial scope and no scope note will be read as full write access.
• Never put a secret value in a profile: no passwords, tokens, keys, or codes, in any field, including evidence references. An evidence reference names a record, such as a log identifier. It never carries the credential itself.

1. Write the limits plainly.

• limits is a list of short plain statements. Cover at least these areas when they apply: task types the agent will not take, domains or file types outside scope, size or volume caps, access the agent lacks that related tasks often need, and conditions that require a fresh check before work starts.
• Write each limit as a testable statement. "No access to private mailboxes" is testable. "Limited capability" is not testable and helps no routing decision.
• A profile with an empty limits list receives a warning from the check script, because every agent has limits and an empty list usually means the section was skipped. If the list is genuinely short, write the one or two limits that exist rather than leaving the list empty.
• Put known weak spots here as well: a skill listed at basic with a narrow range, a tool with rate caps, or a language or format the agent handles poorly. A weak spot stated in limits protects the routing decision. A weak spot found during a run damages trust in every other claim in the profile.

1. Attach evidence to every claim.

• Each skill entry, and each tool entry with access above none, needs at least one evidence entry. Each evidence entry has two fields: kind and ref.
• Use only four kinds:
• link: a URL a reviewer can open to see the work or the record.
• artifact: an identifier for a produced file, report, or other output that a reviewer can retrieve.
• test: an identifier for a named check, suite, or task run with a recorded result.
• log: an identifier for a run record or transcript that shows the work.
• Match evidence strength to claim strength. An advanced level claim needs evidence from a hard case, not three easy ones. A write or execute access claim needs evidence that shows a change or a run, not a read only record.
• Evidence must point at work the profile is claiming. A record from a different agent, a different tool, or a supervised trial where the claimed skill was supplied from outside the agent does not support the claim. Label guided or assisted work as guided in the notes, and set the level from the unguided evidence.
• Date the evidence in the record itself where the record allows it. Evidence older than the freshness limit can still be listed, and the profile notes should say why it still counts, for example the tool and the task type have not changed since that record.
• If a claim has no evidence, there are two allowed moves: gather evidence by running a small check task, or drop the claim or lower its level to match the evidence that exists. Leaving an unevidenced claim in a delivered profile is not an allowed move, and the check script reports it as an error.

1. Set the freshness state.

• Freshness is computed from declared_at and a stated as-of date, not from memory. Age in days is the as-of date minus declared_at. The profile is stale when age is over the freshness limit, which the check script sets at 90 days.
• A stale profile can still be read, and its claims should be treated as unverified until a recheck in step 10. Routing a high risk task from a stale profile without a recheck is a decision the reviewer owns, and the decision record in step 9 must state that the profile was stale.
• Some events make a profile stale at once, whatever its age: a tool is added or removed, access is granted or revoked, a skill claim fails in a real run, or the limits change. On any of these events, the profile needs a new version and a recheck before further routing use.

1. Run the check script and fix every error.

• Save the script from the Operating Rules section as profile_check.py. Save the profile as a JSON file. Run the check with a fixed as-of date so the result can be repeated, for example python3 profile_check.py profile.json --as-of 2026-10-15.
• Read the output in this order: errors first, then warnings, then the counts. Exit code 1 means at least one error exists. Exit code 0 with warnings means the profile passed the mechanical checks and still carries notes a reviewer must read, most often a stale warning.
• Fix errors at the source: add the missing evidence, correct a bad level or access value, remove a duplicate entry, or drop the claim. Do not edit the script to silence a check that is pointing at a real defect.
• Re-run the check after fixes and keep the final output for the Output Contract. A profile is ready for a routing decision only when the final run shows 0 errors and verdict PASS.

1. Use the profile for the routing, delegation, or selection decision.

• Build a match table from the requirements written in step 1: one row per requirement, with the profile claim, the level or access found, the evidence count, and a verdict of met, missing, or below bar.
• Treat missing and below bar the same way: the profile does not support assigning that part of the work. Assign a smaller part that the profile does support, pick a different profile, or record an explicit exception with the reason. Silent exceptions are not allowed; the decision record must carry the exception and the reason.
• Flag excess access as a finding, not as a bonus. A profile that claims write or execute access the task does not need raises the risk of the assignment, and the decision record should note the excess and any guard placed on it.
• Compare profiles on the same fields: level per required skill, evidence count per claim, age in days, and limits that touch the task. Do not compare on total skill count. A long skills list with thin evidence loses to a short list with strong evidence for the required skills.
• Record the decision: the task, the profile agent_id and version used, the as-of date, the stale state, the match table, and the outcome. A decision recorded without a profile version cannot be audited later, because the profile will have changed.

1. Refresh, version, and retire profiles.

• Recheck on a fixed cadence set by the freshness limit, and at once after any event listed in step 7. A recheck means: re-read each claim, confirm or replace its evidence, confirm each access value still holds, and update the limits.
• After a recheck, raise version, set declared_at to the recheck date, and run the check script again. Keep the prior version and its check output in the working record. The history shows whether claims are stable, rising, or repeatedly failing checks.
• Downgrade claims that the recheck cannot support. A level drop or a removed skill is a normal outcome of an honest recheck, and a profile history with no downgrades over many versions is a sign the rechecks are not being done.
• Retire a profile when the agent stops taking work or when its claims no longer describe the agent. Mark the retired state and the date in the working record, and stop routing from it. Do not delete the record; a past decision that used the profile still needs it for audit.

Output Contract

Deliver the profile document plus the check output and a decision ready summary, with these sections, in this order:

1. Profile header: agent_id, version, declared_at, the as-of date used for the check, age in days, and stale state (yes or no).
2. Profile document: the full JSON profile with skills, tools, and limits. Each skill shows name, level, and evidence. Each tool shows name, access, and evidence when access is above none.
3. Check output: the full output of profile_check.py on the delivered profile. On the worked fixture in the Operating Rules section the headline numbers are: skills 4, levels basic 1 and proficient 2 and advanced 1, skills without evidence 1, tools 3, access read 1 and write 1 and execute 1, tools without evidence 1, limits 2, evidence entries 6, errors 2, warnings 0, verdict FAIL. A delivered profile should show errors 0, verdict PASS, skills without evidence 0, tools without evidence 0, and stale no, or carry a stated exception for any value that differs.
4. Match table: one row per requirement from Workflow step 1, with the claim found, evidence count, and a verdict of met, missing, or below bar. This section is required when the profile was built for a specific decision, and can be marked as general use when it was not.
5. Downgrades and gaps: claims dropped or lowered while building or checking the profile, claims with no evidence that were removed, and known weak spots carried into the limits list.
6. Freshness record: the freshness limit used, the next recheck date (declared_at plus the limit), and any event that would force an early recheck for this profile.
7. Limits: limits that affect likely routing decisions, restated in plain text outside the JSON, so a reviewer who reads only the summary still sees them.

Operating Rules

1. Every claim needs evidence. A skill entry with no evidence and a tool entry with access above none and no evidence are errors, not style choices. The check script enforces this rule mechanically, and a delivered profile shows 0 such errors.
2. No secret values in a profile. Passwords, tokens, keys, and codes never appear in any field, including evidence references and limits text. Naming a tool or a record is allowed. Giving its credential is not.
3. Dates are dates. declared_at and every as-of date use YYYY-MM-DD form and refer to the date claims were checked or the date the check was run. A profile with a missing date, a malformed date, or a declared date after the as-of date fails the check.
4. Names are unique within their list. Duplicate skill names and duplicate tool names are errors. Routing reads one entry per name, and a second entry with the same name makes the claim ambiguous.
5. Access claims follow tested use. Claim only the access that evidence shows. Guidance on level wording, freshness limits, and evidence kinds is a choice: pick values that fit the decision, record the choice in the profile notes, and keep the choice fixed for all profiles compared in one decision.
6. Absolute rules in this skill are evidence per claim, no secret values, unique names, valid level and access values, and valid dates, because breaking any of them makes a profile mechanically unreadable for routing. Level definitions, the 90 day freshness limit, and the four evidence kinds are guidance set by this skill: a different set can be used when a decision needs it, if the set is stated in the profile notes and applied to every profile in that decision.
7. No invented registries, services, or calls. A profile is a JSON document and a check result. This skill invents no registry to store it in, no service to fetch it from, and no call to validate it beyond the inlined script. Where a profile is kept is a local choice, recorded in the working record.
8. Use the profile check script below for the mechanical checks. Save it as profile_check.py, run it as its header describes, and put its output in the delivered summary. Report numbers from its output, not from memory of the profile.

Profile check (profile_check.py)

Run: python3 profile_check.py --make-fixture --as-of 2026-10-15 reproduces the worked example. Run it on a real profile with python3 profile_check.py profile.json --as-of 2026-10-15. If --as-of is omitted, the current date is used, and the result cannot be repeated on a later day.

Expected output for the fixture run:

text
agent_id: research-agent-07
version: 3
declared_at: 2026-10-01
age_days: 14
stale: no
skills: 4: code-review, data-cleaning, report-writing, web-search
levels: basic 1, proficient 2, advanced 1
skills_without_evidence: 1: report-writing
tools: 3: browser, payment-api, python
access: none 0, read 1, write 1, execute 1
tools_without_evidence: 1: payment-api
limits: 2
evidence_entries: 6
errors: 2
  error: skill report-writing: no evidence
  error: tool payment-api: no evidence
warnings: 0
verdict: FAIL

The fixture is built to trip the evidence rule twice: the report-writing skill carries an empty evidence list, and the payment-api tool claims write access with an empty evidence list. All other fixture claims carry evidence, the date math gives an age of 14 days for a declared date of 2026-10-01 checked as of 2026-10-15, and the profile is not stale under the 90 day limit.

profile_check.py (uses only the Python standard library). Save this code as profile_check.py:

python
#!/usr/bin/env python3
"""Check an agent capability profile for completeness and freshness.

Usage:
  python3 profile_check.py profile.json --as-of 2026-10-15
  python3 profile_check.py --make-fixture --as-of 2026-10-15

The profile is a JSON object with these top level fields:
  agent_id (string), version (string), declared_at (YYYY-MM-DD),
  skills (list), tools (list), limits (list).

Each skill entry needs name, level, and evidence.
Level must be one of: basic, proficient, advanced.
Each tool entry needs name, access, and evidence when access is not none.
Access must be one of: none, read, write, execute.
Each evidence entry needs kind and ref.
Kind must be one of: link, artifact, test, log.

Freshness: age is as_of minus declared_at in days.
A profile is stale when age is over 90 days.
Exit code is 0 when there are no errors, 1 when there is at least one error.
Warnings do not change the exit code.
"""
import argparse
import json
import sys
from datetime import date

LEVELS = {"basic", "proficient", "advanced"}
ACCESSES = {"none", "read", "write", "execute"}
KINDS = {"link", "artifact", "test", "log"}
STALE_DAYS = 90

FIXTURE = {
    "agent_id": "research-agent-07",
    "version": "3",
    "declared_at": "2026-10-01",
    "skills": [
        {"name": "web-search", "level": "proficient",
         "evidence": [{"kind": "log", "ref": "run-2026-09-28-search"}]},
        {"name": "data-cleaning", "level": "advanced",
         "evidence": [{"kind": "artifact", "ref": "clean-report-v2"},
                      {"kind": "test", "ref": "clean-suite-14"}]},
        {"name": "report-writing", "level": "basic", "evidence": []},
        {"name": "code-review", "level": "proficient",
         "evidence": [{"kind": "link", "ref": "https://example.org/reviews/41"}]},
    ],
    "tools": [
        {"name": "browser", "access": "read",
         "evidence": [{"kind": "log", "ref": "browser-check-09"}]},
        {"name": "python", "access": "execute",
         "evidence": [{"kind": "test", "ref": "python-smoke-03"}]},
        {"name": "payment-api", "access": "write", "evidence": []},
    ],
    "limits": [
        "No access to private mailboxes.",
        "Code review covers Python only.",
    ],
}


def parse_date(text):
    parts = text.split("-")
    if len(parts) != 3:
        return None
    try:
        return date(int(parts[0]), int(parts[1]), int(parts[2]))
    except ValueError:
        return None


def check_evidence(evidence, label, errors, warnings):
    if not isinstance(evidence, list) or len(evidence) == 0:
        errors.append(f"{label}: no evidence")
        return 0
    good = 0
    for entry in evidence:
        if not isinstance(entry, dict):
            errors.append(f"{label}: evidence entry is not an object")
            continue
        kind = entry.get("kind")
        ref = entry.get("ref")
        if kind not in KINDS:
            errors.append(f"{label}: bad evidence kind {kind!r}")
            continue
        if not isinstance(ref, str) or not ref.strip():
            errors.append(f"{label}: evidence ref is empty")
            continue
        good += 1
    return good


def check_profile(profile, as_of):
    errors = []
    warnings = []
    if not isinstance(profile, dict):
        return ["profile is not a JSON object"], warnings, {}
    agent_id = profile.get("agent_id")
    if not isinstance(agent_id, str) or not agent_id.strip():
        errors.append("agent_id: missing or empty")
        agent_id = "-"
    version = profile.get("version")
    if not isinstance(version, str) or not version.strip():
        errors.append("version: missing or empty")
        version = "-"
    declared_raw = profile.get("declared_at")
    declared = parse_date(declared_raw) if isinstance(declared_raw, str) else None
    if declared is None:
        errors.append("declared_at: missing or not YYYY-MM-DD")
        age_days = -1
        stale = "unknown"
    else:
        age_days = (as_of - declared).days
        if age_days < 0:
            errors.append("declared_at: date is after as_of")
            stale = "unknown"
        elif age_days > STALE_DAYS:
            stale = "yes"
            warnings.append(f"profile is stale: age {age_days} days over limit {STALE_DAYS}")
        else:
            stale = "no"
    skills = profile.get("skills")
    if not isinstance(skills, list):
        errors.append("skills: missing or not a list")
        skills = []
    tools = profile.get("tools")
    if not isinstance(tools, list):
        errors.append("tools: missing or not a list")
        tools = []
    limits = profile.get("limits")
    if not isinstance(limits, list):
        errors.append("limits: missing or not a list")
        limits = []
    elif len(limits) == 0:
        warnings.append("limits: empty list, no limits declared")

    skill_names = []
    skills_without = []
    evidence_total = 0
    level_counts = {"basic": 0, "proficient": 0, "advanced": 0}
    for entry in skills:
        if not isinstance(entry, dict):
            errors.append("skills: entry is not an object")
            continue
        name = entry.get("name")
        if not isinstance(name, str) or not name.strip():
            errors.append("skills: entry with missing name")
            continue
        skill_names.append(name)
        level = entry.get("level")
        if level not in LEVELS:
            errors.append(f"skill {name}: bad level {level!r}")
        else:
            level_counts[level] += 1
        before = len(errors)
        evidence_total += check_evidence(entry.get("evidence"), f"skill {name}", errors, warnings)
        if len(errors) > before:
            skills_without.append(name)
    dup_skills = sorted({n for n in skill_names if skill_names.count(n) > 1})

    tool_names = []
    tools_without = []
    access_counts = {"none": 0, "read": 0, "write": 0, "execute": 0}
    for entry in tools:
        if not isinstance(entry, dict):
            errors.append("tools: entry is not an object")
            continue
        name = entry.get("name")
        if not isinstance(name, str) or not name.strip():
            errors.append("tools: entry with missing name")
            continue
        tool_names.append(name)
        access = entry.get("access")
        if access not in ACCESSES:
            errors.append(f"tool {name}: bad access {access!r}")
            continue
        access_counts[access] += 1
        if access == "none":
            continue
        before = len(errors)
        evidence_total += check_evidence(entry.get("evidence"), f"tool {name}", errors, warnings)
        if len(errors) > before:
            tools_without.append(name)
    dup_tools = sorted({n for n in tool_names if tool_names.count(n) > 1})
    for n in dup_skills:
        errors.append(f"skills: duplicate name {n}")
    for n in dup_tools:
        errors.append(f"tools: duplicate name {n}")

    stats = {
        "agent_id": agent_id,
        "version": version,
        "declared_at": declared_raw if isinstance(declared_raw, str) else "-",
        "age_days": age_days,
        "stale": stale,
        "skills": len(skills),
        "skill_names": sorted(skill_names),
        "level_counts": level_counts,
        "skills_without": sorted(skills_without),
        "tools": len(tools),
        "tool_names": sorted(tool_names),
        "access_counts": access_counts,
        "tools_without": sorted(tools_without),
        "limits": len(limits),
        "evidence_total": evidence_total,
        "dup_skills": dup_skills,
        "dup_tools": dup_tools,
    }
    return errors, warnings, stats


def print_report(errors, warnings, stats):
    print(f"agent_id: {stats.get('agent_id', '-')}")
    print(f"version: {stats.get('version', '-')}")
    print(f"declared_at: {stats.get('declared_at', '-')}")
    print(f"age_days: {stats.get('age_days', -1)}")
    print(f"stale: {stats.get('stale', 'unknown')}")
    print(f"skills: {stats.get('skills', 0)}: {', '.join(stats.get('skill_names', [])) if stats.get('skill_names') else '-'}")
    lc = stats.get("level_counts", {})
    print(f"levels: basic {lc.get('basic', 0)}, proficient {lc.get('proficient', 0)}, advanced {lc.get('advanced', 0)}")
    sw = stats.get("skills_without", [])
    print(f"skills_without_evidence: {len(sw)}: {', '.join(sw) if sw else '-'}")
    print(f"tools: {stats.get('tools', 0)}: {', '.join(stats.get('tool_names', [])) if stats.get('tool_names') else '-'}")
    ac = stats.get("access_counts", {})
    print(f"access: none {ac.get('none', 0)}, read {ac.get('read', 0)}, write {ac.get('write', 0)}, execute {ac.get('execute', 0)}")
    tw = stats.get("tools_without", [])
    print(f"tools_without_evidence: {len(tw)}: {', '.join(tw) if tw else '-'}")
    print(f"limits: {stats.get('limits', 0)}")
    print(f"evidence_entries: {stats.get('evidence_total', 0)}")
    print(f"errors: {len(errors)}")
    for e in errors:
        print(f"  error: {e}")
    print(f"warnings: {len(warnings)}")
    for w in warnings:
        print(f"  warning: {w}")
    verdict = "FAIL" if errors else "PASS"
    print(f"verdict: {verdict}")


def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("path", nargs="?", default=None)
    parser.add_argument("--make-fixture", action="store_true")
    parser.add_argument("--as-of", default=None)
    args = parser.parse_args()
    if args.as_of:
        as_of = parse_date(args.as_of)
        if as_of is None:
            print("error: --as-of must be YYYY-MM-DD")
            return 2
    else:
        as_of = date.today()
    if args.make_fixture:
        profile = FIXTURE
    elif args.path:
        try:
            with open(args.path, "r", encoding="utf-8") as fh:
                profile = json.load(fh)
        except (OSError, json.JSONDecodeError) as exc:
            print(f"error: cannot read profile: {exc}")
            return 2
    else:
        print("error: give a profile path or --make-fixture")
        return 2
    errors, warnings, stats = check_profile(profile, as_of)
    print_report(errors, warnings, stats)
    return 1 if errors else 0


if __name__ == "__main__":
    raise SystemExit(main())