archival_research_basics v1.0.0

name: archival_research_basics description: "Work an archive collection well: read a finding aid, scope a request to boxes and folders, handle material with care, record provenance, cite items so they can be re-requested, and log restrictions before planning any use. Use when visiting an archive, preparing a request from a finding aid, or recording and citing archival material for later work." metadata: { "includeInPrompt": true }

Archival Research Basics

Purpose

Give archival work a record that another researcher can follow back to the same items. The routine starts with a finding aid, narrows a broad interest to named boxes and folders, tracks what was requested and how it was handled, records provenance for every item used, writes citations that support a fresh request for the same item, and logs restrictions before any use is planned. A citation that lacks a repository, a collection, a box, or a folder does not point anywhere precise, so the work treats those four fields as required from the start.

Scope:

• deep-research-workflows runs web research streams and triangulates web claims. This skill does not run web research streams.
• public-records-lookup searches US registries, courts, and filings. This skill does not search those sources.
• research-methods is general source-quality rigor. This skill applies rigor to one setting only, the archive collection.
• field-notes-basics is the researcher's own observation record. This skill records the material consulted and its provenance, which is a different record from personal observations in the field.
• This skill works archive collections: finding aids, collection scope, requesting and handling material, provenance and citation of archival items, and noting restrictions.
• Verification note, 2026-10-21: archival_check.py below was run with Python 3.12.3, the version python3 --version returned on the drafting machine, against a fixture built by the script itself with --make-fixture. The fixture run returned entries_total: 7, entries_with_all_fields: 3, entries_missing_fields: 4, citations_rerequestable: 3, citations_not_rerequestable: 4, restriction_conflicts: 1, duplicate_ids: 1, verdict: REVIEW. The script uses only the Python standard library (os and sys), so no package version is quoted and no install step is needed. What was NOT verified: no specific archive's reading room rules, request system, or catalog were tested. Rules vary by repository and this draft says to confirm them on site.

Workflow

1. Start with a question and a short scope note.

• Write the question the visit should answer in one or two plain sentences, plus the date range and the kinds of material that could answer it.
• Write what sits outside the visit as well. A collection can hold far more than one visit can cover, and a stated limit keeps the request in step 3 narrow.
• Name the output the visit feeds, such as a report, an article, or an exhibit, because that output sets how much provenance and citation detail each item needs.
• If the question is still broad, such as a whole life or a whole organisation over decades, split it into a first visit question and a list of later questions before searching for a collection.

1. Find candidate collections, then pick by fit.

• Search the repository catalog and any collection guides it publishes, using names, places, organisations, and subject terms from the scope note.
• Record each candidate with its collection title, its repository, and the reason it might fit. Keep weak candidates on the list with the reason marked weak, rather than dropping them silently.
• Prefer a collection whose dates and scope cover the question over a larger collection that only touches it at the edges.
• When two collections look equal, pick the one with the clearer finding aid and the fewer access limits for the first visit, and note the other for later.

1. Read the finding aid in full before requesting anything.

• Collection title and identifier: record both exactly as the finding aid gives them. The identifier is what request forms and citations rely on.
• Dates: record the inclusive dates and note any bulk dates the aid states. A mismatch between the collection dates and the question dates is a reason to pause and recheck fit.
• Extent: record the size as stated, such as a number of boxes or linear feet. Extent sets the scale of the request. A large extent means step 4 matters more, not less.
• Arrangement: record how the material is ordered, such as by series, by date, or by correspondent. Arrangement tells you where a given item is likely to sit and why neighbours sit beside it.
• Scope and content note: record what the aid says the collection holds, and what it says it lacks. Noted gaps save a wasted request.
• Access notes: record every stated limit before planning the visit, including closed material, advance notice periods, and any permission step. Step 9 returns to these notes.
• Container list: when the aid gives one, work from it. It names the boxes and folders the request in step 4 will cite.

1. Scope the request to boxes and folders.

• List the specific boxes and folders that match the question, taken from the container list or the arrangement note. Request named parts, not the whole collection.
• Give each requested unit a one line reason tied to the scope note. A reason per box makes a later cut easy to defend and a later return visit easy to plan.
• Order the list by expected value. If time runs short, the strongest boxes have already been seen.
• Set a plain cap for the visit based on extent and available time, and write the cap beside the list. Material left unrequested goes on a named return list, not into a silent hope.

1. Confirm local rules on site or on the repository site before the visit.

• Confirm the reading room rules for that repository: hours, registration steps, what may be brought in, and how requests are placed and timed. Do not carry rules from one repository into another.
• Confirm handling rules as stated there: supports for bound volumes, glove use for photographs, pencil or device rules, and copying or photography limits.
• Confirm request limits, such as how many boxes may be out at once or how far ahead a request must be placed, and adjust the list in step 4 to fit.
• Write the confirmed rules at the top of the research log for that visit. Rules that were not confirmed get marked unconfirmed, and staff are asked on arrival.

1. Request material and log each request as it happens.

• Submit requests using the collection identifier plus the box and folder numbers from step 4, in the form the repository asks for.
• Log each request in the research log at the time it is placed: repository, collection, box, folder, and time of request when the repository tracks it.
• Log what actually arrived as well. A substituted box, a folder reported missing, or a unit held back for condition reasons goes in the log beside the request, not in memory.
• Keep one research log per visit, dated, so logs from separate visits never merge into one unclear record.

1. Handle material with care and keep its order.

• Work with one box at a time and one folder at a time, unless staff state a different rule for that repository.
• Keep the original order inside each folder and each box. Order is part of the evidence. Do not sort, group, or set aside items in a new sequence.
• Take one folder out at a time where the repository uses place markers, use the marker, and return the folder to its exact place before taking the next.
• Keep items flat and supported, turn pages from a corner with a clean dry hand or as staff direct, and never force a binding, a fold, or a fastener.
• Note condition as you go: torn, brittle, faded, damp, mould suspected, or items loose that look misfiled. Report damage or suspected mould to staff and record it in the log beside that entry.
• Note where work stopped in a folder, so a return to the same folder starts cleanly and no item is handled twice by mistake.

1. Record provenance for every item used.

• For each item you copy, quote, photograph, or rely on, write a log entry in this block form, with fields in this order:
• ITEM: <item id> where the item id is a short code you assign once and never reuse for a different item.
• repository: <repository name>
• collection: <collection title>
• box: <box number>
• folder: <folder number or title>
• restriction: <none, or the restriction that applies>
• planned_use: <unrestricted, or the limited use planned>
• Add a short title or description line and a date line for the item itself when the item carries them, so the entry can be told apart from a near neighbour in the same folder.
• Record how the item was used during the visit, such as read only, notes taken, copy requested, or photographed where allowed, with the copy or image reference when one exists.
• Write provenance at the time of use. Reconstructing box and folder numbers later from notes is where most citation errors start.

1. Log restrictions before planning any use.

• For each entry, check the finding aid access notes and any notice supplied with the material, and record the restriction in plain words: closed, redacted, fragile, or permission needed, plus the date or condition that ends it when one is stated.
• Closed means do not plan to read, copy, or cite the item until the closure ends or a stated review grants access.
• Redacted means record which part was withheld when that can be seen, and do not fill the gap from guesswork.
• Fragile means plan handling and copying around the condition limit, and do not plan a use that needs repeated handling.
• Permission needed means name whose permission, for what use, and record whether it was asked for and granted. Planned publication, exhibition, broadcast, or online posting each count as a use that needs the check.
• Write the planned use beside each restriction. An item flagged with a restriction and planned for unrestricted use is a conflict, and step 10 flags it for correction before the use goes ahead.

1. Write citations that can be re-requested, then check the log.

• Cite each archival item with five parts, in this order: repository, collection, box, folder, item. The item part is the item title or description plus its date when it has one.
• Test each citation with one question: could another researcher hand this citation to the same repository and get the same item back? If a part is missing, the answer is no, and the citation goes back to step 8 for the missing field.
• Save the script from the Operating Rules section as archival_check.py, write the visit log in the block form from step 8, and run python3 archival_check.py LOG_FILE on it.
• Fix what the checker reports before planning further use: fill missing repository, collection, box, or folder fields from the finding aid or a fresh request, change a planned use that conflicts with a restriction, and merge or renumber duplicate item ids so each id names one item.
• Re-run the checker after fixes and keep the final output with the visit record. Numbers quoted later come from that output, not from memory of the log.

Output Contract

Deliver these items, in this order:

1. Scope note: the question from Workflow step 1, the date range and material kinds in scope, what was left out, and the output the visit feeds.
2. Finding aid summary: per collection consulted, the title and identifier, inclusive dates, extent, arrangement, scope note points, and access notes, taken from the finding aid itself.
3. Request list: the boxes and folders requested in step 4, in planned order, each with a one line reason and its request outcome from step 6.
4. Research log: the full dated log for the visit, one block per item used, in the block form from step 8, with condition notes from step 7 included.
5. Checker output: the full output of archival_check.py for the final log run. On the worked fixture in the Operating Rules section the headline numbers are: entries_total: 7, entries_with_all_fields: 3, entries_missing_fields: 4, citations_rerequestable: 3, citations_not_rerequestable: 4, restriction_conflicts: 1, duplicate_ids: 1, verdict: REVIEW.
6. Citation list: one citation per item used, each with repository, collection, box, folder, and item, in the form from step 10.
7. Restriction and use plan: one row per restricted item, with the restriction, its end date or condition when stated, the permission state when permission is needed, and the use actually planned.
8. Limits: anything the visit did not cover, stated plainly (for example boxes left on the return list, local rules that stayed unconfirmed, or items whose provenance fields could not be completed).

Operating Rules

1. Request parts, not wholes. Every request names boxes and folders from the finding aid. A request for a whole large collection without a scoped reason is a planning fault, not a shortcut.
2. Original order is kept. Items stay in the sequence the folder holds them in, folders stay in box order, and any apparent misfile is reported and logged, never corrected by hand.
3. Provenance is written at the time of use. An item with no log entry does not get cited, quoted, or planned for use until its entry exists with all four required fields.
4. The four citation fields are required: repository, collection, box, and folder. A citation missing any one of them cannot be re-requested and does not ship in the citation list.
5. Restrictions bind use. A restriction recorded in the log limits the planned use beside it until the restriction ends, a stated review grants access, or the named permission is granted and recorded.
6. Local rules are confirmed per repository. Reading room, handling, and copying rules from one repository are never assumed to hold at another. Unconfirmed rules are marked unconfirmed and checked on site.
7. Duplicate item ids are errors. Each assigned item id names one item only. A reused id gets resolved by merging the entries or renumbering, and the change is noted in the log.
8. Use the checker below for the mechanical checks. Save it as archival_check.py, run it as its header describes, and put its full output in the deliverable as item 5 of the Output Contract. Report numbers from its output, not from memory of the log.

Archival check (archival_check.py)

Run: python3 archival_check.py --make-fixture fixture builds the worked fixture in a folder named fixture, holding a research_log.txt file. Then run: python3 archival_check.py fixture/research_log.txt to reproduce the output below. Run it on a real visit with python3 archival_check.py LOG_FILE, where LOG_FILE is the dated research log for that visit in the block form from Workflow step 8.

Expected output for the fixture run:

text
log_file: fixture/research_log.txt
entries_total: 7
entries_with_all_fields: 3
entries_missing_fields: 4
ENTRY entry=1 id=ARC-001 missing=- citation_rerequestable=yes restriction_conflict=no duplicate=no
ENTRY entry=2 id=ARC-002 missing=repository citation_rerequestable=no restriction_conflict=no duplicate=no
ENTRY entry=3 id=ARC-003 missing=collection citation_rerequestable=no restriction_conflict=no duplicate=no
ENTRY entry=4 id=ARC-004 missing=box citation_rerequestable=no restriction_conflict=no duplicate=no
ENTRY entry=5 id=ARC-005 missing=folder citation_rerequestable=no restriction_conflict=no duplicate=no
ENTRY entry=6 id=ARC-006 missing=- citation_rerequestable=yes restriction_conflict=yes duplicate=no
ENTRY entry=7 id=ARC-001 missing=- citation_rerequestable=yes restriction_conflict=no duplicate=yes
citations_rerequestable: 3
citations_not_rerequestable: 4
restriction_conflicts: 1
duplicate_ids: 1
verdict: REVIEW

The fixture is built to show each checker result clearly: one complete entry passes, four entries each miss one required field (repository, collection, box, and folder in turn), one complete entry pairs a closed restriction with an unrestricted planned use, and one entry reuses the first item id. That gives entries_missing_fields: 4, citations_not_rerequestable: 4, restriction_conflicts: 1, duplicate_ids: 1, and a verdict of REVIEW.

archival_check.py (uses only the Python standard library). Save this code as archival_check.py:

python
#!/usr/bin/env python3
"""Check an archival research log for completeness and use conflicts.

Usage:
  python3 archival_check.py --make-fixture DIR
  python3 archival_check.py LOG_FILE

The research log is a plain text file made of entry blocks
separated by blank lines. Each block starts with an item line,
then field lines in the form key: value.

  ITEM: ARC-001
  repository: Example Repository
  collection: Example Papers
  box: 3
  folder: 12
  restriction: none
  planned_use: unrestricted

Required fields for a citation that can be re-requested are
repository, collection, box, and folder. A missing or empty
required field is reported per entry. An entry counts as a
restriction conflict when its restriction value is not none
or empty and its planned_use value is unrestricted. A repeated
item id counts as a duplicate from its second use onward.

Output lists each entry with its missing fields, citation
status, restriction conflict flag, and duplicate flag, then
prints summary counts. Exit code is 0 when the log file can
be read, 1 when it cannot. Standard library only.
"""
import os
import sys

REQUIRED = ["repository", "collection", "box", "folder"]
OPEN_RESTRICTIONS = ["", "none", "no", "open"]


def parse_log(text):
    entries = []
    blocks = []
    current = []
    for line in text.splitlines():
        if line.strip() == "":
            if current:
                blocks.append(current)
                current = []
        else:
            current.append(line)
    if current:
        blocks.append(current)
    for block in blocks:
        item_id = "-"
        fields = {}
        for line in block:
            stripped = line.strip()
            if ":" not in stripped:
                continue
            key, _, value = stripped.partition(":")
            key = key.strip().lower()
            value = value.strip()
            if key == "item":
                item_id = value if value else "-"
            else:
                fields[key] = value
        entries.append((item_id, fields))
    return entries


def check(log_path):
    if not os.path.isfile(log_path):
        print(f"error: log file not found: {log_path}")
        return 1
    with open(log_path, "r", encoding="utf-8", errors="replace") as fh:
        text = fh.read()
    entries = parse_log(text)
    seen = set()
    rows = []
    missing_count = 0
    rerequestable = 0
    conflicts = 0
    duplicates = 0
    for item_id, fields in entries:
        missing = [name for name in REQUIRED if not fields.get(name)]
        if missing:
            missing_count += 1
        citation_ok = not missing
        if citation_ok:
            rerequestable += 1
        restriction = fields.get("restriction", "").strip().lower()
        planned = fields.get("planned_use", "").strip().lower()
        conflict = restriction not in OPEN_RESTRICTIONS and planned == "unrestricted"
        if conflict:
            conflicts += 1
        duplicate = item_id != "-" and item_id in seen
        if duplicate:
            duplicates += 1
        if item_id != "-":
            seen.add(item_id)
        rows.append((item_id, missing, citation_ok, conflict, duplicate))
    total = len(entries)
    with_all = total - missing_count
    print(f"log_file: {log_path}")
    print(f"entries_total: {total}")
    print(f"entries_with_all_fields: {with_all}")
    print(f"entries_missing_fields: {missing_count}")
    for idx, (item_id, missing, citation_ok, conflict, duplicate) in enumerate(rows, start=1):
        missing_label = ",".join(missing) if missing else "-"
        print(
            f"ENTRY entry={idx} id={item_id} missing={missing_label} "
            f"citation_rerequestable={'yes' if citation_ok else 'no'} "
            f"restriction_conflict={'yes' if conflict else 'no'} "
            f"duplicate={'yes' if duplicate else 'no'}"
        )
    print(f"citations_rerequestable: {rerequestable}")
    print(f"citations_not_rerequestable: {total - rerequestable}")
    print(f"restriction_conflicts: {conflicts}")
    print(f"duplicate_ids: {duplicates}")
    has_issue = missing_count > 0 or conflicts > 0 or duplicates > 0
    print(f"verdict: {'REVIEW' if has_issue else 'PASS'}")
    return 0


def make_fixture(base):
    os.makedirs(base, exist_ok=True)
    log_path = os.path.join(base, "research_log.txt")
    blocks = [
        (
            "ITEM: ARC-001\n"
            "repository: Example Repository\n"
            "collection: Example Papers\n"
            "box: 3\n"
            "folder: 12\n"
            "restriction: none\n"
            "planned_use: unrestricted\n"
        ),
        (
            "ITEM: ARC-002\n"
            "collection: Example Papers\n"
            "box: 3\n"
            "folder: 14\n"
            "restriction: none\n"
            "planned_use: unrestricted\n"
        ),
        (
            "ITEM: ARC-003\n"
            "repository: Example Repository\n"
            "box: 4\n"
            "folder: 2\n"
            "restriction: none\n"
            "planned_use: unrestricted\n"
        ),
        (
            "ITEM: ARC-004\n"
            "repository: Example Repository\n"
            "collection: Example Papers\n"
            "folder: 7\n"
            "restriction: none\n"
            "planned_use: unrestricted\n"
        ),
        (
            "ITEM: ARC-005\n"
            "repository: Example Repository\n"
            "collection: Example Papers\n"
            "box: 5\n"
            "restriction: none\n"
            "planned_use: unrestricted\n"
        ),
        (
            "ITEM: ARC-006\n"
            "repository: Example Repository\n"
            "collection: Example Papers\n"
            "box: 6\n"
            "folder: 1\n"
            "restriction: closed\n"
            "planned_use: unrestricted\n"
        ),
        (
            "ITEM: ARC-001\n"
            "repository: Example Repository\n"
            "collection: Example Papers\n"
            "box: 3\n"
            "folder: 12\n"
            "restriction: none\n"
            "planned_use: unrestricted\n"
        ),
    ]
    with open(log_path, "w", encoding="utf-8") as fh:
        fh.write("\n".join(blocks))
    print(f"fixture_log: {log_path}")
    return 0


def main():
    args = sys.argv[1:]
    if len(args) == 2 and args[0] == "--make-fixture":
        return make_fixture(args[1])
    if len(args) != 1:
        print("usage: archival_check.py LOG_FILE")
        print("       archival_check.py --make-fixture DIR")
        return 2
    return check(args[0])


if __name__ == "__main__":
    raise SystemExit(main())