--- name: pdf_form_filling description: "Fill PDF forms from data without breaking them: inspect the form and list every field with type and required status, map data to fields on a written mapping sheet, fill with a script that preserves the form, flatten only when the job calls for it, and verify by reading values back and rendering pages. Triggers on: fill this PDF form, complete a fillable form, batch fill forms from a spreadsheet, flatten a filled form, why is my checkbox empty, PDF form fields not showing values." --- # PDF Form Filling ## Purpose Fill fillable PDF forms (AcroForm) from structured data and prove the result is right. The workflow is inspect first, map on paper, fill with validation, then verify twice: read the stored values back, and render the pages to check the values actually show and fit. It covers single fills from JSON, batch fills from CSV, and the separate decision to flatten a copy for filing or sending. The scripts use pypdf, a pure Python library with no system binaries. Install it with: pip install pypdf ## Workflow ### 1. Inspect the form before touching it - Run the inspector on every new form, and again whenever the form's version changes: ``` python3 ~/workspace/skills/pdf-form-filling/scripts/inspect_form.py form.pdf --json fields.json ``` - Read the full field list: name, type (text, checkbox, radio group, choice, signature), required and read-only flags, current value, allowed values for choice and button fields, character limit, and the pages each field appears on. - Stop if the inspector reports NO_FIELDS. The file is a flat PDF that only looks like a form, or an XFA-only form. Neither can be filled as data. Get the fillable AcroForm original instead of typing on top of the picture. - Note required fields and character limits now. Both are cheaper to plan around than to hit mid-fill. - Use fully qualified field names. Names can repeat under different parents, as sender.city and receiver.city, and the dotted name is the only safe address. ### 2. Write the mapping sheet - Build the sheet from the inspection output, one row per field: field name, type, required, source, transform, example value, notes. The format and a worked example are in `references/mapping-sheet.md`. - Pull allowed values from the inspection, never from the labels printed beside the fields on the page. A dropdown labeled with full state names may store two letter codes, and a checkbox's checked value is defined inside the form. - Leave fields the job does not fill out of the data entirely. An absent field stays untouched; an empty string in the map invites confusion about whether it was cleared on purpose. ### 3. Fill, with validation in front Single fill from JSON: ``` python3 ~/workspace/skills/pdf-form-filling/scripts/fill_form.py template.pdf --data values.json --out filled.pdf ``` Batch fill from CSV, one output per row: ``` python3 ~/workspace/skills/pdf-form-filling/scripts/fill_form.py template.pdf --batch rows.csv --out-dir out/ --name-from id ``` - The script validates the whole record before writing anything: unknown field names, choice and radio values outside the allowed set, values longer than a field's character limit, and required fields left empty all stop the run with the reasons printed. Fix the data or the map, not the script's checks. The --allow-extra and --allow-missing-required flags exist for jobs that genuinely need them; say why when you use one. - Give checkbox values as true or false. The script reads the form's own on-state (often /Yes, sometimes /On, /1, or a custom name) and stores that. Never write a state name into your data by hand: a wrong name stores silently and the box renders empty in every viewer. - Give radio values as the option name from the inspection, with or without the leading slash. - The fill preserves the form. Output stays a working fillable form with the same fields, filled. Nothing is flattened unless step 5 asks for it. ### 4. Verify by read-back - Run the verifier against the values you intended: ``` python3 ~/workspace/skills/pdf-form-filling/scripts/verify_form.py filled.pdf --expected values.json ``` - Every field must pass. A fail means the stored value differs from the intended one; investigate before sending anything. Read-back catches wrong-field fills, bad option values, and values a viewer would never show. - For batch jobs, verify a sample that includes the first row, the last row, and any row with unusual values (longest name, every optional field filled, every optional field empty). ### 5. Flatten only when the job calls for it - Decide on purpose, using `references/flatten-and-failure-modes.md`. Filing systems that require it, final record copies, and print-bound sends get flattened. Anything anyone still needs to edit or extract data from does not. - Keep the unflattened filled PDF as the record. Flatten a copy: ``` python3 ~/workspace/skills/pdf-form-filling/scripts/fill_form.py template.pdf --data values.json --out filed-copy.pdf --flatten python3 ~/workspace/skills/pdf-form-filling/scripts/verify_form.py filed-copy.pdf --expected values.json --require-flattened ``` - The flattened check proves no fields remain. It cannot prove the values survived, because flattened values are page content, not fields. That proof comes from step 6. ### 6. Render and look at the pages - Render every page of the final file: pdftoppm -png -r 100 final.pdf page - Look at each rendered page. Confirm every value sits inside its field box with nothing clipped, checkboxes show their mark, and the marked radio option is the intended one. - Treat the radio check as mandatory after flattening. pypdf 6.21.4 can drop a selected radio mark during flattening while the stored value was correct, a failure only the render shows. fill_form.py warns when it flattens a form that has a radio group. If a mark is missing, keep the unflattened fill and flatten it with a viewer-grade tool such as qpdf (qpdf --generate-appearances --flatten-annotations=all), then render and check again. - For flattened files, also run pdftotext on the output and confirm the values appear as page text. ## Output Contract A finished job delivers: - The filled PDF, still a working form, plus a flattened copy only when the job called for one. - The mapping sheet and the data file used, kept with the job so the next run can repeat or audit the fill. - The verifier output showing every checked field passing, and a note that the rendered pages were looked at, with any field that clipped or rendered oddly named. - A one-line summary: form name, fields filled out of fields total, single or batch (with output count), flattened or left fillable. ## Operating Rules 1. Never fill a form you have not inspected in this run. Field names, options, and limits come from the inspection output, not from the page labels and not from memory of a similar form. 2. Never hardcode a checkbox state. On-states belong to the individual form. Read them from the form or let the script convert true and false. 3. Validation failures stop the job. Do not loosen the checks to make a bad data file pass; fix the data or the mapping sheet. 4. Flattening is one way. The unflattened filled copy is the record and is never the file you flatten in place. 5. Read-back is not the whole verification. Stored values can be right while the page renders wrong (clipped text, an empty-looking box, a lost radio mark). The render check is part of done, not an extra. 6. Signature fields hold digital signatures, not typed names. Filling one with text does not sign anything. If the job needs a real signature, use an e-signature workflow instead and say so. 7. Form data is often personal data (names, addresses, ID numbers, health and tax details). Keep data files and filled forms in the job's own folder, and do not paste field values into unrelated chats, tickets, or logs. 8. If the form fights back (XFA packet, fields missing from the field list but visible as page widgets, a viewer showing values the read-back denies), stop and report what the inspection and read-back actually showed. Do not stack workarounds on a form you do not understand yet.