video_upscaling_restoration v1.0.0

name: "video_upscaling_restoration" description: "Restore and upscale low-quality footage: pick the right AI upscaler, order deinterlace/denoise/upscale correctly, avoid common artifacts, and know when to leave the original alone. Triggers on old footage cleanup, VHS digitization, upscaling to HD/4K, grainy or interlaced video repair."

Video Upscaling & Restoration

Purpose

Turn low-quality video (old digitizations, VHS captures, compressed downloads, low-res clips) into the best version it can be, without the overprocessed plastic look. The skill covers the full restoration order of operations, AI upscaler selection by material type, free and paid tooling, and the judgment calls where restoration should stop.

Workflow

1. Assess before touching anything

• Probe the file: ffprobe -v error -select_streams v:0 -show_entries stream=width,height,r_frame_rate,avg_frame_rate,field_order,codec_name -of default=noprint_wrappers=1 input.mp4
• Watch a 10-second sample at full resolution. Catalog the defects: interlacing (comb teeth on motion), noise, compression blocking, blur, telecine judder, bad color, shake.
• Write down the plan per defect. Never start with "enhance" as a single button.

2. Archive the original first

Copy the source to a lossless or high-bitrate working copy (FFV1 in MKV is cheap insurance). Every destructive step runs on the copy, never on the only master. Intermediates between stages should be high-bitrate (CRF 14-16 H.264) or lossless, never a delivery codec.

3. Deinterlace (only if actually interlaced)

• Confirm field_order is not progressive before touching the filter. Deinterlacing progressive footage damages it (shimmer on moving edges).
• Quality-first: QTGMC via VapourSynth (Hybrid or StaxRip wrap this well). It doubles frame rate; halve back for single-rate output.
• FFmpeg-only substitute: bwdif=mode=1:parity=0:deint=1 (doubles to 59.94p). Often visually indistinguishable from QTGMC on ordinary material.
• Telecine (film-on-NTSC judder): inverse telecine with fieldmatch=order=auto:field=auto,decimate, restoring the native 23.976 fps. Do NOT deinterlace film telecine.

4. Repair the picture before enhancing it

Order inside the chain matters: denoise FIRST (denoising after sharpening just sharpens the noise), then compression artifact reduction, then color correction.

• Light denoise: hqdn3d or Topaz Nyx. Heavy VHS grain: nlmeans.
• Neat Video is the take-by-take heavy option when footage quality varies across the tape.
• Color/levels AFTER denoise, not before. Grain, if you re-add any, goes LAST, at final output resolution (grain added before an upscale gets magnified).

5. Upscale with an AI model matched to the material

See references/model-picker.md for the full table. The short version:

• Topaz Video AI: Proteus = general default, Artemis = old/degraded (VHS, film grain), Iris = faces and close-ups, Nyx = noisy/dark footage, Starlight = heavy degradation, Astra = AI-generated footage.
• Free/local: Real-ESRGAN (per-frame; fine for images, shimmers on video unless mitigated) or Video2X.
• Always test on a 5-15 second range with faces, motion, texture, and hard lighting before committing to the full render. Judge one exported frame against the source frame, side by side.

6. Frame interpolation is opt-in, not default

• Never auto-convert 24 fps footage to 60 fps. Interpolation (Topaz Apollo/Aion, RIFE) can produce rubbery motion, duplicated limbs, unstable backgrounds.
• Keep native frame rate for documentary realism, anime, stop-motion, and anything with deliberate judder. Interpolate only for smooth slow-motion or explicit delivery requirements.

7. Stop before it looks synthetic

The stop rule: improve one defect at a time, export a comparison, and stop when the source starts looking artificial. Wax skin, shimmering water, painterly backgrounds, and unstable faces all mean you went one step too far. Back off one stage, not one slider.

Output Contract

• A restoration that reads as the original footage at its best, not a different video. No combing, no blocking, no wax faces, native frame rate preserved unless the user asked otherwise.
• The untouched original archived, the staged intermediates reproducible (keep the filter chain and model names in a log file next to the output).
• A short verdict on anything you chose NOT to fix and why (source was too degraded, motion blur cannot be undone, AI detail would be fabrication).

Operating Rules

1. Deinterlace is conditional: verify interlacing first, never run it blind.
2. Order is deinterlace -> denoise -> color -> upscale -> (optional) interpolate. Never sharpen or upscale noise.
3. One defect, one pass, one comparison. No stacked mystery settings.
4. Test ranges first: 5-15 seconds, representative material, judged against source frames.
5. Never invent model capabilities or version numbers; confirm the current model list and API schema from the vendor before promising a specific one.
6. Motion blur, missed focus, and bad lighting cannot be fixed by upscaling; they only get more visible. Say so instead of promising.
7. For documentary, journalistic, or evidentiary footage: AI-invented detail is fabrication. Clean the signal, do not generate a new one.