---
name: heuristic_evaluation
description: "Run an expert usability review with no participants: pick a named heuristic set, walk every main flow against each heuristic, log violations with location and evidence, rate severity, group findings, and write the report. Use when an interface needs a structured expert review before, or instead of, testing with people."
metadata: { "includeInPrompt": true }
---

# Heuristic Evaluation

## Purpose

Run a heuristic evaluation: one or more reviewers inspect an interface against a named set of usability heuristics, log every violation they find with evidence, rate its severity, and turn the grouped findings into a report a team can act on. No participants are recruited and no sessions are run. The reviewers supply the judgment, so the method stands or falls on how well each finding is documented.

Scope:
- usability-testing-basics is moderated tests with participants. accessibility-checklist is WCAG checks on HTML. This skill owns expert review of an interface against named heuristics.
- This skill covers choosing the heuristic set, walking flows, logging and rating violations, grouping duplicates, and writing the report. It does not cover running sessions with people, checking code against WCAG criteria, fixing the interface, or retesting a fix, although a report from this skill often feeds those jobs.
- A heuristic evaluation finds likely problems cheaply and early, including on prototypes that are not ready for participants. It does not prove that real users will hit a problem, how often they will hit it, or whether they can complete a task. State that limit in every report.

## Workflow

1. Pick the heuristic set and record it in full.
   - Default to Nielsen's 10 usability heuristics, the standard set for general interface review. The list below was checked on 2026-10-14 against the Nielsen Norman Group article at https://nngroup.com/articles/ten-usability-heuristics/ and the names are given as published there:
     1. Visibility of System Status
     2. Match Between the System and the Real World
     3. User Control and Freedom
     4. Consistency and Standards
     5. Error Prevention
     6. Recognition Rather than Recall
     7. Flexibility and Efficiency of Use
     8. Aesthetic and Minimalist Design
     9. Help Users Recognize, Diagnose, and Recover from Errors
     10. Help and Documentation
   - Treat the heuristics as broad principles for judging an interface, not as detailed rules that settle every case. Two reviewers can apply the same heuristic to the same screen and disagree. The evidence logged in step 5 is what makes a finding arguable or solid, not the heuristic name alone.
   - A short working gloss for each heuristic, in this skill's words. Use it while walking flows, and check the source article for the full treatment before a first evaluation:
     - Visibility of System Status: the interface tells the user what state it is in and what just happened, fast enough for the user to act on it.
     - Match Between the System and the Real World: wording, order, and concepts follow the user's world and language, not internal terms or system logic.
     - User Control and Freedom: the user can leave, cancel, undo, or back out of a state they entered by mistake, without a long detour.
     - Consistency and Standards: the same words, controls, and patterns mean the same thing across the product, and the product follows conventions its users already know from elsewhere.
     - Error Prevention: the design stops common errors before they happen, through constraints, good defaults, and confirmation at risky points.
     - Recognition Rather than Recall: options and information the user needs are visible or easy to get back to, so the user does not have to hold them in memory.
     - Flexibility and Efficiency of Use: experienced users have faster paths, such as shortcuts or repeat actions, while first-time users are not forced onto them.
     - Aesthetic and Minimalist Design: screens carry what the task needs and little else, so relevant content is not competing with clutter.
     - Help Users Recognize, Diagnose, and Recover from Errors: error messages say in plain language what went wrong and what to do next, close to where the error happened.
     - Help and Documentation: help exists, is easy to search, is written around the user's task, and gives concrete steps.
   - Other sets exist and suit narrower jobs, for example a set written for a specific product type or platform. If a different set is used, name it, give its source, list every item in the plan, and use that list for the whole evaluation. Do not mix items from two sets mid-review, and do not add a personal favorite heuristic without marking it as an addition.
   - Record in the plan: product and version or build under review, heuristic set and its source, reviewer names or IDs, date, devices and screen sizes covered, and the flows in step 2.

2. List the flows to walk before opening the interface.
   - A heuristic evaluation that only visits the home screen and the settings page misses most violations. Build the flow list from what the product is for:
     - Core tasks: the 3 to 6 jobs the product exists to do, each written as a goal, for example complete a purchase, find a past order, change account details.
     - First use: sign up, onboarding, empty states, and the first run of each core task with no data in place.
     - Return use: sign in again, resuming an unfinished task, and repeat actions a frequent user would take.
     - Error and edge paths: invalid input, failed payment or save, lost connection where it can be simulated, empty search results, and deleting or overwriting something.
     - Help paths: finding help from inside a task, and following it back to the task.
     - Destructive and hard to reverse actions: delete, cancel a subscription or order, send, publish, pay.
   - Give each flow an ID and a start and end state, for example F2 Checkout, starts at a cart with two items, ends at an order confirmation.
   - Walk every listed flow. If time forces a cut, cut a whole flow and record it as not reviewed. Half-walking every flow produces shallow findings everywhere and solid findings nowhere.

3. Set up the log before the first walk.
   Use one row per violation, in a spreadsheet or a CSV with these columns: ID, heuristic, flow, location, title, expected, actual, evidence, severity, reviewer.
   - Heuristic: the exact name from the chosen set. A violation can break more than one heuristic. Log it once under the heuristic it breaks most directly, and name the second one in the actual field rather than logging the same problem twice.
   - Location: screen or step name plus enough detail to find it again, for example Checkout / payment submit, or Settings / profile / save control. A location like "checkout" on its own is too broad to act on.
   - Evidence: a screenshot filename, a recording timestamp, or a short written description of the exact state, whichever the team can store and share. Write the description even when a screenshot exists, because screenshots get separated from reports.
   - Keep the log in the CSV shape the tally script in the Operating Rules section reads: heuristic, location, severity, title. Extra columns can sit alongside those four, but those four keep their names and order of meaning.

4. Walk each flow twice, in two different passes.
   - Pass one, the task pass: do the flow as a user would, start to finish, at normal speed. At each step ask what the user knows at this point, what they have to remember from earlier steps, what they can do if they make a mistake here, and what the interface tells them after each action. Log violations as they appear rather than holding them for the end of the flow.
   - Pass two, the heuristic pass: walk the same flow again slowly, one heuristic at a time, screen by screen. For each screen and each heuristic ask the matching question, for example for Error Prevention: what is the easiest wrong thing to do on this screen, and does the design stop it, warn about it, or let it through silently?
   - The two passes find different things. The task pass catches broken feedback, dead ends, and missing exits, because the reviewer feels them while moving. The heuristic pass catches consistency breaks, recall load, and clutter, which are easy to walk past when focused on completing the task.
   - Deliberately try the wrong thing at least once per flow: submit an empty or invalid form, go back mid-flow, cancel a dialog, tap the same submit control twice, leave and return. Several heuristics, Error Prevention and Help Users Recognize, Diagnose, and Recover from Errors among them, can only be judged by seeing an error happen.
   - Note positive observations separately, with severity 0 in step 7. A pattern that follows a heuristic well is worth recording so a later redesign does not remove it by accident.

5. Write each violation so a reader who was not there can judge it.
   Each log entry needs four parts. An entry missing one of them goes back for a rewalk of that spot rather than into the report as is.
   - Location: flow ID, screen, and control or content involved.
   - Expected: what the heuristic leads a user to expect here, stated plainly. Example: after submitting a payment, the user expects a clear signal that the payment is processing or done.
   - Actual: what the interface does instead, stated as observed behavior, not as a verdict. Example: after tapping Pay, the screen does not change for several seconds and the Pay control stays active, so a second tap is possible.
   - Evidence: screenshot or description from step 3, plus the steps that got the reviewer there if the state is not reachable from the flow start in an obvious way.
   - Keep verdict words out of the actual field. "Confusing" and "bad" are conclusions. "The label reads Export ledger while the menu item that opened this screen reads Download report" is evidence a reader can weigh.
   - One problem per entry. If a screen breaks two heuristics in two different ways, that is two entries. If one broken pattern repeats on eight screens, that is one entry that lists all eight locations, not eight entries.

6. Reject or rewrite vague findings before rating anything.
   Vague findings waste a team's time because nobody can tell whether a fix worked. Test each entry: could a designer change something specific in response to it, and could a reviewer later check whether the problem is gone?
   - Weak: "Navigation is confusing." No location, no behavior, no heuristic, nothing to check.
   - Stronger: "Consistency and Standards, F1 Sign up / step 2 of 3. Progress label reads Step 2 Account, while step 1 was labeled Your details. Expected: one labeling scheme across the steps. Actual: the scheme changes between steps, and the total step count appears on step 1 only."
   - Weak: "Error messages need work." Which error, where, saying what?
   - Stronger: the payment error entry in the Output Contract sample, which names the screen, the code-only message shown, and the missing cause and next step.
   - Common vague patterns to rewrite on sight: a finding that names a whole area instead of a screen, a finding with no expected field, a finding whose title is a fix ("add a confirmation dialog") rather than a problem, and a finding that restates the heuristic ("violates visibility of system status") without saying what happened.
   - A fix suggestion may be added after the problem is fully stated, marked as a suggestion. It never replaces the problem statement, because the team may solve the problem a different way.

7. Rate severity on impact, and record the reasoning.
   Use this scale. This scale is this skill's convention, set out here so a report is self-explanatory. It is not presented as a scale published by the Nielsen Norman Group or by any other source, so print the definitions in the report rather than assuming readers know them.
   - 4: Blocker. The problem will stop many users from completing a core flow, or will cause a wrong result the user cannot detect or undo, such as a duplicate order or lost work. Fix before release.
   - 3: Major. The flow can be completed, but with a real risk of error, delay, or abandonment at this point. Examples: no way out of a form, an error message that gives no path forward. Fix on the current cycle.
   - 2: Minor. Users can complete the flow but hesitate, take a detour, or have to work around the problem, for example remembering a filter they applied because the interface hides it. Fix when the area is next worked on.
   - 1: Cosmetic. Completion and accuracy are unaffected. Examples: clutter that competes with the main action, inconsistent capitalization. Fix opportunistically.
   - 0: Not a problem. A positive observation, or a suspected violation that a closer look did not support. Kept in the log so it is not raised again as new.
   Rate on three questions, and note the answers in the entry when the rating is not obvious:
   - Impact: if a user hits this, what happens to their task, from a moment of hesitation up to lost work or money?
   - Reach: how many users and how many flows meet this problem? A blocker on a rare admin screen and a blocker on checkout do not carry the same weight, and the report should say which is which.
   - Persistence: does the user learn a way around it after one encounter, or does it cost them every time?
   Rate the problem as found, not the effort of fixing it. A one-line label change can be severity 3 if the wrong label sends users to the wrong place. When two reviewers rate the same finding far apart, discuss the three questions above, and keep the higher rating where the evidence shows a user could lose work, money, or a completed task.

8. Group the findings and remove duplicates.
   - Run the tally script in the Operating Rules section on the log. Its counts by heuristic and by severity go into the report summary, and its grouping shows where findings cluster.
   - Merge duplicates: entries from different reviewers, or from the two passes, that describe the same underlying problem in the same place become one finding that keeps both reviewers' evidence and the higher severity, with a note that it was found more than once. Independent discovery by two reviewers is a signal worth keeping in the merged entry.
   - Keep near duplicates separate when the location or the cause differs. The same vague error message in checkout and in settings may be one shared component or two separate texts, and the fix differs. Record which one it appears to be.
   - Group the merged findings for the report in two views: by severity, highest first, for fixing order, and by heuristic, for pattern reading. A cluster under one heuristic, for example five findings under Recognition Rather than Recall, points at a systemic cause, such as hiding state the user needs, and the report should name that pattern rather than leaving the team to fix five screens one at a time.
   - Check coverage before writing: any flow never walked, any heuristic with zero findings across every flow, and any screen visited in pass one only. A zero count is a prompt to rewalk that ground, not proof that the ground is clean.

9. Use more than one reviewer when the stakes justify it, and reconcile openly.
   - A single reviewer finds the problems that reviewer is primed to see, and misses others. Two or three reviewers walking independently, then merging in step 8, is a practical working pattern for a report a team will spend real effort on. This is guidance based on how the method behaves, not a guaranteed count of reviewers for a given result.
   - Reviewers walk and log separately first. Sharing findings mid-walk anchors the second reviewer and cuts the value of having two.
   - Reconcile in one session: merge duplicates, settle severity disagreements using step 7, and keep any finding one reviewer can evidence even if the other did not see it. Disagreement is recorded in the entry, not resolved by dropping the finding.
   - For a low-stakes or early prototype review, one reviewer is acceptable. The report then says so, and lists single-reviewer coverage as a limit.

10. Write the report and stop at the report.
    - Follow the Output Contract below for structure and order.
    - Order findings by severity, highest first, and keep the heuristic view as a summary table rather than a second full listing.
    - Give each finding a fix direction, stated as the outcome a fix should achieve, for example keep the applied filter state visible on the results view. Do not hand over finished screen designs as findings. Design choices belong to the team that owns the interface.
    - Close with limits: flows not walked, devices not covered, parts of the product that were stubbed or unavailable, and the standing limit that an expert review predicts problems rather than proving user impact. Recommend a moderated test from usability-testing-basics when a severity 4 or 3 finding turns on how real users actually behave.

## Output Contract

Deliver an evaluation report with these sections, in this order:

1. Summary: product and version reviewed, heuristic set and its source, reviewers, dates, devices and screen sizes, flows walked, and the headline result in 3 to 5 bullet points.
2. Tally: total findings and the count at each severity level, plus the count by heuristic, produced by findings_tally.py. On the worked fixture in the Operating Rules section the tally is: total 9, severity 4: 2, severity 3: 3, severity 2: 2, severity 1: 1, severity 0: 1. By heuristic, each of the 9 heuristics represented in the fixture has 1 finding.
3. Coverage table: one row per flow, marked walked in both passes, walked once, or not walked, with any screens or states that could not be reached.
4. Findings: one entry per merged finding, sorted by severity from high to low, each with ID, title, heuristic, severity with a one-line reason, flow and location, expected, actual, evidence, and a fix direction.
5. Patterns: clusters by heuristic or by area, each stated in one or two sentences, with the finding IDs that support it.
6. What worked: severity 0 observations and patterns that follow the heuristics well, stated briefly so a redesign keeps them.
7. Method note: heuristic set and source, number of reviewers and whether they worked independently, the severity scale used, and any merges or rating disagreements worth recording.
8. Limits: flows, devices, or states not covered, prototype parts that were stubbed, single-reviewer coverage if that applies, and what this method cannot establish about real user behavior.

A sample finding entry:

- ID: H-01
- Title: Card error shows a code with no cause or next step
- Heuristic: Help Users Recognize, Diagnose, and Recover from Errors
- Severity: 3. The user is stopped at payment and given no path forward short of guessing.
- Flow and location: F2 Checkout / payment submit, card details form
- Expected: an error at payment says in plain language what went wrong and what the user can change to proceed.
- Actual: submitting an expired card shows only the text Error 4021 under the form. No cause, no field is marked, and the entered card details are cleared, so the user has to retype all of them to try again.
- Evidence: screenshot checkout-card-error.png, plus written note of the steps: entered an expired test card, submitted once, observed the code-only message and the cleared form.
- Fix direction: name the problem in plain language next to the field involved, keep the details the user entered, and state the action that will let them continue.

## Operating Rules

1. No participants, no exceptions inside this skill. The moment a review recruits people and asks them to attempt tasks, it has become a usability test and belongs to usability-testing-basics. Reviewers trying the interface themselves, including trying wrong inputs, stays inside this skill.
2. Name the heuristic set and its source in the plan and the report. A finding that cites a heuristic the chosen set does not contain gets recast under a set item that fits, or cut.
3. Every finding carries location and evidence. A finding a second reviewer cannot locate from the entry alone is not ready for the report.
4. Severity uses the 0 to 4 scale in Workflow step 7 only, and the report prints its definitions. Do not import a second scale, and do not present this scale as a published standard.
5. Behavior first, verdict second. Actual fields describe what the interface did. Interpretation goes in the title, the pattern section, or the fix direction, where it is labeled as judgment.
6. Counts in the report come from the tally script output, not from a hand count of the log.
7. Heuristics stay guidance. A heuristic supports a finding, it does not settle an argument on its own. Where a product has a deliberate reason to depart from a heuristic, record the departure and the reason rather than forcing a violation.
8. Keep accessibility findings in their own lane. If a reviewer spots a problem that is really a WCAG question, such as a contrast ratio or a missing text alternative, note it and route it to accessibility-checklist for a proper check rather than rating it here by eye.
9. Use the findings tally script below for the report counts. Save it as findings_tally.py, prepare a violations CSV in the format shown there, then run it.
10. No invented claims about the method. This skill states no fixed share of problems a review will find, no guaranteed reviewer count, and no statistics about the source material. If a report needs a claim like that, find a primary source for it first or leave it out.

### Findings tally (findings_tally.py)

Input format: a CSV file with the header heuristic,location,severity,title and one row per violation. Severity is an integer from 0 to 4 on the scale in Workflow step 7. Quote any field that contains a comma, as in the fixture below.

Example violations.csv (the fixture used for the expected output below):

```csv
heuristic,location,severity,title
Visibility of System Status,Checkout / payment submit,4,No feedback after tapping Pay so users tap again
Error Prevention,Checkout / payment submit,4,Duplicate order created with no confirmation step
User Control and Freedom,Checkout / address form,3,No cancel or back path out of the address form
Consistency and Standards,Settings / profile,3,Save button label and position differ from other forms
"Help Users Recognize, Diagnose, and Recover from Errors",Checkout / payment submit,3,Card error shows a code with no plain-language cause or fix
Recognition Rather than Recall,Search / results,2,Applied filters are hidden so users must remember them
Flexibility and Efficiency of Use,Search / results,2,No way to repeat a recent search
Aesthetic and Minimalist Design,Home / hero section,1,Hero carries three competing calls to action
Help and Documentation,Settings / billing,0,Help link opens task-specific billing guidance in context
```

Run: python3 findings_tally.py violations.csv

Expected output for that file:

```text
total: 9
severity 4: 2
severity 3: 3
severity 2: 2
severity 1: 1
severity 0: 1
findings by heuristic:
Aesthetic and Minimalist Design: 1 (1:1)
Consistency and Standards: 1 (3:1)
Error Prevention: 1 (4:1)
Flexibility and Efficiency of Use: 1 (2:1)
Help Users Recognize, Diagnose, and Recover from Errors: 1 (3:1)
Help and Documentation: 1 (0:1)
Recognition Rather than Recall: 1 (2:1)
User Control and Freedom: 1 (3:1)
Visibility of System Status: 1 (4:1)
findings by severity (highest first):
4 | Error Prevention | Checkout / payment submit | Duplicate order created with no confirmation step
4 | Visibility of System Status | Checkout / payment submit | No feedback after tapping Pay so users tap again
3 | Consistency and Standards | Settings / profile | Save button label and position differ from other forms
3 | Help Users Recognize, Diagnose, and Recover from Errors | Checkout / payment submit | Card error shows a code with no plain-language cause or fix
3 | User Control and Freedom | Checkout / address form | No cancel or back path out of the address form
2 | Flexibility and Efficiency of Use | Search / results | No way to repeat a recent search
2 | Recognition Rather than Recall | Search / results | Applied filters are hidden so users must remember them
1 | Aesthetic and Minimalist Design | Home / hero section | Hero carries three competing calls to action
0 | Help and Documentation | Settings / billing | Help link opens task-specific billing guidance in context
```

findings_tally.py (Python 3, standard library only). Save this code as findings_tally.py:

```python
#!/usr/bin/env python3
"""Group heuristic evaluation findings by heuristic and severity. Stdlib only.

Input: CSV file with header heuristic,location,severity,title and one row
per violation. Severity must be an integer from 0 to 4 (this skill's
convention, see SKILL.md Workflow step 7).
Usage: python3 findings_tally.py violations.csv
"""
import csv, sys
from collections import Counter, defaultdict

def load(path):
    rows = []
    with open(path, newline="", encoding="utf-8") as f:
        r = csv.DictReader(f)
        need = ["heuristic", "location", "severity", "title"]
        if r.fieldnames is None or any(c not in r.fieldnames for c in need):
            raise ValueError("CSV must have header heuristic,location,severity,title")
        for line_no, row in enumerate(r, start=2):
            raw = (row.get("severity") or "").strip()
            try:
                sev = int(raw)
            except ValueError:
                raise ValueError(f"line {line_no}: severity {raw!r} is not an integer")
            if sev < 0 or sev > 4:
                raise ValueError(f"line {line_no}: severity {sev} is outside 0 to 4")
            rows.append({
                "heuristic": (row.get("heuristic") or "").strip(),
                "location": (row.get("location") or "").strip(),
                "severity": sev,
                "title": (row.get("title") or "").strip(),
            })
    if not rows:
        raise ValueError("no findings")
    return rows

def main():
    if len(sys.argv) != 2:
        print("usage: python3 findings_tally.py violations.csv", file=sys.stderr)
        return 2
    try:
        rows = load(sys.argv[1])
    except (OSError, ValueError) as e:
        print(f"error: {e}", file=sys.stderr)
        return 2
    print(f"total: {len(rows)}")
    counts = Counter(r["severity"] for r in rows)
    for sev in sorted(counts, reverse=True):
        print(f"severity {sev}: {counts[sev]}")
    print("findings by heuristic:")
    by_h = defaultdict(list)
    for r in rows:
        by_h[r["heuristic"]].append(r)
    for h in sorted(by_h):
        items = by_h[h]
        sev_counts = Counter(x["severity"] for x in items)
        parts = ", ".join(f"{s}:{sev_counts[s]}" for s in sorted(sev_counts, reverse=True))
        print(f"{h}: {len(items)} ({parts})")
    print("findings by severity (highest first):")
    for r in sorted(rows, key=lambda x: (-x["severity"], x["heuristic"], x["title"])):
        print(f"{r['severity']} | {r['heuristic']} | {r['location']} | {r['title']}")
    return 0

if __name__ == "__main__":
    raise SystemExit(main())
```