Lore

Axial Coding

Axial coding is the step that follows Open Coding in Hamel Husain's error-analysis methodology: once open codes (short, free-text descriptions of what went wrong in each trace) have been collected across a sample of traces, group and classify them into a smaller set of named categories.

Apply: feed the full list of collected open codes to an LLM and ask it to propose five or six axial-code categories (e.g. tour-scheduling issues, human-handoff issues, formatting errors), then label each trace's open code with the category it belongs to.

The resulting categories tell a team where to invest — a category with many traces and a clear right/wrong answer is a candidate for a Code-Based (Reference-Based) Evaluation; a category that's subjective or hard to rule-check is a candidate for an LLM-as-judge. See Eval Investment Triage for that decision. Axial coding also produces the human ground-truth labels later reused, for free, to validate an LLM judge.

LLM-Assisted Clustering and Taxonomy Completeness

Practically, axial coding is done by exporting the raw Open Coding notes to a spreadsheet/CSV and prompting an LLM (a Claude project or ChatGPT) to synthesize the notes into named, actionable failure-mode categories — then editing those categories by hand rather than accepting them as-is. When using an LLM to auto-label further notes against the resulting categories, adding a catch-all "none of the above" option surfaces gaps: if notes land there often, the taxonomy is incomplete and the team should return to open coding to add or reword categories rather than forcing bad fits. Once categories are finalized, frequency across them is tallied with basic-counting-frequency-analysis.

'None of the Above' as a Taxonomy Diagnostic

A deliberate 'none of the above' bucket, left open when an LLM auto-labels Open Coding notes into failure-mode categories, doubles as a diagnostic rather than a dumping ground: if the model reaches for it often, the taxonomy is incomplete, not the data. High 'none of the above' volume is the signal to go back and add categories, not to force-fit notes into existing ones.