1,200 chairs went into one lot. The inspector opened 80 of them, found four with a seat-frame gap, and passed the lot at AQL 2.5. The delivery came up twelve chairs short, all four defectives traced to one production week, and the report line about zero critical defects describing the sample rather than the lot.

One question on the purchase order

A company was fitting out a staff canteen in a single building in another country. The order was 1,200 upholstered dining chairs and 240 solid oak dining tables, five seats to a table, 1,440 units in total, shipping in two forty-foot high cube containers.

The purchase order ran to six pages. It carried model numbers, seat height, frame section, timber species, fabric composition, delivery window and packaging instructions. What it did not carry was a tolerance table or a defect list, and those two absences turned out to be the whole case.

One line of the six pages was about inspection: third-party inspection before shipment, AQL 2.5, general level II. That single line was the entire quality gate on a $178,800 order. And the first question the buyer's procurement lead asked, before anything was booked, was the obvious one: how many units to inspect. On a lot of 1,200 chairs, how many units is that?

The honest answer is 80 chairs and 32 tables, and it was the right answer. It was also, as it turned out, the smallest part of the answer, and the part the buyer was most confident about.

Where the number 80 comes from

ISO 2859-1 does not sample by a percentage of the order. It samples by a code letter, and the code letter is chosen by the lot size and nothing else. You find the range your lot size falls into, take the letter, and the letter carries a fixed sample size for each of the three inspection levels.

For a lot of 1,200 chairs the range is 501 to 1,200, which is code letter J. At general level II, code J takes a sample of 80 units. For the 240 tables the range is 151 to 280, which is code letter G, and code G at level II takes 32 units. That is where 80 and 32 came from. No judgement was exercised and no judgement could be: the table produces the number.

Read the share column in that table and the uncomfortable part shows up immediately. A bigger order gets a smaller share of itself opened. Eighty out of 1,200 is 6.7 per cent. Fifty out of 500 is 10 per cent. Twenty out of 150 is 13 per cent. The sample size climbs in steps but the lot size climbs continuously, so the fraction of your order that any inspector will ever see falls as the order grows. On a 3,200-chair order it is 3.9 per cent.

None of this is a criticism of the standard. It is doing exactly what it was designed to do, which is to make a statistical decision about a population without opening all of it. The design rests on an assumption, and the assumption is that the lot is homogeneous: that the units are alike enough that the ones you did not open are reasonably represented by the ones you did. That assumption is where the trouble starts.

The same 1,200 chairs, three different answers

The 1,200 chairs were not made in one run. They were made over three weeks in three batches of 400, with three deliveries of frame components and two batches of upholstery foam, and the factory was entirely open about it. The purchase order said 1,200 chairs. It did not say what a lot was.

There is a detail in the code table worth pausing on. One extra chair changes the answer. At 1,200 chairs the lot is code J and 80 chairs get opened. At 1,201 it is code K and 125 get opened, a 56 per cent increase in inspection from a single additional unit. Lot sizes are almost never chosen with a code letter in mind, and the code letter is doing more work here than the inspector is.

That single omission in the purchase order produces three different answers to the buyer's question, and all three are defensible under the standard.

How to read the acceptance number

Acceptance and rejection numbers come from the same code letter row. They are not percentages and they are not counts of the whole lot. They are counts inside the sample, and the rule is simple: accept the lot if the number of defectives in the sample is at or below the acceptance number, reject it if the number reaches the rejection number.

For the chairs, code J, the acceptance number at AQL 2.5 is 5 and the rejection number is 6. The inspector found 4 major defects in the 80 chairs. Four is at or below five, so the lot was accepted. The two minor defects were counted against AQL 4.0, where the acceptance number is 7, so they changed nothing either. The tables, code G at AQL 2.5, carry an acceptance number of 2; one major defect was found in 32 tables, and one is at or below two, so the tables were accepted as well.

There is a step in there that is easy to miss and worth slowing down on. Four defectives in a sample of 80 is a 5 per cent observed defect rate, and the acceptance number of 5 out of 80 is 6.25 per cent of the sample. The plan is set up to accept a lot whose true nonconforming rate sits at or below 2.5 per cent, which on 1,200 chairs is 30 chairs, and at that point the sampling table still only produces an acceptance number of 5 out of 80. With 80 units you cannot tell 2.5 per cent from 5 per cent. The plan accepts a lot showing 5 per cent, because at this sample size the two are the same number as far as the arithmetic can tell.

Which means the buyer's mental model was wrong in a specific and expensive way. Passing AQL 2.5 on this order did not mean the goods were sound. It meant the lot was sound enough, at the resolution 80 chairs allows, where sound enough is calibrated to tolerate up to 30 nonconforming chairs out of 1,200. On a $178,800 order that is a large number of chairs to have accepted without knowing it.

What the report actually found

The findings section ran to two and a half pages and it is worth reading as a document, because it shows what a passing AQL verdict looks like on paper.

Four majors, two minors and one table

Both lots passed. The report was issued, the balance was released, and two forty-foot containers were booked. Every number in that table was correct, and every one of them was measured against a classification the inspector had supplied rather than one the buyer had written.

The majors were a visible gap at the seat-to-frame joint, two to three millimetres, on four of the eighty chairs opened. The minors were smeared carton labels on two cartons. The table defect was an open joint along the apron of one table top. Nothing else was recorded as a defect, because nothing else had been written down as one.

The count the sampling plan never made

The carton count matched the packing list exactly. Forty cartons of chairs, thirty to a carton, 1,200 chairs. When the inspector opened the cartons to count the units inside them, the fortieth carton held 18 chairs, not 30.

1,188 chairs against 1,200 ordered. Twelve short. The error was invisible to the sampling plan, and it would have been invisible to any sampling plan, because a sampling plan samples units and never counts them. You can open 80 chairs and learn something about the quality of the chairs you opened, and learn nothing whatsoever about how many chairs exist.

The shortfall was worth $1,104 at the order price. The inspection that found it was two man-days at the Order and Quality Check rate of $199, so $398. The count alone returned nearly three times the cost of the visit that caught it. Which is the argument for the boring check: a quantity problem is a counting problem, and it is solved by counting.

One batch, and the rate hiding inside it

The four joint gaps in the sample were not spread across the order. All four came from the third production batch, the last week of production. Of the 80 chairs opened, 27 came from that batch, and 4 of those 27 had the defect.

Four out of 27 is 14.8 per cent. The chairs from the first two batches, 53 of the 80 opened, had none. Apply 14.8 per cent to the 400 chairs in that batch and you get about 60 chairs with a seat-frame gap, and the whole-lot sample rate of 5 per cent was simply those 60 chairs spread across 1,200 and diluted. The lot passed at AQL 2.5 because the plan measured the average of a lot in which most of the defect sat in one week of production.

That is not a failure of sampling either. A plan drawing 80 chairs from 1,200 is answering the question is this lot consistent, and on that question the answer was yes. What the buyer needed to know was whether the lot was uniform, and 80 units spread across three production weeks cannot answer that. It is 4 defectives in one week out of 27 sampled, which is the difference between a lot that passes and a lot that should have been held back.

Sixty chairs with a structural gap at the seat joint, at $92 each, is $5,520. The inspection cost $398.

Zero critical defects found

The report recorded, under the critical defect heading, no critical defects found. That line is true, and it is the most misleading line in the report, because it describes the sample rather than the lot.

This order had a zero-acceptance characteristic, which is the sort of thing that is easy to miss and expensive to miss. The welded base frame carries the whole load of the chair and was classed critical, and under AQL critical is accepted at zero, so a single critical defect found in the sample rejects the whole lot. That is the rule, and it is worth understanding exactly what it does and does not do. It does not say the lot contains no critical defects. It says that if the sample contains one, the lot is rejected. Whether the sample contains one depends on the sample size and on how many there are, and on a lot of 1,200 with a sample of 80, the arithmetic is not encouraging.

One cracked weld and a clean report

If the lot contained exactly one chair with a cracked base weld, the 80-chair sample had a 6.7 per cent chance of opening it. So on a clean result the inspector could report zero critical defects and have a 93 per cent chance of reporting that on a lot that did contain one. If there were ten of them, it was a coin flip.

There is no version of this where AQL 2.5 at general level II gives the buyer the guarantee they were reading it as. A zero-acceptance characteristic on a 1,200-unit lot needs either a near-full inspection of that characteristic or an explicit acceptance that a single-unit critical defect is unlikely to be seen. What the buyer had instead was a report line that read like an assurance and was not one.

Eight lines to put in the purchase order

The lot passed AQL 2.5 and it should not have been a close call, because none of the four numbers above is a question the standard can answer. All of them are questions the buyer has to answer first, and the standard then measures against. None of this is expensive. It is a page of the purchase order that was not written.

• Define the lot. Say whether 1,200 chairs are one lot for sampling purposes or one lot per production week. It is the difference between 80 chairs opened and 150, and between an acceptance number of 5 and 3.
• Write your own defect classification. Critical, major and minor, each with named examples from your own product, and state which characteristics are zero-acceptance. Without it the inspector supplies a reasonable default, and the acceptance numbers count something you did not choose.
• Name the standard, the level and the AQL in full: ISO 2859-1, single sampling, normal inspection, general level II, AQL 2.5 for major and 4.0 for minor. One line, no ambiguity about what will be applied.
• Write the action and not only the number. What happens at the acceptance number and what happens at the rejection number: accept with the defectives recorded, or reject and re-inspect after rework, and who pays for the second visit.
• Put the count outside the sampling plan. A 100 per cent opened-carton unit count against the packing list, plus a 100 per cent carton count. Never sample a quantity, because the arithmetic of a sample says nothing about the count.
• For any zero-acceptance characteristic, require 100 per cent inspection of that characteristic and say who signs it off. Sampling a critical attribute and reporting zero found is not the same as the attribute being absent.
• Put a tolerance on every dimension that carries one, and say which dimensions have none. A measurement without a limit cannot be graded as a major defect, however many units you open.
• Say what the report has to contain: the lot definition used, the code letter and sample size, the acceptance and rejection numbers applied, the count result, and the list of what was not verified.

What the sample size could not decide

The buyer's question was how many units to inspect, and the answer was 80 chairs and 32 tables. It was correct, it was free, and it was the only part of the inspection the buyer did not have to think about.

The 80 told them how many chairs to open. It did not tell them what to open them against, because the purchase order carried no defect classification and the inspector supplied one. It did not tell them how many chairs were in the boxes, because a sampling plan does not count. It did not tell them whether the lot was uniform across three production weeks, because 80 units spread over three weeks cannot answer that. And it did not tell them what a critical defect was worth on their own product, because AQL only counts criticals if it happens to open them.

That is the part that is easy to leave out. The sample size is a number you get for nothing, and it is worth having. The classification, the lot definition, the count, and the treatment of the critical characteristic are four decisions only the buyer can make, and they decide whether the 80 units that were opened meant anything at all. Our report on this order ran to eight sections. The first seven said what was checked. The eighth said what was not, and it is the section that would have told the buyer that a lot passing at AQL 2.5 was never the thing they were buying.