A hotel group was refurbishing eight properties and ordered 320 upholstered guest-room armchairs from a factory in Foshan. The pre-shipment report opened 50 of them, recorded four units with play at the arm bracket, and recommended release of the lot. The buyer's quality manager could not reproduce that verdict from the report, because the number the four findings were compared against was never printed on it.

A verdict is a comparison of two numbers

A hotel group was refurbishing eight properties and ordered 320 upholstered guest-room armchairs from a factory in Foshan. The unit price was $96 and the order was $30,720. Thirty per cent, $9,216, was paid on order. Seventy per cent, $21,504, was due against a pre-shipment inspection report, and the report recommended release.

The inspection was run by a third-party agency the buyer had used before. Fifty of the 320 chairs were opened. The report recorded one major defect and four minor defects, no critical defects, and the single word PASS.

The buyer's quality manager read that report the way she reads every report before she releases a balance, which is backwards from the top: she took the findings and the sampling line and tried to reconstruct the verdict. She could not do it. The report told her that four findings had been recorded as minor, and it did not tell her what number those four were judged against. Without that number, PASS was not a conclusion she could check. It was a conclusion she had to take on trust, and she had a balance of $21,504 to release against it.

What an acceptance decision actually needs

An AQL acceptance decision is a comparison between two numbers. The first is how many nonconforming units the inspector found in the sample. The second is the accept number, the largest count that still lets the lot pass. Find the accept number and the decision is arithmetic. Leave it out and the decision is a claim.

The accept number is not a property of the AQL value on its own. AQL 2.5 is a column in a table; the accept number lives in a row. The row is chosen by the lot size, through the sample size code letter, and the column is chosen by the defect class the finding was graded into. Change the row or change the column and the same count lands on the other side of the line.

That means four things have to be printed for a verdict to be reproducible: the lot size, the inspection level, the AQL used for each defect class, and the accept and reject numbers for each of those classes. Any one of them missing and the arithmetic cannot be redone by the reader. The reader can read the verdict, but they cannot audit it, cannot challenge it, and cannot tell a near miss from a comfortable pass.

Four findings, and the number that decides them

This is the table the inspector was working from, at general level II under normal inspection, exactly as the AQL calculator on this site returns it. The four findings in this case were four arm brackets with play in them.

Read the four across each row and the verdict changes with the row. At a lot of 320 the same four are a rejection if they are graded major and an acceptance if they are graded minor, and that acceptance is by a single defect. At a lot of 240 the same four would have rejected the lot in either column. At a lot of 800 they would have passed in either column.

So four findings is not a result. It is the left-hand side of a comparison whose right-hand side the report did not print, and the lot size and the grade of the defect are what set the right-hand side. One of those is worth noticing on its own. The accept number is a count inside the sample, and it grows with the lot, so a fixed count of findings becomes easier to pass as the order gets bigger. A bigger order does not make the same four chairs harder to reject. What decides the verdict is the count in the sample and the row it was read from.

The worksheet said what the summary sheet did not

She asked the inspection agency for the worksheet behind the summary sheet, and got it. The worksheet recorded lot 320, general level II, sample 50, AQL 2.5 major and 4.0 minor, one major and four minor found, and an acceptance.

The four minors were the arm brackets. The worksheet showed them graded against the factory's own defect schedule, which put play or gap under 3 mm with no visible movement into the minor column. The buyer's schedule put anything over 1.0 mm of lateral movement under a 20 N load into the major column. The inspector had been handed the factory's schedule and had applied it correctly. Three millimetres and one millimetre are the same defect and they are not the same rule.

Under the buyer's own schedule those four brackets join the one existing major and the lot is five majors, which is five against a reject number of four and therefore a rejection. Under the factory's schedule it is one major and four minor and the lot passes. Same chairs, same sample, same AQL column, opposite verdicts, and the only document that could have shown which one was applied was a summary sheet with no number on it.

Sixteen chairs, measured one at a time

The factory measured arm play on all 320 chairs with a dial gauge under a 20 N lateral load at the top of the arm, as part of the corrective action it was already carrying out on those units. A Level 3 verification, two man-days at $299 a day, went back to the factory to check the first report item by item, verify the factory's own measurements against a sample of its own work, and re-sample the reworked lot once the rework was finished.

Sixteen of the 320 had more than 1.0 mm of movement. That is 5.00 per cent of the lot, twice the AQL 2.5 that had been applied to it, and all four of the units in the original sample were among the sixteen.

The sample had been telling them. Four findings in fifty is 8.0 per cent of the sample, twice the AQL 4.0 those four were judged against and more than three times the AQL 2.5 the buyer's own schedule would have judged them against. The grading is what turned that signal into a pass, and the accept number that was not printed is what stopped anyone reading it as a signal at all.

Six things the report should have carried

A verdict is reproducible when the reader can rebuild it. That takes six numbers and one definition, and every one of them is a field the inspection body already had somewhere in the report it wrote.

None of them is difficult to produce. All of them are difficult to produce after the fact.

• The lot size that was inspected, in units, and the number of units the sample was drawn from. These two fix the row of the table and the row fixes the accept and reject numbers. Printed together they are one line: lot 320 units, code letter H, sample 50.
• The inspection level, written out as general level I, II or III. The level changes the sample size for the same lot, and a sample size with no level behind it cannot be checked against anything.
• The AQL applied to each defect class, not one AQL for the whole report. Critical, major and minor are judged against different columns, and a finding moves between columns when its grade moves.
• The accept number and the reject number for each class actually applied. These two are the verdict. The defect count is only the left-hand side of the comparison.
• The count found in the sample for each class, so the count and the accept number can be set side by side by the reader without hunting through two sections of the document.
• The defect classification schedule used, with the measurement method for each defect: what was measured, with what, under what load, and the numeric limit that separates one class from the next. This is the line that stops a factory's schedule and a buyer's schedule being applied to the same chairs.

The lines that belong in the purchase order

None of those six fields is something the buyer has to ask for after the inspection. Every one of them can be fixed before the goods are made, and the classification schedule is the one that has to travel with the inspection instruction rather than sit in a quality annex. There is a seventh point, and it decides whether the other six matter at all: a sampling plan is a probability, and for some defects the probability is not good enough. Say so on the order, while it costs nothing to say.

• State the lot. Write the inspection lot as one consignment of the order, in units. Where production is split, say whether each batch is a separate lot or part of one, because the accept number changes with the lot size and a 240-unit lot rejects the same four findings that a 320-unit lot accepts.
• State the level and the AQL for each defect class in the order itself: ISO 2859-1, single sampling, normal inspection, general level II, AQL 2.5 major, 4.0 minor, zero for critical.
• Attach the defect classification schedule as a numbered document and send it to the inspector as part of the inspection instruction. It is the only document that says what counts as major, and in this case it was the one document that never reached the man holding the gauge.
• Give every defect the buyer cannot absorb a number and a method: arm fixing play, measured as lateral displacement at the top of the arm under a 20 N load, more than 1.0 mm is a major defect.
• Say what the report has to print, and make the balance conditional on it. A report that does not carry the six fields above is not a report against this order.
• For a defect that cannot be absorbed at any rate, do not rely on AQL at all. Put a zero acceptance number on that characteristic, or require 100 per cent measurement of it, and say which.

The limit of the arithmetic

Even with a perfect report and a correctly graded finding, this plan would have passed this lot most of the time. At the true rate of 5.00 per cent, a level II plan on 320 units with accept 3 and reject 4 rejects a lot 22.85 per cent of the time. It passes 77.15 per cent of lots that are that bad. A plan that stays on normal inspection indefinitely keeps paying that price, which is why the standard's own switching rules exist: after two rejected lots out of five consecutive lots, the next lot is judged at tightened inspection, where the accept numbers are lower. The report has to say which of those it was working to.

A zero acceptance number on that one defect, at the same sample size of 50, rejects 93.86 per cent of lots at the same rate. One hundred per cent measurement of the arm fixing rejects all of them, and costs 320 chairs at about $4 each, $1,280, or 4.17 per cent of this order.

That is the honest limit of a sampling plan, and it is why the missing accept number is not the whole story of this case. The number that was not printed is what hid a wrong verdict. The threshold written into the purchase order is what decides whether a wrong verdict is likely to matter at all.

What the money says

The verification that finally produced a reproducible verdict was two man-days of Full Verification at $299 a day, $598 against a $30,720 order, or 1.95 per cent. The third-party pre-shipment inspection the buyer had paid for separately is not in that figure.

What the $598 bought was the sixteen chairs, $1,536 of goods, 5.00 per cent of the order, that the factory had to re-do at its own cost before loading, and a $21,504 balance held for eleven days. It also bought the thing that actually mattered, which is a document the buyer could repeat. The next shipment on this factory went out against the same six fields, and nobody had to reconstruct anything.

Two chairs per guest room across eight properties, sixteen loose arm brackets is one defective chair in every ten rooms in a refurbishment that was due to open. That is the number the report's arithmetic was about, and it is not a number the arithmetic on the page contained.

What the case is actually about

A report that says PASS is not a verdict until somebody can rebuild the arithmetic behind it. The number to rebuild it is small: the lot size, the level, the AQL for each class, and the accept and reject numbers. Print them and the report can be checked, challenged and repeated on the next shipment. Leave them out and the only way to find out what was decided is to go and ask.

And when the buyer goes and asks, the answer is usually that the inspector was working from the factory's defect schedule, because the buyer's never reached him. That is a document that was written and never sent, and it is the cheapest thing in this whole case to fix.

The AQL table on this site prints the lot size, the code letter, the sample size and the accept and reject numbers for any order quantity. It is the same set of fields a report has to carry, in the order a reader wants to check them. If a report does not carry them, there is nothing in it to disagree with.