The good parts your vision model throws away
A defect detector needs an operating threshold and two error counts. A worked example shows why a high detection rate can still send many acceptable parts for review.

An inspection model can look excellent while sending too many good parts into the reject bin. The defect rate changes what a headline accuracy number means.
A factory buying vision inspection needs an answer at the level of a part: what should happen to this item as it passes the camera? A score measured on a collection of photographs answers a narrower question. Connecting the two requires a defined defect, a decision threshold, and a way to count the mistakes after installation.
Start with the decision a defect should trigger
Consider an illustrative line producing 10,000 parts. Suppose 100 parts have the defect being inspected. A system that catches 90 of them has a 90% detection rate for that defect. If it also rejects 99 of the 9,900 acceptable parts, the reject stream contains 189 parts, of which 99 were acceptable.
Those figures are an example, not a reported factory result. They show why the team needs two counts. Missed defects describe what escapes. False rejects describe the acceptable production sent for unnecessary review or disposal. A purchasing decision based only on the first count can leave the second cost undiscovered.
Moving a threshold usually changes both counts. A more sensitive setting may catch a faint defect while also reacting to harmless surface variation. The quality team needs to decide which trade-off is acceptable for the particular defect. A cosmetic mark and a missing safety component need separate acceptance criteria.
A benchmark makes a narrower promise
The MVTec AD 2 dataset provides industrial inspection scenarios with defect-free training images. Its test material includes lighting conditions that need not appear in the training set. That makes a change in the appearance of acceptable production part of the evaluation.
The 2026 paper describing the dataset also distinguishes identifying a defective image from locating the affected pixels. A method can perform differently on those tasks. A score for tracing a defect's outline cannot be read directly as the percentage of defective parts that a production line will catch.
More recent evaluation continues that concern. The October 2026 VAND 4.0 report describes remaining difficulties with industrial anomaly segmentation and shifting conditions. Its benchmark findings support testing the limits of the methods; they do not supply a universal detection rate for every factory application.
This is why I would remove a generic claim that vision AI achieves 95-99% detection from a purchase brief. It needs the defect definition, the sample population, and the operating threshold before it can help choose equipment. A result on one material under one camera arrangement cannot settle another plant's acceptance decision.
A working installation has a defined action
BMW's description of automated surface processing in Regensburg gives a concrete example. Inspection identifies potential paint-surface problems, processing equipment treats the relevant locations, and marking helps with final inspection. BMW reports that automated surface processing has been in series production since March 2022.
The useful feature of that account is the connection between observation and action. There is a particular surface, a location on that surface, and a processing step that can respond. It does not establish a payback period for an unrelated inspection system. Nor does it turn paint inspection into proof that the same camera can find contamination inside packaged food.
For a new project, the equivalent description should fit in ordinary language. A visible chip causes the part to move to a review station. A missing component stops the assembly from advancing. An uncertain image is retained for an inspector. Each statement gives the software a defined consequence that can be tested.
Count errors in the production population
An evaluation set assembled to contain many defects is useful for examining failure cases. Its proportion of defective parts can differ sharply from normal output. The illustrative calculation above shows how that difference changes the composition of the reject stream.
I would therefore keep separate reports for deliberately selected defect examples and a representative production run. The first helps establish whether the model can recognise the problem. The second shows how frequently its decisions create work for the plant. Both reports need the same documented version of the model and threshold.
Reviewing rejected parts creates another source of information. The reviewer should be able to record whether the model found the intended defect, reacted to an acceptable variation, or received an image that could not support a decision. Combining those outcomes into one vague category hides the work needed to improve the station.
The acceptance test should also have an owner. If the quality engineer defines the defect but a software team changes the threshold later, the original test can stop describing the running system. A controlled change needs a new evaluation on the relevant production conditions.
A camera can support a quality gate when its errors are measured in terms the line can act on. The buying question is how many unacceptable parts escape at the chosen setting, together with what it costs to handle the acceptable parts that the system rejects.