Label Defect Detection: Camera-Based Label Inspection on Packaging Lines
Business

Label Defect Detection: Camera-Based Label Inspection on Packaging Lines

Pick a method per label defect, check every label against the production order rather than an operator's choice, and judge the pilot by escape and false reject rates per defect type.

Short answer

Label defect detection is a camera system that checks the label on every pack on a packaging line, and most lines can have one. Missing, shifted or skewed labels are caught by classic measurement; wrong product, language or allergen versions by barcode reading, pattern matching or OCR against the production order; print defects by reference comparison; wrinkles, bubbles and tears by learned models. Round bottles need several cameras and glossy labels need diffuse light. Unreadable packs are rejected, and the pilot is judged by escape and false reject rates per defect type.

What is label defect detection, and can a camera do it?

Label defect detection is a camera system on a filling or packaging line that checks the label on every pack: is it there, is it placed correctly, does it belong to the right product, is it undamaged; most lines that apply labels can have one. A label inspection system is not one algorithm. A missing label is caught by simple measurement, a wrong label by a barcode read or text comparison, and wrinkles by learned models. So first list which defect costs you most, then pick a method per defect.

Note: in Turkish, "etiket" also means the annotations used to train a model; if that is what you need, see our article on few-shot vision and which annotations defect detection needs. This article covers the label on the pack.

What can camera-based label inspection catch, and what can it not?

A camera catches defects on the visible face of the label; it cannot see inside the container, predict whether the adhesive will hold, or judge the text against the regulations:

  • Presence and position: missing, shifted or skewed label, swapped front and back labels, double label.
  • Identity: a label from another product, another language or market version, or an outdated design.
  • Print: missing colour, smears, fading, misregistration.
  • Application damage: wrinkles, bubbles, tears, lifted edges, incomplete shrink on sleeve labels.

The camera does not know which sauce is in the bottle, only whether the label matches the production order. If the approved artwork itself is wrong, the system approves that mistake on every pack; artwork approval is not the camera's job.

Which method catches which label defect?

Use the simplest reliable method per defect: measurement for what can be measured, reading and comparison for identity, reference comparison or a learned model for appearance.

DefectMethodWhat it needsWatch out for
Missing, shifted or skewed labelClassic measurement of label edge position and angle against the containerTolerance in millimetres and degreesTolerance may differ by container diameter
Wrong product, language or artwork versionBarcode or 2D code read against the production order; pattern matching; OCR on a distinguishing lineProduction order, product code, approved artwork imageIf versions share a barcode, the barcode alone is not enough
Print defectsCompare the aligned image with a reference of the approved labelGood sample images per productNeeds alignment first; sensitive to lighting changes
Wrinkles, bubbles, tears, lifted edgesLearned classifier, or anomaly detection trained on good samples onlyGood samples plus defect samples confirmed by quality staffAcceptance limit written down with photo examples

The variation model documentation of MVTec's HALCON library shows how reference comparison works: good images produce an ideal image plus a variation image showing how much grey-value change is allowed at each point, and training images must share the same position and rotation or be aligned by matching first. For labels, the position check runs before the print check.

For wrinkles and bubbles, whose shape you cannot predict, anomaly detection trained on defect-free samples is a good start; the PatchCore paper tackles exactly this cold-start problem of fitting a model with defect-free images only. But the model finds what is different, not what is unacceptable; whether a small crease passes depends on the limit your quality team writes.

Why is a wrong or mis-declared allergen label the costliest defect?

Because a skewed label is cosmetic, while a wrong label gives the consumer wrong information and can end in a recall. A peer-reviewed study by FDA researchers of food allergen recalls in fiscal years 2007 to 2012 reports that allergen labelling problems are the most common cause of recalls of FDA-regulated food, and that using the wrong package or label was the most frequent problem behind them.

In Türkiye, the Turkish Food Codex Regulation on Food Labelling and Consumer Information, published in the Official Gazette on 26 January 2017, lists the allergenic substances in its Annex 1 among the mandatory particulars in Article 9, and Article 24 requires them to be emphasised in the ingredients list by font, style or background colour. It has since been amended; check the current consolidated text.

So design the wrong-label check separately from the presence check, with the expected product taken from the production order, not an operator's pick list. Where language versions share a barcode, or recipes differ only in the allergen line, add a pattern or OCR check on the distinguishing area: allergen box, language line or artwork version code.

How do you choose cameras and lighting for round bottles and glossy labels?

A round container needs several cameras around it, or a station that rotates it in front of one camera, and the lighting follows the label material. Near a cylinder's edge the label is compressed and reflects differently, so one camera reliably sees under half the circumference. Unoriented containers typically need three or four cameras with overlapping views, with the images unwrapped in software; oriented containers, or lines where only the front label matters, need fewer.

  • Glossy and metallic labels: direct light causes glare; use a dome or broad diffuse light, plus a polarising filter if needed.
  • Transparent labels: the edge shows only with light from the side or behind, and the product shows through, so test with different fill colours.
  • Sleeve labels: inspect after the shrink tunnel, not before it.

The camera sits after the labeller and before the reject point; a short exposure or strobe prevents motion blur.

How are triggering, rejection and unreadable items wired into the line?

A sensor triggers the camera, the decision travels with the pack to the reject point, and a pack the system could not decide on always goes to reject. Tracking by conveyor encoder position keeps the reject on the right pack when line speed changes. Write these rules down early:

  • A pack that was unreadable, not fully in frame or not decided in time is rejected and counted separately, never passed as good.
  • A second sensor confirms the reject actually happened; a full reject bin stops the line.
  • A set number of consecutive rejects stops the line, because the cause is usually a wrong label roll.
  • Every reject is stored with its image and reason; the operator screen shows recent rejects, and batch counts go to the MES.

Who teaches the system a new label at changeover, and what do you need to provide?

Each product code has a recipe, and an authorised person teaches a new label from the artwork approved by quality and good samples from the first run; the operator only starts the production order. The recipe holds the expected barcode, the approved artwork image, position tolerances, inspection regions and reject thresholds. After changeover, the first packs are approved by hand. Measure in the pilot whether learned models need retraining for each new artwork.

What you provide: good samples of every product and label type; physical samples or photos of defects that actually happened; approved artwork with the product code and barcode list; a quality lead who writes the acceptance limits; line speed, PLC and reject details; and short access to the line. You do not need thousands of defect images to start.

Where do date code, lot code and barcode checks fit?

They are adjacent checks that can share the camera but answer another question. Article 11 of the same regulation requires a lot mark or number; date and lot codes are usually printed after labelling, read with OCR and compared with the production order. For barcodes, reading and verifying are different jobs. The GS1 General Specifications distinguish a scanner from a verifier: a verifier grades symbol quality with ISO/IEC 15416 for linear barcodes or ISO/IEC 15415 for 2D symbols, but does not check that the human-readable text matches the encoded data. Inline, the camera reads every barcode against the order; print-quality grades are sampled separately with a verifier.

What do you measure in a pilot, and how do you fill in the acceptance sheet?

Judge the pilot with two numbers per defect type: escape rate and false reject rate. The escape rate is how many of 100 truly defective packs pass the system; the rest is the catch rate. The false reject rate is how many of 100 good packs are rejected for nothing, split by which check fired. Prepare defective samples deliberately and feed them in marked.

Defect typeHow test samples are preparedSplit results byAcceptance target
Wrong product or language versionBottles labelled with each wrong version, on every shiftProduct code, language version, shiftStrictest target; every escape investigated
Missing labelUnlabelled bottlesLine speedStrict; an easy test for measurement
Skewed or shifted labelBottles labelled just inside and just outside toleranceContainer diameter, labellerBorderline packs reported separately
Print defectsDefective labels collected from wasteArtwork, colourBased on the visibility limit
Wrinkles, bubbles, tearsNatural defects from the line, classified by quality staffLabel material, lighting conditionBased on the photo acceptance limit

Report results per product, shift and lighting condition; keep the test set out of training. Even the slowest decision must reach the reject point in time at full speed. Set targets from your own error costs and the measured escape rate of today's manual inspection, not from a brochure. Zero escapes has limits too: with none in n samples, the statistical "rule of three" puts the upper bound on the true rate at roughly 3/n with about 95% confidence. We cover writing the criteria in our article on acceptance tests for a vision inspection pilot.

Hypothetical example: two sauces that differ only in the allergen line

The following example is hypothetical; the numbers only illustrate the method. A sauce maker fills two recipes, with and without sesame, into the same round bottle. The labels differ only in the product name and allergen box, and the export versions share a barcode. The pilot uses four cameras, a recipe from the production order, barcode comparison and pattern matching on the allergen box.

In two weeks of shadow mode, 96 of 48,000 good bottles are rejected for nothing: 96 / 48,000 = 0.2% false rejects. Of 300 wrongly labelled bottles fed in marked, 3 pass: 3 / 300 = 1% escapes. All three escapes came on the night shift, which received 100 of the test bottles: 3% at night against 0 / 200 by day. The images show glare on the allergen box from an open loading door; the wrong-label check is not accepted until a curtain is fitted and the test repeated.

What drives the cost of a label inspection system?

Cost depends mainly on:

  • Unoriented round containers, meaning more cameras and image unwrapping.
  • Glossy, metallic or transparent label stock and the lighting it needs.
  • Many product codes and frequent new artwork.
  • High line speed, which tightens exposure and decision time.
  • Wrong-label, print and cosmetic checks all wanted at once.
  • No reject mechanism, encoder or MES connection yet.

A quote should itemise hardware (cameras, lenses, lighting, enclosures, sensors, reject), sample collection and defect classification, software (checks, recipes, operator screen), PLC and MES integration, commissioning and pilot, and monitoring with support for new labels.

Where do label inspection projects most often go wrong?

Most stumble on master data, defaults and change management, not the camera:

  • The expected product comes from an operator's pick list, not the production order.
  • A default setting passes unreadable packs as good.
  • A wrinkle model is trained before anyone writes down the acceptance limit.
  • The pilot covers one product and the day shift only.
  • Nobody owns the recipe when new artwork arrives; the system rejects the new labels and the operator switches the check off.

Hatson Tech builds camera-based checks for label presence, position, identity and cosmetic defects on packaging lines as part of its computer vision quality control service; the first meeting and the keşif, a discovery look at the label defects on your line, are free.

Key takeaways

  • Catch missing and skewed labels by measurement, wrong labels by reading and comparison, and wrinkles or bubbles with learned models; no single method fits every defect.
  • Take the expected product from the production order, not an operator's pick list, and add a separate check on the allergen box or language line when versions share a barcode.
  • Unoriented round bottles usually need three or four cameras with overlapping views, or a rotation station, to see the whole label.
  • Always reject and count separately any pack that was unreadable or not decided in time.
  • Judge the pilot by escape rate and false reject rate per defect type, split by product, shift and lighting condition.

Frequently asked questions

How much does a label inspection system cost?

There is no meaningful single figure. Cost grows with the number of product codes and artwork changes, round unoriented containers that need several cameras, glossy or transparent label stock, line speed, how many checks you want and whether a reject mechanism and MES connection already exist. Ask for a quote that itemises hardware, data work, software, integration and commissioning.

Do we need AI just to check that a label is present?

No. Presence, position and skew are measured reliably with classic image processing, and simple presence can even be done with a label sensor. Learned models earn their place for cosmetic defects such as wrinkles, bubbles and tears, whose shape varies too much to describe with fixed rules.

Can wrong labels be detected just by reading the barcode?

Only if every product, language and artwork version carries its own barcode. When export versions or recipes share one barcode, the barcode read passes the wrong label. Then add pattern matching or OCR on the area that differs, such as the allergen box, the language line or an artwork version code.

Can we use the cameras we already have on the line?

Sometimes. An existing camera can be reused if its resolution, lens, mounting position and lighting suit the label check and it is triggered per pack. Round containers often need more viewing angles than one camera gives, and glossy labels often need different lighting, so test with your own samples first.

Does a label inspection system replace the operator?

No. It takes over checking every pack at line speed, but people still teach new labels, approve the first packs after a changeover, review rejected packs, clear stoppages and keep the acceptance limits up to date. The system makes those decisions visible with images instead of relying on spot checks.

Does label verification work with our ERP or MES?

It should. The most important link is taking the expected product from the production order in the MES or ERP instead of an operator's choice. In the other direction, reject counts per batch and defect type, with images, can be written back for traceability and quality reports.

All articles