ZJU-Leaper: A Benchmark Dataset for Fabric Defect Detection

Datasets Comparison

Fabric Patterns

Dataset Statistics

Fabric Groups

• Group 1: White Plain, Thick Stripe, Thin Stripe;

• Group 2: Dot Pattern, Houndstooth, Gingham, Knot Pattern;

• Group 3: Twill Plaid, Blue Plaid, Brown Plaid, Gray Plaid, Red Plaid;

• Group 4: Floral Print 1, Floral Print 2, Floral Print 3;

• Group 5: Pattern 1, Pattern 2, Pattern 3, Pattern 4;

Task Settings

• Setting 1: Normal samples only (annotation-free) — when images of defect-free fabrics can be obtained cheaply, and those of defects are difficult to acquire.

• Setting 2: Small amount of defect data (with mask annotation) — when a small amount of defect data and annotation information can be obtained at a low cost, along with many normal images.

• Setting 3: A large amount of defect data (with label annotation) — when methods for the first two settings fail to train the inspection model effectively, and labeling information of defects can be quickly obtained.

• Setting 4: A large amount of defect data (with bounding-box annotation) — more labor is involved in marking defects with bounding boxes that contain rough position and size information of defects.

• Setting 5: A large amount of defect data (with mask annotation) — used as the last resort where the effort to mask all defects in a large number of samples becomes necessary.

Evaluation Metrics

1. Pixel-level

   $$\boldsymbol{Dice = F_1^{pix} = \frac{TP^{pix}}{TP^{pix} + \frac{1}{2}(FP^{pix} + FN^{pix})}}$$

2. Region-level

   $$Pre^{reg} = \frac{\#correct\ proposed\ regions}{\#total\ proposed\ regions} \\ Rec^{reg} = \frac{\#detected\ ground \text{\textendash} truth\ regions}{\#total\ ground \text{\textendash} truth\ regions} \\ \boldsymbol{F_{1}^{reg} = 2 \cdot \frac{Pre^{reg} \cdot Rec^{reg}}{Pre^{reg} + Rec^{reg}}}$$

3. Sample-level

   $$\boldsymbol{F_1^{sam} = \frac{TP^{sam}}{TP^{sam} + \frac{1}{2}(FP^{sam} + FN^{sam})}}$$

Other Resources

• Dataset download link
  • http://bit.ly/zjuleaper

• Dataset API
  • https://github.com/nico-zck/ZJU-Leaper-Dataset/tree/master/dataset_api

• Codes of baseline experiments
  • https://github.com/nico-zck/ZJU-Leaper-Dataset/tree/master/baselines

• Implementation of existing methods
  • https://github.com/nico-zck/ZJU-Leaper-Dataset/tree/master/related_work

Related Paper

The paper titled "ZJU-Leaper: A Benchmark Dataset for Fabric Defect Detection and a Comparative Study" was accepted by IEEE Transactions on Artificial Intelligence.

Citation Tip

@article{zju-leaper,
  author={Zhang, Chenkai and Feng, Shaozhe and Wang, Xulongqi and Wang, Yueming},
  journal={IEEE Transactions on Artificial Intelligence}, 
  title={ZJU-Leaper: A Benchmark Dataset for Fabric Defect Detection and a Comparative Study}, 
  year={2020},
  volume={1},
  number={3},
  pages={219-232},
  doi={10.1109/TAI.2021.3057027}
  issn={2691-4581},
  publisher={IEEE},
}

Performance Benchmark