Flexible and robust detection for assembly automation with YOLOv5: a case study on HMLV manufacturing line

Artificial intelligence (AI); Assembly automation; High-mix low-volume (HMLV); You only look once (YOLO); Artificial intelligence; Case-studies; High-mix low-volume; High-mix/low volumes; Manufacturing lines; Robust detection; Small and medium-sized enterprise; Volume manufacturing; You only look once; Software; Industrial and Manufacturing Engineering; Artificial Intelligence

Abstract :

[en] Automating assembly processes in High-Mix, Low Volume (HMLV) manufacturing remains challenging, especially for Small and Medium-sized Enterprises (SMEs). Consequently, many companies still rely on a significant amount of manual operations with an overall low degree of automation. The emergence of artificial intelligence-based algorithms offers potential solutions, enabling assembly automation compatible with multiple products and maintaining overall production flexibility. This paper investigates the application of the YOLO (You Only Look Once) object detection algorithm in an HMLV production line within an SME. The performance of the algorithm was tested for different cases, namely, (a) on different products having similar product features, (b) on completely new products, and (c) under different lighting conditions. The algorithm achieved precision and recall greater than 98% and mAP50:95 greater than 97%.

Disciplines :

Mechanical engineering

Author, co-author :

SIMETH, Alexej ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Engineering (DoE)

KUMAR, Atal Anil ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Engineering (DoE)

PLAPPER, Peter ; University of Luxembourg

External co-authors :

Language :

English

Title :

Flexible and robust detection for assembly automation with YOLOv5: a case study on HMLV manufacturing line

Publication date :

May 2024

Journal title :

Journal of Intelligent Manufacturing

ISSN :

0956-5515

eISSN :

1572-8145

Publisher :

Springer

Peer reviewed :

Peer Reviewed verified by ORBi

Additional URL :

https://link.springer.com/content/pdf/10.1007/s10845-024-02411-5.pdf

Available on ORBilu :

since 01 August 2024

Statistics

Number of views

136 (5 by Unilu)

Number of downloads

43 (0 by Unilu)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

OpenAlex citations

WoS citations^™

Bibliography

A. Abu-Samah M.K. Shahzad E. Zamai Bayesian based methodology for the extraction and validation of time bound failure signatures for online failure prediction Reliability Engineering & System Safety 2017 167 616 62 10.1016/j.ress.2017.04.016
Adobe. Lizenzfreie Stockfotos und Bilder. Retrieved from https://stock.adobe.com/de/photos
A. Alduaij N.M. Hassan Adopting a circular open-field layout in designing flexible manufacturing systems International Journal of Computer Integrated Manufacturing 2020 33 6 572 589 10.1080/0951192X.2020.1775300
Bochkovskiy, A., Wang, C. Y., & Liao, H. Y. M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934.
Y.W. Chen J.M. Shiu An implementation of YOLO-family algorithms in classifying the product quality for the acrylonitrile butadiene styrene metallization The International Journal of Advanced Manufacturing Technology 2022 119 11–12 8257 826 10.1007/s00170-022-08676-5
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., et al. (2016). The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3213–3223).
Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., et al. (2017). Deformable convolutional networks. In Proceedings of the IEEE international conference on computer vision (pp. 764–773).
Dalal, N., Triggs, B. (2005). Histograms of oriented gradients for human detection. In 2005 IEEE computer society conference on computer vision and pattern recognition (Vol. 1, pp. 886–893). IEEE.
T. Diwan G. Anirudh J.V. Tembhurne Object detection using YOLO: Challenges, architectural successors, datasets and applications Multimedia Tools and Applications 2023 82 6 9243 927 10.1007/s11042-022-13644-y
A. Downs Z. Kootbally W. Harrison P. Pilliptchak B. Antonishek M. Aksu et al. Assessing industrial robot agility through international competitions Robotics and Computer-Integrated Manufacturing 2021 70 10211 10.1016/j.rcim.2020.102113
M. Elgendy Deep learning for vision systems 2020 Simon and Schuster
M. Everingham S.A. Eslami L. Van Gool C.K. Williams J. Winn A. Zisserman The pascal visual object classes challenge: A retrospective International Journal of Computer Vision 2015 111 98 136 10.1007/s11263-014-0733-5
M. Everingham L. Van Gool C.K. Williams J. Winn A. Zisserman The pascal visual object classes (voc) challenge International Journal of Computer Vision 2010 88 303 33 10.1007/s11263-009-0275-4
Felzenszwalb, P., McAllester, D., & Ramanan, D. (2008). A discriminatively trained, multiscale, deformable part model. In 2008 IEEE conference on computer vision and pattern recognition (pp. 1–8). IEEE.
Felzenszwalb, P. F., Girshick, R. B., & McAllester, D. (2010). Cascade object detection with deformable part models. In 2010 IEEE computer society conference on computer vision and pattern recognition (pp. 2241–2248). IEEE.
R. Fernandes J.B. Gouveia C. Pinho Product mix strategy and manufacturing flexibility Journal of Manufacturing Systems 2012 31 3 301 31 10.1016/j.jmsy.2012.02.001
M.L. Francies M.M. Ata M.A. Mohamed A robust multiclass 3D object recognition based on modern YOLO deep learning algorithms Concurrency and Computation: Practice and Experience 2022 34 1 e651 10.1002/cpe.6517
Girshick, R. (2015). Fast r-cnn. In Proceedings of the IEEE international conference on computer vision (pp. 1440–1448).
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 580–587).
R. Girshick J. Donahue T. Darrell J. Malik Region-based convolutional networks for accurate object detection and segmentation IEEE Transactions on Pattern Analysis and Machine Intelligence 2015 38 1 142 158 10.1109/TPAMI.2015.2437384
He, K., Gkioxari, G., Dollár, P., & Girshick, R. (2017). Mask r-cnn. In Proceedings of the IEEE international conference on computer vision (pp. 2961–2969).
P. Holtewert T. Bauernhansl Interchangeable product designs for the increase of capacity flexibility in production systems Procedia CIRP 2016 50 252 257 10.1016/j.procir.2016.04.129
L. Jiao F. Zhang F. Liu S. Yang L. Li Z. Feng et al. A survey of deep learning-based object detection IEEE Access 2019 7 128837 12886 10.1109/ACCESS.2019.2939201
L.T. Jiao P.W. Guo B. Hong P. Feng Vehicle wheel weld detection based on improved YOLO v4 algorithm Computer Optics 2022 46 2 271 279
Jocher, G., Chaurasia, A., Stoken, A., Borovec, J., Kwon, Y., Michael, K., et al. (2022) ultralytics/yolov5: v7.0—YOLOv5 SOTA realtime instance segmentation. Retrieved from https://zenodo.org/record/7347926
K. Johansen S. Rao M. Ashourpour The role of automation in complexities of high-mix in low-volume production-a literature review Procedia CIRP 2021 104 1452 1457 10.1016/j.procir.2021.11.245
Karaulova, T., Andronnikov, K., Mahmood, K., & Shevtshenko, E. (2019). Lean automation for low-volume manufacturing environment. In B. Katalinic (Ed.), Proceedings of the 30th DAAAM international symposium (pp. 0059–0068). DAAAM International.
J. Kaur W. Singh Tools, techniques, datasets and application areas for object detection in an image: A review Multimedia Tools and Applications 2022 81 27 38297 3835 10.1007/s11042-022-13153-y
M. Kleindienst C. Ramsauer Der Beitrag von Lernfabriken zu Industrie 4.0-Ein Baustein zur vierten industriellen Revolution bei kleinen und mittelständischen Unternehmen Industrie-Management 2015 3 41 44
J. Li J. Gu Z. Huang J. Wen Application research of improved YOLO V3 algorithm in PCB electronic component detection Applied Sciences 2019 9 18 375 10.3390/app9183750
Lin, T. Y., Goyal, P., Girshick, R., He, K., & Dollár, P. (2017). Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision (pp. 2980–2988).
Lin, T. Y., Maire, M., Belongie, S., Hays, J., Perona, P, Ramanan, D., et al. (2014). Microsoft coco: Common objects in context. In Computer vision–ECCV 2014: 13th European conference, Zurich, Switzerland, September 6–12, 2014, Proceedings, Part V 13 (pp. 740–755). Springer.
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., et al. (2016). Ssd: Single shot multibox detector. In Computer vision–ECCV 2016: 14th European conference, Amsterdam, The Netherlands, October 11–14, Proceedings, Part I 14 (pp. 21–37). Springer.
Lowe, D. G. (1999). Object recognition from local scale-invariant features. In Proceedings of the seventh IEEE international conference on computer vision (Vol. 2, pp. 1150–1157). IEEE.
Malisiewicz, T., Gupta, A., & Efros, A. A. (2011). Ensemble of exemplar-svms for object detection and beyond. In 2011 international conference on computer vision (pp. 89–96). IEEE.
Mo, Z., Chen, L., & You, W. (2019). Identification and detection of automotive door panel solder joints based on YOLO. In Chinese control and decision conference (CCDC) (pp. 5956–5960). IEEE.
R. Müller M. Vette-Steinkamp A. Kanso Position and orientation calibration of a 2D laser line sensor using closed-form least-squares solution IFAC-PapersOnLine 2019 52 13 689 694 10.1016/j.ifacol.2019.11.136
S.S. Park V.T. Tran D.E. Lee Application of various YOLO models for computer vision-based real-time pothole detection Applied Sciences 2021 11 23 1122 10.3390/app112311229
P. Pierleoni A. Belli L. Palma L. Sabbatini A versatile machine vision algorithm for real-time counting manually assembled pieces Journal of Imaging 2020 6 6 4 10.3390/jimaging6060048
Redmon, J., & Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767.
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 779–788).
Redmon, J., & Farhadi, A. (2017). YOLO9000: Better, faster, stronger. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7263–7271).
Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems, 28.
Z. Ren F. Fang N. Yan Y. Wu State of the art in defect detection based on machine vision International Journal of Precision Engineering and Manufacturing-Green Technology 2022 9 2 661 69 10.1007/s40684-021-00343-6
Tahmina, T., Garcia, M., Geng, Z., & Bidanda, B. (2022). A survey of smart manufacturing for high-mix low-volume production in defense and aerospace industries. In: International conference on flexible automation and intelligent manufacturing (p. 237–245). Springer.
J. Terven D.M. Córdova-Esparza J.A. Romero-González A comprehensive review of YOLO architectures in computer vision: From YOLOv1 to YOLOv8 and YOLO-NAS Machine Learning and Knowledge Extraction 2023 5 4 1680 1716 10.3390/make5040083
Tkachenko, M., Malyuk, M., Holmanyuk, A., & Liubimov, N. (2020) Label studio: Data labeling software. Retrieved from https://github.com/heartexlabs/label-studio
Transeth, A. A., Stepanov, A., Linnerud, Å. S., Ening, K., & Gjerstad, T. (2020). Competitive high variance, low volume manufacturing with robot manipulators. In 3rd international symposium on small-scale intelligent manufacturing systems (SIMS) (pp. 1–7). IEEE.
S. Vaidya P. Ambad S. Bhosle Industry 4.0–a glimpse Procedia Manufacturing 2018 20 233 238 10.1016/j.promfg.2018.02.034
Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. In Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition (Vol. 1, pp. I–I). IEEE.
P. Viola M.J. Jones Robust real-time face detection International Journal of Computer Vision 2004 57 137 154 10.1023/B:VISI.0000013087.49260.fb
L. Yi C. Siedler Y. Kinkel M. Glatt P. Kölsch J.C. Aurich Object detection in factory based on deep learning approach Procedia CIRP 2021 104 1029 103 10.1016/j.procir.2021.11.173
Zhang, S., Wen, L., Bian, X., Lei, Z., & Li, S. Z. (2018). Single-shot refinement neural network for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4203–4212).
Z. Zhang A flexible new technique for camera calibration IEEE Transactions on Pattern Analysis and Machine Intelligence 2000 22 11 1330 1334 10.1109/34.888718
Zhao, Q., Sheng, T., Wang, Y., Tang, Z., Chen, Y., Cai, L., et al. (2019). M2det: A single-shot object detector based on multi-level feature pyramid network. In Proceedings of the AAAI conference on artificial intelligence (Vol. 33, pp. 9259–9266).
X. Zheng J. Chen H. Wang S. Zheng Y. Kong A deep learning-based approach for the automated surface inspection of copper clad laminate images Applied Intelligence 2021 51 1262 1279 10.1007/s10489-020-01877-z
Zheng, Z., Wang, P., Liu, W., Li, J., Ye, R., & Ren, D. (2019) Distance-IoU loss: Faster and better learning for bounding box regression. Retrieved from http://arxiv.org/abs/1911.08287
R.Y. Zhong X. Xu E. Klotz S.T. Newman Intelligent manufacturing in the context of industry 4.0: A review Engineering 2017 3 5 616 63 10.1016/J.ENG.2017.05.015
Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., & Torralba, A. (2017). Scene parsing through ade20k dataset. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 633–641).
Z. Zou K. Chen Z. Shi Y. Guo J. Ye Object detection in 20 years: A survey Proceedings of the IEEE 2023 111 3 257 276 10.1109/JPROC.2023.3238524