Multi-Task Learning Using Uncertainty for Realtime Multi-Person Pose Estimation

Ghasemi-Naraghi, Z.; Nickabadi, A.; Safabakhsh, R.

doi:10.22061/jecei.2023.9848.657

نشریات مستقل دانشگاه در سامانه ارزیابی نشریات علمی وزارت علوم

نشریه معماری وشهرسازی پایدار موفق به اخذ رتبه علمی-پژوهشی شد

تعداد نشریات	13
تعداد شماره‌ها	200
تعداد مقالات	2,008
تعداد مشاهده مقاله	2,679,689
تعداد دریافت فایل اصل مقاله	1,936,975

	Multi-Task Learning Using Uncertainty for Realtime Multi-Person Pose Estimation
Journal of Electrical and Computer Engineering Innovations (JECEI)
مقاله 10، دوره 12، شماره 1، فروردین 2024، صفحه 147-162 اصل مقاله (1.33 M)
نوع مقاله: Original Research Paper
شناسه دیجیتال (DOI): 10.22061/jecei.2023.9848.657
نویسندگان
Z. Ghasemi-Naraghi؛ A. Nickabadi^* ؛ R. Safabakhsh
Artificial Intelligence and Robotics Department, Faculty of Computer Engineering and Information Technology, Amirkabir University of Technology, Tehran , Iran.
تاریخ دریافت: 03 تیر 1402، تاریخ بازنگری: 21 شهریور 1402، تاریخ پذیرش: 16 مهر 1402
چکیده
Background and Obejctives: Multi-task learning is a widespread mechanism to improve the learning of multiple objectives with a shared representation in one deep neural network. In multi-task learning, it is critical to determine how to combine the tasks loss functions. The straightforward way is to optimize the weighted linear sum of multiple objectives with equal weights. Despite some studies that have attempted to solve the realtime multi-person pose estimation problem from a 2D image, major challenges still remain unresolved. Methods: The prevailing solutions are two-stream, learning two tasks simultaneously. They intrinsically use a multi-task learning approach for predicting the confidence maps of body parts and the part affinity fields to associate the parts to each other. They optimize the average of the two tasks loss functions, while the two tasks have different levels of difficulty and uncertainty. In this work, we overcome this problem by applying a multi-task objective that captures task-based uncertainties without any additional parameters. Since the estimated poses can be more certain, the proposed method is called “CertainPose”. Results: Experiments are carried out on the COCO keypoints data sets. The results show that capturing the task-dependent uncertainty makes the training procedure faster and causes some improvements in human pose estimation. Conclusion: The highlight advantage of our method is improving the realtime multi-person pose estimation without increasing computational complexity.
کلیدواژه‌ها
Realtime Multi-Person Pose Estimation؛ Multi-Task Learning؛ Loss Function؛ Task-Dependent Uncertainty

مراجع
[1] Y. Yang, D. Ramanan, “Articulated pose estimation with flexible mixtures-of-parts,” in Proc. CVPR 2011: 1385–1392, 2011. [2] P. F. Felzenszwalb, D. P. Huttenlocher, “Pictorial structures for object recognition,” Int. J. Comput. Vision, 61(1): 55–79, 2005. [3] X. Chen, A. L. Yuille, “Articulated pose estimation by a graphical model with image dependent pairwise relations,” in Proc. NIPS, 2014. [4] K. He, X. Zhang, S. Ren, J. Sun, “Deep residual learning for image recognition,” in Proc. the IEEE Conference on Computer Vision and Pattern Recognition: 770–778, 2016. [5] S. Ren, K. He, R. Girshick, J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” in Proc. NIPS, 2015. [6] G. Papandreou, T. Zhu, N. Kanazawa, A. Toshev, J. Tompson, C. Bregler, K. Murphy, “Towards accurate multi-person pose estimation in the wild,” in Proc. the IEEE Conference on Computer Vision and Pattern Recognition: 4903–4911, 2017. [7] K. He, G. Gkioxari, P. Dollár, R. Girshick, “Mask r-cnn,” in Proc. the IEEE International Conf. on Computer Vision: 2961–2969, 2017. [8] S. E. Wei, V. Ramakrishna, T. Kanade, Y. Sheikh, ” Convolutional pose machines,” in Proc. the IEEE Conference on Computer Vision and Pattern Recognition: 4724–4732, 2016. [9] H. S. Fang, S. Xie, Y. W. Tai, C. Lu, “Rmpe: Regional multi-person pose estimation,” in Proc. the IEEE International Conference on Computer Vision: 2334–2343, 2017. [10] L. Ladicky, P. H. Torr, A. Zisserman, “Human pose estimation using a joint pixel-wise and part-wise formulation,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition: 3578– 585, 2013. [11] U. Iqbal, J. Gall, “Multi-person pose estimation with local joint-to-person associations,” in Proc. European Conference on Computer Vision: 627–642, 2016. [12] L. Pishchulin, E. Insafutdinov, S. Tang, B. Andres, M. Andriluka, P. V. Gehler, B. Schiele, “Deepcut: Joint subset partition and labeling formulti person pose estimation,” in Proc. the IEEE Conference on Computer Vision and Pattern Recognition: 4929–4937, 2016. [13] E. Insafutdinov, L. Pishchulin, B. Andres, M. Andriluka, B. Schiele, “Deepercut: A deeper, stronger, and faster multi-person pose estimation model,” in Proc. European Conference on Computer Vision: 34–50, 2016. [14] Z. Cao, T. Simon, S. E. Wei, Y. Sheikh, “Realtime multi-person 2dpose estimation using part affinity fields,” in Proc. the IEEE Conf. Computer Vision and Pattern Recognition: 7291–7299, 2017. [15] X. Zhu, Y. Jiang, Z. Luo, “Multi-person pose estimation for posetrack with enhanced part affinity fields,” in Proc. ICCV PoseTrack Workshop, 7, 2017. [16] D. Osokin, “Real-time 2d multi-person pose estimation on cpu: Lightweight openpose,” in Proc. the 8th International Conference on Pattern Recognition Applications and Methods: 744–748, 2019. [17] Z. Cao, G. Hidalgo, T. Simon, S. E. Wei, Y. Sheikh, “Openpose: real- time multi-person 2d pose estimation using part affinity fields,” IEEE Trans. Pattern Anal. Mach. Intell., 43(1): 172–186, 2019. [18] OpenPose library. https://github.com/CMU-Perceptual-Computing-Lab/openpose. [19] H. Liu, D. Luo, S. Du, T. Ikenaga, “Resolution irrelevant encoding and difficulty balanced loss based network independent supervision for multi-person pose estimation,” in Proc. 13th International Conf. Human System Interaction (HSI): 112–117, 2020. [20] G. H. Martınez, “OpenPose: Whole-Body Pose Estimation,” April, 2019. [21] T. Gong, T. Lee, C. Stephenson, V. Renduchintala, S. Padhy, A. Ndi-rango, G. Keskin, O. H. Elibol, “A comparison of loss weighting strategies for multi task learning in deep neural networks,” IEEE Access, 7: 141627–141632, 2019. [22] Q. Dang, J. Yin, B. Wang, W. Zheng, “Deep learning based 2d human pose estimation: A survey,” Tsinghua Sci. Technol., 24(6): 663– 676, 2019. [23] C. Zheng, W. Wu, T. Yang, S. Zhu, C. Chen, R. Liu, J. Shen, N. Kehtarnavaz, M. Shah, “Deep learning-based human pose estimation: A survey,” arXiv preprint arXiv:2012.13392, 2020. [24] T. L. Munea, Y. Z. Jembre, H. T. Weldegebriel, L. Chen, C. Huang, C. Yang, “The progress of human pose estimation: A survey and taxonomy of models applied in 2d human pose estimation,” IEEE Access, 8: 133330–133348, 2020. [25] W. Gong, X. Zhang, J. Gonzàlez, A. Sobral, T. Bouwmans, C. Tu, E. H. Zahzah, “Human pose estimation from monocular images: A comprehensive survey,” Sensors, 16(12): 1966, 2016. [26] G. Rogez, C. Schmid, “Mocap-guided data augmentation for 3d pose estimation in the wild,” in Proc. Advances in Neural Information Processing Systems (NIPS): 3108–3116, 2016. [27] H. Jiang, “Finding human poses in videos using concurrent matching and segmentation,” in Proc. Asian Conference on Computer Vision: 228–243, 2010. [28] H. Sidenbladh, F. De la Torre, M. J. Black, “A framework for modeling the appearance of 3d articulated figures,” in Proc. Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580): 368–375, 2000. [29] J. J. Tompson, A. Jain, Y. LeCun, C. Bregler, “Joint training of a convolutional network and a graphical model for human pose estimation,” in Proc. Advances in Neural Information Processing Systems, 27, 2014. [30] V. Ramakrishna, D. Munoz, M. Hebert, J. A. Bagnell, Y. Sheikh, “Pose machines: Articulated pose estimation via inference machines,” in Proc. European Conf. Computer Vision: 33–47, 2014. [31]MSCOCO Dataset, https://cocodataset.org/#home. [32] T. Simon, H. Joo, I. Matthews, Y. Sheikh, “Hand keypoint detection in single images using multiview bootstrapping,” in Proc. the IEEE Conf. Computer Vision and Pattern Recognition: 1145–1153, 2017. [33] S. Kreiss, L. Bertoni, A. Alahi, “Pifpaf: Composite fields for human pose estimation,” in Proc. the IEEE/CVF Conference on Computer Vision and Pattern Recognition: 11977–11986, 2019. [34] N. Nakano, T. Sakura, K. Ueda, L. Omura, A. Kimura, Y. Iino, S. Fukashiro, S. Yoshioka, “Evaluation of 3d markerless motion capture accuracy using openpose with multiple video cameras,” Fron. sports Active Living, 2(50), 2020. [35] N. D. Reddy, L. Guigues, L. Pishchulin, J. Eledath, S. G. Narasimhan, “Tessetrack: End-to-end learnable multi-person articulated 3d pose tracking,” in Proc. the IEEE/CVF Conference on Computer Vision and Pattern Recognition: 15190–15200, 2021. [36] N. D. Reddy, M. Vo, S. G. Narasimhan, “Occlusion-net: 2d/3d occluded keypoint localization using graph networks,” in Proc. the IEEE/CVF Conference on Computer Vision and Pattern Recognition: 7326–7335, 2019. [37] Y. Cheng, B. Wang, B. Yang, R. T. Tan, “Monocular 3d multi-person pose estimation by integrating top-down and bottom-up networks,” in Proc. the IEEE/CVF Conference on Computer Vision and Pattern Recognition: 7649–7659, 2021. [38] H. Tu, C. Wang, W. Zeng, “Voxelpose: Towards multi-camera 3d human pose estimation in wild environment,” in Proc. 16th European Conference on Computer Vision–ECCV 2020, Part I 16: 197–212, 2020. [39] G. Zhang, J. Liu, H. Li, Y. Q. Chen, L. S. Davis, “Joint human detection and head pose estimation via multistream networks for rgb-d videos,” IEEE Signal Process. Lett., 24(11): 1666–1670, 2017. [40] M. Schwarz, H. Schulz, S. Behnke, “Rgb-d object recognition and pose estimation based on pre-trained convolutional neural network features,” in Proc. 2015 IEEE International Conference on Robotics and Automation (ICRA): 1329–1335, 2015. [41] A. Krull, E. Brachmann, F. Michel, M. Y. Yang, S. Gumhold, C. Rother, “Learning analysis-by-synthesis for 6d pose estimation in rgb-d images,” in Proc. the IEEE International Conference on Computer Vision: 954–962, 2015. [42] Y. Gal, :Uncertainty in deep learning,” University of Cambridge 1(3), 2016. [43] A. Kendall, Y. Gal, “What uncertainties do we need in bayesian deep learning for computer vision?” in Proc. Advances in Neural Information Processing Systems: 5574–5584, 2017. [44] F. K. Gustafsson, M. Danelljan, T. B. Schon, “Evaluating scalable bayesian deep learning methods for robust computer vision,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops: 318–319, 2020. [45] I. Misra, A. Shrivastava, A. Gupta, M. Hebert, “Cross-stitch networks for multi-task learning,” in Proc. the IEEE Conference on Computer Vision and Pattern Recognition: 3994–4003, 2016. [46] Z. Chen, V. Badrinarayanan, C. Y. Lee, A. Rabinovich, “Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks,” in Proc. International Conference on Machine Learning: 794–803, 2018. [47] A. Kendall, Y. Gal, R. Cipolla, “Multi-task learning using uncertainty to weigh losses for scene geometry and semantics,” in Proc. the IEEE Conference on Computer Vision and Pattern Recognition: 7482–7491, 2018. [48] T. Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, C.L. Zitnick, “Microsoft coco: Common objects in context. In Proc. European Conf. Computer Vision: 740–755, 2014. [49] M. Ruggero Ronchi, P. Perona, “Benchmarking and error diagnosis in multi-instance pose estimation,” in Proc. the IEEE International Conference on Computer Vision: 369–378, 2017. [50] A. Newell, Z. Huang, J. Deng, “Associative embedding: End-to-end learning for joint detection and grouping,” in Proc. Advances in Neural Information Processing Systems: 2278–2288, 2017. [51] M. Kocabas, S. Karagoz, E. Akbas, “Multiposenet: Fast multi-person pose estimation using pose residual network,” in Proc. the European Conference on Computer Vision (ECCV): 417–433, 2018.
آمار تعداد مشاهده مقاله: 172 تعداد دریافت فایل اصل مقاله: 100

سامانه مدیریت نشریات علمی. قدرت گرفته از سیناوب

پیوندهای مفید

پیوندهای مفید

اخبار و اعلانات

آمار

Multi-Task Learning Using Uncertainty for Realtime Multi-Person Pose Estimation