Depth Estimation and Deblurring from a Single Image using an Optimized-throughput Coded Aperture

Masoudifar, M.; Pourreza, H. R.

doi:10.22061/jecei.2022.8630.537

نشریات مستقل دانشگاه در سامانه ارزیابی نشریات علمی وزارت علوم

نشریه معماری وشهرسازی پایدار موفق به اخذ رتبه علمی-پژوهشی شد

تعداد نشریات	13
تعداد شماره‌ها	200
تعداد مقالات	2,008
تعداد مشاهده مقاله	2,679,471
تعداد دریافت فایل اصل مقاله	1,936,868

	Depth Estimation and Deblurring from a Single Image using an Optimized-throughput Coded Aperture
Journal of Electrical and Computer Engineering Innovations (JECEI)
مقاله 25، دوره 11، شماره 1، فروردین 2023، صفحه 51-64 اصل مقاله (1.27 M)
نوع مقاله: Original Research Paper
شناسه دیجیتال (DOI): 10.22061/jecei.2022.8630.537
نویسندگان
M. Masoudifar^* ¹؛ H. R. Pourreza²
¹Department of Computer Engineering, Hakim Sabzevari University, Sabzevar, Iran.
²Computer Engineering Department, Ferdowsi University of Mashhad, Mashhad, Iran.
تاریخ دریافت: 02 بهمن 1400، تاریخ بازنگری: 08 اردیبهشت 1401، تاریخ پذیرش: 07 خرداد 1401
چکیده
Background and Objectives: Depth from defocus and defocus deblurring from a single image are two challenging problems caused by the finite depth of field in conventional cameras. Coded aperture imaging is a branch of computational imaging, which is used to overcome these two problems. Up to now, different methods have been proposed for improving the results of either defocus deblurring or depth estimation. In this paper, an asymmetric coded aperture is proposed which improves results of depth estimation and defocus deblurring from a single input image. Methods: To this aim, a multi-objective optimization function taking into consideration both deblurring results and depth discrimination ability is proposed. Since aperture throughput affects on image quality, our optimization function is defined based on illumination conditions and camera specifications which yields an optimized throughput aperture. Because the designed pattern is asymmetric, defocused objects on two sides of the focal plane can be distinguished. Depth estimation is performed using a new algorithm, which is based on perceptual image quality assessment criteria and can discern blurred objects lying in front or behind the focal plane. Results: Extensive simulations as well as experiments on a variety of real scenes are conducted to compare the performance of our aperture with previously proposed ones. Conclusion: Our aperture has been designed for indoor illumination settings. However, the proposed method can be utilized for designing and evaluating appropriate aperture patterns for different imaging conditions.
کلیدواژه‌ها
Coded Apertures؛ Depth from Defocus؛ Defocus Deblurring

مراجع
[1] A. P. Pentland, "A new sense for depth of field," IEEE Trans. Pattern Anal. Mach. Intell., 9(4): 523-531, 1987. [2] M. Subbarao, N. Gurumoorthy, "Depth recovery from blurred edges," in Proc. Computer Society Conference on Computer Vision and Pattern Recognition, CVPR'88: 498-503, 1988. [3] A. Sellent, P. Favaro, "Which side of the focal plane are you on?," in Proc. 2014 IEEE International Conference on Computational Photography (ICCP): 1-8, 2014. [4] A. Levin, R. Fergus, F. Durand, W. T. Freeman, "Image and depth from a conventional camera with a coded aperture," ACM Trans. Graphics, 26(3): 70, 2007. [5] A. Sellent, P. Favaro, "Optimized aperture shapes for depth estimation," Pattern Recognit. Lett., 40: 96-103, 2014. [6] A. Veeraraghavan, R. Raskar, A. Agrawal, A. Mohan, J. Tumblin, "Dappled photography: Mask enhanced cameras for heterodyned light fields and coded aperture refocusing," ACM Trans. Graphics, 26(3): 1-12, 2007. [7] C. Zhou, S. Nayar, "What are good apertures for defocus deblurring?" in Proc. 2009 IEEE International Conference on Computational Photography (ICCP): 1-8, 2009. [8] B. Masia, L. Presa, A. Corrales, D. Gutierrez, "Perceptually optimized coded apertures for defocus deblurring," Comput. Graphics Forum, 31(6): 1867-1879, 2012. [9] Y. Takeda, S. Hiura, K. Sato, "Fusing depth from defocus and stereo with coded apertures," in Proc 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR): 209-216, 2013. [10] C. Zhou, S. Lin, S. K. Nayar, "Coded aperture pairs for depth from defocus and defocus deblurring," Int. J. Comput. Vision, 93(1): 53-72, 2011. [11] A. Konak, D. W. Coit, A. E. Smith, "Multi-objective optimization using genetic algorithms: A tutorial," Reliab. Eng. Syst. Saf., 91(9): 992-1007, 2006. [12] K. Mitra, O. S. Cossairt, A. Veeraraghavan, "A framework for analysis of computational imaging systems: Role of signal prior, sensor noise and multiplexing," IEEE Trans. Pattern Anal. Mach. Intell., 36(10): 1909-1921, 2014. [13] V. Paramonov, I. Panchenko, V. Bucha, A. Drogolyub, S. Zagoruyko, "Depth camera based on color-coded aperture," in Proc. the IEEE Conference on Computer Vision and Pattern Recognition Workshops: 1-9, 2016. [14] V. Aslantas, "A depth estimation algorithm with a single image," Opt. express, 15(8): 5024-5029, 2007. [15] S. Zhuo, T. Sim, "Defocus map estimation from a single image," Pattern Recognit., 44(9): 1852-1858, 2011. [16] J. Lin, X. Ji, W. Xu, Q. Dai, "Absolute depth estimation from a single defocused image," IEEE Trans. Image Process., 22(11): 4545-4550, 2013. [17] X. Zhu, S. Cohen, S. Schiller, P. Milanfar, "Estimating spatially varying defocus blur from a single image," IEEE Trans. Image Process., 22(12): 4879-4891, 2013. [18] S. Gur, L. Wolf, "Single image depth estimation trained via depth from defocus cues," in Proc. the IEEE/CVF Conference on Computer Vision and Pattern Recognition: 7683-7692, 2019. [19] P. Hambarde, S. Murala, "S2DNet: Depth estimation from single image and sparse samples," IEEE Trans. Comput. Imaging, 6: 806-817, 2020. [20] A. N. Rajagopalan, S. Chaudhuri, "Optimal selection of camera parameters for recovery of depth from defocused images," in Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition: 219-224, 1997. [21] M. Watanabe, S. K. Nayar, "Rational filters for passive depth from defocus," Int. J. Comput. Vision, 27(3): 203-225, 1998. [22] P. Favaro, S. Soatto, "A geometric approach to shape from defocus," IEEE Trans. Pattern Anal. Mach. Intell., 27(3): 406-417, 2005. [23] Matsui, H. Nagahara, R.I. Taniguchi, "Half-sweep imaging for depth from defocus," Image Vision Comput., 32(11): 954-964, 2014. [24] M. Ye, X. Chen, Q. Li, J. Zeng, S. Yu, "Depth from defocus measurement method based on liquid crystal lens," Opt. Express, 26(2): 28413-28420, 2018. [25] S. W. Hasinoff, K. N. Kutulakos, "Confocal stereo," Int. J. comput. vision, 81(1): 82-104, 2009. [26] A. Rajagopalan, S. Chaudhuri, U. Mudenagudi, "Depth estimation and image restoration using defocused stereo pairs," IEEE Trans. Pattern Anal. Mach. Intell., 26(11): 1521-1525, 2004. [27] S. Hiura, T. Matsuyama, "Depth measurement by the multi-focus camera," in Proc. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition: 953-959, 1998. [28] O. Cossairt, M. Gupta, S. K. Nayar, "When does computational imaging improve performance?," IEEE Trans. Image Process., 22(2): 447-458, 2013. [29] Y. Weiss, W. T. Freeman, "What makes a good model of natural images?," in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR'07): 1-8, 2007. [30] P. E. Debevec, J. Malik, "Recovering high dynamic range radiance maps from photographs," in ACM SIGGRAPH 2008 classes: 1-10, 2008. [31] O. Cossairt, "Tradeoffs and limits in computational imaging," Columbia University, 2011. [32] K. Deb, A. Pratap, S. Agarwal, T. Meyarivan, "A fast and elitist multiobjective genetic algorithm: NSGA-II," IEEE Trans. Evol. Comput., 6(2): 182-197, 2002. [33] Y. Gao, "Population size and sampling complexity in genetic algorithms," in Proc. the Bird of a Feather Workshops: 178-181, 2003. [34] S. Theodoridis, R. Chellappa, Academic Press Library in Signal Processing: Image, Video Processing and Analysis, Hardware, Audio, Acoustic and Speech Processing: Elsevier Science, 2013. [35] M. Martinello, "Coded aperture imaging," Heriot-Watt University, Edinburgh, Scotland, 2012. [36] Y. Liu, J. Wang, S. Cho, A. Finkelstein, S. Rusinkiewicz, "A no-reference metric for evaluating the quality of motion deblurring," ACM Trans. Graph., 32(6): 175, 2013. [37] B. Hu, L. Li, J. Qian, "Perceptual quality evaluation for motion deblurring," IET Comput. Vision, 12(6): 796-805, 2018. [38] D. Krishnan, T. Tay, R. Fergus, "Blind deconvolution using a normalized sparsity measure," in Proc. 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR): 233-240, 2011. [39] G. Blanchet, L. Moisan, "An explicit sharpness index related to global phase coherence," in Proc. 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP): 1065-1068, 2012. [40] M. Masoudifar, H. R. Pourreza, "Depth estimation from a single defocused image using no reference image quality assessment metrics," presented at the Fifth International Conference on Technology Development in Iranian Electrical Engineering, 2021. [41] A. Delong, A. Osokin, H. N. Isack, Y. Boykov, "Fast approximate energy minimization with label costs," Int. J. comput. vision, 96(1): 1-27, 2012.
آمار تعداد مشاهده مقاله: 455 تعداد دریافت فایل اصل مقاله: 366

سامانه مدیریت نشریات علمی. قدرت گرفته از سیناوب

پیوندهای مفید

پیوندهای مفید

اخبار و اعلانات

آمار

Depth Estimation and Deblurring from a Single Image using an Optimized-throughput Coded Aperture