ОБЗОР МЕТОДОВ УЛУЧШЕНИЯ ВИЗУАЛЬНОГО КАЧЕСТВА ИЗОБРАЖЕНИЙ И ВИДЕО В НЕБЛАГОПРИЯТНЫХ ПОГОДНЫХ УСЛОВИЯХ

В современном мире обработка изображений применяется в различных отраслях деятельности человека. Из-за различных факторов помех ухудшение изображения и видео серьезно снижает точность и эффективность отслеживания и распознавания целей. Следовательно, восстановление изображения становится серьезной проблемой в области компьютерного зрения.

Материалы и методы, результаты и обсуждения В данной исследовании проведѐн обзор методов улучшения визуального качества изображений и видео при искажении их погодными явлениями. Методы классифицированы по видам и различиям погодных явлений. Приведены примеры их апробации на изображениях и на видео. Сделаны выводы о каждом из видов алгоритмов.

Заключение В статье были исследованы различные методы улучшения визуального качества изображений и видео в неблагоприятных погодных условиях. Каждый метод был детально изучен, были рассмотрены их достоинства и недостатки, что позволило прийти к следующим выводам: 1) методы, использующие физические модели, очень эффективны, но являются вычислительно сложными, в связи с этим данный метод целесообразнее использовать для постобработки изображений и видео. 2) Методы, основанные на гистограммах более простые, но они подходят только для статичных погодных условий (туман, дымка, мгла). 3) Наиболее перспективными являются методы, основанные на обучении. Нейронные сети позволяют решать более сложные задачи, в силу возможности распараллеливания информации и дальнейшего обучения. Так же данные методы отлично справляются с улучшение качества как изображений, так и видео. Методы, основанные на обучение применимы как к статичным погодным условиям (туман, дымка, мгла), так и к динамичным (снег, дождь, град), что делает их более универсальными для решения данной проблемы.

1. Kshitiz Garg, Shree K. Nayar. Detection and Removal of Rain from Videos. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (1), July 2004, pp. 528–535.

2. G. Doretto, A. Chiuso, Y.N. Wu, and S. Soatto. Dynamic textures.IJCV, 51(2):91– 109, 2003.

3. K. Garg and S. K. Nayar. Photometric Model for Raindrops. Columbia University Technical Report, 2003.

4. Peter Barnum, Takeo Kanade, Srinivasa G Narasimhan. Spatio-Temporal Frequency Analysis for Removing Rain and Snow from Videos. International Journal of Computer Vision (86), Issue 2, January 2010, pp 256-274.

5. K. Garg and S. K. Nayar. When does a camera see rain? In ICCV, 2005.

6. K. Garg and S. K. Nayar. Photorealistic rendering of rain streaks. In SIGGRAPH, 2006.

7. Z. Xu, X. Liu and N. Ji, "Fog Removal from Color Images using Contrast Limited Adaptive Histogram Equalization," 2009 2nd International Congress on Image and Signal Processing, 2009, pp. 1-5, doi: 10.1109/CISP.2009.5301485.

8. Robby T. Tan, ―Visibility in Bad Weather from a Single Image‖, Computer Vision and Pattern Recognition, 2008, pp. 1-8.

9. Robby T. Tan, Niklas Pettersson and Lars Petersson, ―Visibility Enhancement for Roads with Foggy or Hazy Scenes‖, Proceedings of the 2007 IEEE Intelligent Vehicles Symposium, 2007, pp.19-24.

10. Dongjun Kim, Changwon Jeon, Bonghyup Kang and Hanseok Ko, ―Enhancement of Image Degraded by Fog Using Cost Function Based on Human Visual Model‖, Proceedings of IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, 2008, pp. 64-67.

11. Nicolas Hautière, Jean-Philippe Tarel. Rain or Snow Detection in Image Sequences Through Use of a Histogram of Orientation of Streaks. International Journal of Computer Vision 93(3), July 2011, pp 348-367.

12. N. Jacobs, B. W., N. Fridrich, A. Abrams, K. Miskell, B. Brswell A. Richardson, R. Pless, The global network of outdoor webcams: Properties and apllications, in: ACM International Conference on Advances in Geographic Information Systems, 2009.

13. P. Barnum, S. Narasimhan, T. Kanade, Analysis of rain and snow in frequency space, International Journal of Computer Vision 86 (2-3) (2010) 256–274.

14. J.-P. Tarel, N. Hautière, Fast visibility restoration from a single color or gray level image, in: IEEE International Conference on Computer Vision, 2009, pp. 2201–2208.

15. Y. Lu, J. Cai, H. Zheng and Y. Zeng, "A Deep Meta-Learning Neural Network for Single Image Rain Removal," 2020 13th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), 2020, pp. 231-237, doi: 10.1109/CISPBMEI51763.2020.9263524.

16. S. Gupta and R. K. Sunkaria, "Real-time salt and pepper noise removal from medical images using a modified weighted average filtering," 2017 Fourth International Conference on Image Information Processing (ICIIP), Shimla, 2017, pp. 1-6.

17. L.-F. Shi et al., ―Removing haze ppapers from single image via exponential inference with support vector data description,‖ IEEE Trans. Multimedia, vol. 20, no. 9, pp. 2503–2512, Sep. 2018.

18. K. Zhang, W. Zuo, Y. Chen, D. Meng and L. Zhang, "Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising," in IEEE Transactions on Image Processing, vol. 26, no. 7, pp. 3142-3155, July 2017.

19. H. Zhang and V. M. Patel, "Density-Aware Single Image De-raining Using a Multistream Dense Network," 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, 2018, pp. 695-704.

20. W. Yang, R. T. Tan, J. Feng, Z. Guo, S. Yan and J. Liu, "Joint rain detection and removal from a single image with contextualized deep networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 6, pp. 1377-1393, 1 June 2020.

21. L. Shen, Z. Yue, Q. Chen, F. Feng and J. Ma, "Deep joint rain and haze removal from a single image," 24th International Conference on Pattern Recognition (ICPR), Beijing, 2018, pp. 2821-2826.

22. X. Fu, J. Huang, X. Ding, Y. Liao and J. Paisley, "Clearing the skies: a deep network architecture for single-image rain removal," IEEE Transactions on Image Processing, vol. 26, no. 6, pp. 2944-2956, June 2017.

23. X. Fu, J. Huang, X. Ding, Y. Liao, and J. Paisley, "Removing rain from single images via a deep detail network," Proc. IEEE Conf. Comput. Vis. Pattern Recognit, Jul. 2017, pp. 1715–1723.

24. Li X,Wu J,Lin Z,et al. "Recurrent squeeze-and-excitation context aggregation net for single image deraining," Proceedings of the European Conference on Computer Vision( ECCV) , 2018, pp.254-269. Authorized licensed.

25. V. Santhaseelan and V. K. Asari, ―Utilizing local phase information to remove rain from video,‖ International Journal on Computer Vision, vol. 112, no. 1, pp. 71–89, 2015

26. P. C. Barnum, S. Narasimhan, and T. Kanade, ―Analysis of rain and snow in frequency space,‖ Int. J. Comput. Vis., vol. 86, no. 2–3, pp.256–274, 2010.

27. Automatic Single-Image-Based Rain Streaks Removal via Image Decomposition, IEEE Transactions on Image Processing (21), No. 4, April 2012, pp. 1742-1755.

28. X. Zhang, H. Li, Y. Qi, W. K. Leow, and T. K. Ng, “Rain removal in video by combining temporal and chromatic properties,” in Proc. IEEE Int. Conf. Multimedia Expo., Toronto, ON, Canada, Jul. 2006, pp. 461–464.

29. N. Brewer and N. Liu, “Using the shape characteristics of rain to identify and remove rain from video,” Lecture Notes Comput. Sci., vol. 5342/2008, pp. 451–458, 2008.

30. J. Bossu, N. Hautière, and J. P. Tarel, “Rain or snow detection in image sequences through use of a histogram of orientation of streaks,” Int. J. Comput. Vis., vol. 93, no. 3, pp. 348– 367, Jul. 2011.

31. M. S. Shehata, J. Cai, W. M. Badawy, T. W. Burr, M. S. Pervez, R. J. Johannesson, and A. Radmanesh, “Video-based automatic incident detection for smart roads: The outdoor environmental challenges regarding false alarms,” IEEE Trans. Intell. Transp. Syst., vol. 9, no. 2, pp. 349–360, Jun. 2008.

32. M. Roser and A. Geiger, “Video-based raindrop detection for improved image registration,” in IEEE Int. Conf. Comput. Vis.Workshops, Kyoto, Sep. 2009, pp. 570–577.

33. J. C. Halimeh and M. Roser, “Raindrop detection on car windshields using geometric–photometric environment construction and intensitybased correlation,” in Proc. IEEE Intell. Veh. Symp., Xi’an, China, Jun. 2009, pp. 610–615. Google Goggles [Online]. Available: http://www.google.com/mobile/goggles/

34. D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis., vol. 60, no. 2, pp. 91–110, Nov. 2004.

35. H. Baya, A. Essa, T. Tuytelaarsb, and L. V. Gool, “Speeded-up robus features (SURF),” Comput. Vis. Image Understand., vol. 110, no. 3, pp. 346–359, Jun. 2008.

36. Zhen Jia, Hongcheng Wang, Rodrigo Caballero, Ziyou Xiong, Jianwei Zhao, Alan Finn. A Two-Step Approach to See-Through Bad Weather for Surveillance Video Quality Enhancement. Proceedings of the IEEE International Conference on Robotics and Automation (23), Issue 6, November 2012, pp 1059-1082.

37. R. Dale-Jones and T. Tjahjadi. A study and modification of the local histogram equalization algorithm. Pattern Recognition, 26(9):1373– 1381, 2007. D. Forsyth and J. Ponce. Computer Vision: A Modern Approach. Prentice Hall, 2003.

38. A. Golan and A. Levy. Method of adaptive image contrast enhancement. US Patent 20070031055.

39. R. C. Gonzalez and R. E. Woods. Digital Image Processing. Prentice Hall, 2001.

40. K. He, J. Sun, and X. Tang. Single image haze removal using dark channel prior. In Proceedings of 2009 IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), July 2009.

41. Z. Jia, H. Wang, R. Caballero, Z. Xiong, J. Zhao and A. Finn. Realtime content adaptive contrast enhancement for see-through fog and rain. In Proceedings of 2009 IEEE International Conference on International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pages 1378–1381, March 2010. H. Malm, M. Oskarsson, E. Warrant, P. Clarberg, J. Hasselgren, and C. Lejdfors. Adaptive enhancement and noise reduction in very low light-level video. In Proceedings of IEEE 11th International Conference on Computer vision, pages 1–8, October 2007.

42. S. Narasimhan and S. Nayar. Contrast Restoration of Weather Degraded Images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(6):713–724, Jun 2003.

43. M. Piccardi. Background subtraction techniques: a review. In Proceedings of 2004 IEEE International Conference on Systems, Man and Cybernetics, pages 3099–3104, October 2004.

44. A. M. Reza. Realization of the contrast limited adaptive histogram equalization (clahe) for real-time image enhancement. The Journal of VLSI Signal Processing, 38(1):35–44, 2004.

45. S. Roth and M. J. Black. Fields of experts: A framework for learning image priors. In Proceedings of 2005 IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), pages 860–867, June 2005.

46. R. S. Szeliski. Locally adapted histogram equalization. US Patent 6650774.

47. R. T. Tan. Visibility in bad weather from a single image. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2008), June 2008.

48. Y. Tian, Q. Wan, and F. Wu. Local histogram equalization based on the minimum brightness error. In Proceedings of The Fourth International Conference on Image and Graphics, pages 58–61, August 2007.

49. Q. Wang and R. Ward. Fast image/video contrast enhancement based on weighted thresholded histogram equalization. IEEE Transactions on Consumer Electronics, 53(2):757–764, May 2007.

50. X. Zhang, H. Li, Y. Qi, W. K. Leow, and T. K. Ng. Rain removal in video by combining temporal and chromatic properties. In Proceedings of 2006 IEEE International Conference on Multimedia and Expo, pages 461–464, July 2006.