Class-aware single image to 3D object translational autoencoder

GÜZEL TURHAN C., BİLGE H. Ş.

IET IMAGE PROCESSING, cilt.14, sa.13, ss.3046-3053, 2020 (SCI-Expanded) identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 14 Sayı: 13
  • Basım Tarihi: 2020
  • Doi Numarası: 10.1049/iet-ipr.2019.1152
  • Dergi Adı: IET IMAGE PROCESSING
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Academic Search Premier, PASCAL, Aerospace Database, Applied Science & Technology Source, Business Source Elite, Business Source Premier, Communication Abstracts, Compendex, Computer & Applied Sciences, INSPEC, Metadex, Civil Engineering Abstracts
  • Sayfa Sayıları: ss.3046-3053
  • Anahtar Kelimeler: image segmentation, feature extraction, image reconstruction, image coding, class-aware single image, 3D object translational autoencoder, three-dimensional domain, reconstruction problem, middle-level features, skip connections, 2D features, category-agnostic model, class-annotations, 3D reconstruction models, intersection-over-union score, object segmentation model, reconstruction performance, skipped volumetric class-aware AE, class-aware nature
  • Gazi Üniversitesi Adresli: Evet

Özet

The performances of generative adversarial network (GAN) and autoencoder (AE) models on images have been gathering a great deal of interest in terms of transferring them to three-dimensional (3D) domain. In this study, single image to object reconstruction problem was focused by presenting a novel 2D-to-3D AE model inspired by the recent improvements. To benefit from middle-level features, a model with skip connections was constructed by transferring 2D features to 3D domain. Moreover, the authors considered class-awareness for obtaining a category-agnostic model using limited class-annotations. Apart from recent 3D reconstruction models, they adapted the intersection-over-union score based objective, which is used in the object segmentation model, for improving reconstruction performance. With all these contributions, they call their model as skipped volumetric class-aware AE (SkipVCAE). According to experimental studies, proposed model obtained higher scores than the state-of-the-art model given. The results have proven its performance as a category-specific and category-agnostic model together owing to its class-aware nature. In the further analysis, it was seen that presented model yielded satisfactory results on a single image to object modelling compared to its multi-view version thanks to class-awareness.