Class-aware single image to 3D object translational autoencoder


GÜZEL TURHAN C. , BİLGE H. Ş.

IET IMAGE PROCESSING, cilt.14, sa.13, ss.3046-3053, 2020 (SCI İndekslerine Giren Dergi) identifier identifier

  • Cilt numarası: 14 Konu: 13
  • Basım Tarihi: 2020
  • Doi Numarası: 10.1049/iet-ipr.2019.1152
  • Dergi Adı: IET IMAGE PROCESSING
  • Sayfa Sayıları: ss.3046-3053

Özet

The performances of generative adversarial network (GAN) and autoencoder (AE) models on images have been gathering a great deal of interest in terms of transferring them to three-dimensional (3D) domain. In this study, single image to object reconstruction problem was focused by presenting a novel 2D-to-3D AE model inspired by the recent improvements. To benefit from middle-level features, a model with skip connections was constructed by transferring 2D features to 3D domain. Moreover, the authors considered class-awareness for obtaining a category-agnostic model using limited class-annotations. Apart from recent 3D reconstruction models, they adapted the intersection-over-union score based objective, which is used in the object segmentation model, for improving reconstruction performance. With all these contributions, they call their model as skipped volumetric class-aware AE (SkipVCAE). According to experimental studies, proposed model obtained higher scores than the state-of-the-art model given. The results have proven its performance as a category-specific and category-agnostic model together owing to its class-aware nature. In the further analysis, it was seen that presented model yielded satisfactory results on a single image to object modelling compared to its multi-view version thanks to class-awareness.