Reconstructing Plants in 3D from a single Image using Analysis by Synthesis

Abstract

Mature computer vision techniques allow the reconstruction of challenging 3D objects from images. However, due to high complexity of plant topology, dedicated methods for generating 3D plant models must be devised. We propose to generate a 3D model of a plant, using an analysis-by-synthesis method mixing information from a single image and a priori knowledge of the plant species. First, our dedicated skeletonisation algorithm generates a possible branch- ing structure from the foliage segmentation. Then, a 3D generative model, based on a parametric model of branching systems that takes into ac- count botanical knowledge is built. The resulting skeleton follows the hierarchical organisation of natural branching structures. An instance of a 3D model can be generated. Moreover, varying parameter values of the generative model (main branching structure of the plant and foliage), we produce a series of candidate models. The reconstruction is improved by selecting the model among these proposals based on a matching criterion with the image. Realistic results obtained on di erent species of plants illustrate the performance of the proposed method.