An inverse scattering approach for geometric body generation: a machine learning perspective

1.   Introduction

With the rapid technological advancement today, the access to realistic 3D human shapes is of great importance in both computer vision and graphics, and has various applications in different industries including virtual game design, film making, bioinformatics^[41], healthcare^[38], and especially, those related to garment design. Some applications involve fitting predictions, virtual try-on simulations^{[13,16,23,34]} or size recommendations^[5,6,7], that help to recommend relevant clothing which would fit specific occasions or fashion trends for online customers. Such applications require a critical ingredient on digital transformation from humans bodies to digital 3D shapes, such that the shapes maintain some of the main features from human bodies.

The traditional approaches to access reliable digital information of a human body are through laser range scanners^[1], stereo reconstruction^[18,22,33] or structured light methods for 3D sensing^[11,20,24]. However, considering the cost of data storage, network transmission and expensive scanning equipment, it is rather unpractical to scan individuals for each application. Hence many studies have been done to generate 3D human shapes based on partial input information. These prior systems can be mainly classified into three types: marker-based systems, silhouette-based systems and measurement-based systems. Marker-based system estimates dynamic 3D human body shapes by capturing a sparse set of marker positions. These techneqiues proceed by using a single static scan or multiple scans and a marker motion capture sequence of the person^[4]. For the static case, silhouette-based system estimates human body shapes based on a set of input images by fitting the silhouette in each view ^[3,8,9,15]. Some apporaches also combine with machine learning that build a correlation between a training dataset of 3D body shapes and a set of 2D images, and then predict a shape based on the correlation^[14].

Although marker-based systems and silhouette-based systems could yield satisfactory reconstructions on 3D human body shapes under tight dresses or naked human shapes, most of the schemes are so computationally expensive and the results are easily affected if heavy or loose clothes are worn. To overcome these difficulties, a great deal of efforts have been devoted to the investigation of simple and fast measurement-based systems ^[17,32,35]. Typically, one considers the landmarks or circumferences from the human structures at specific locations as characteristic values. Since such characteristic values are linear or curvilinear, they are relatively invariant to articulation changes than those silhouettes measurements. If the set of characteristic values is well selected, one can achieve meaningful estimation for both global and local body shapes. Kart et al. (2011) built a system which only requires to input some personal information, such as weight, height and age as well as a 2D photograph. The decision algorithm then determines the human shape according to the measurements and the body mass index (BMI)^[12]. Seo et al. ^[35] presented a human body generation apporach by taking the anthropometric measurements, e.g., stature, crotch length, arm length, neck girth, chest/bust girth, underbust girth, waist girth and hip girth as input. They derived the relationship between the input characteristic values and the preprocessing database of 3D scanned data of human body models by using radial basis interpolation. At run-time, the system generates new human body shapes from the user input characteristic values by fitting the template model onto each scanned data.

In this paper, we develop a completely novel methodology for the geometric body generation, which fulfils the following two basic requirements: (ⅰ) the geometric body generation is automatically determined by the input characteristic sets; (ⅱ) the predicted geometric shape fits for all input characteristic values and moreover it can well approximate the exact geometric body possessing the aforesaid characteristic values. The proposed method follows a machine-learning flavour that generates the inferred geometric body with the customized characteristic parameters from a training dataset. The training dataset consists of some preprocessed body shapes associated with appropriately sampled characteristic parameters. One of the critical ingredients and novelties of our method is the borrowing of inverse scattering techniques in the theory of wave propagation to the body generation. This is done by establishing a delicate one-to-one correspondence between a geometric body and the far-field pattern of a source scattering problem governed by the Helmholtz system. It in turn enables us to establish a one-to-one correspondence between the geometric body space and the function space defined by the far-field patterns. Hence, the far-field patterns can act as the shape generators. The shape generation with prescribed characteristic parameters is achieved by first manipulating the shape generators in the function space and then reconstructing the corresponding geometric body from the obtained shape generator by a stable multiple-frequency Fourier method. The proposed method is in sharp difference from the existing methodologies in the literature, which usually treat the human body as a suitable Riemannian manifold and the generation is based on non-Euclidean approximation and interpolation. In fact, in all of the literature mentioned earlier on manifold learning of body generation, one typically uses Principal Component Analysis (PCA) or Principal Geodesic Analysis (PGA). PCA and PGA are used for optimal reduction of the data and thus efficient deformation by computing statistics on Euclidean manifolds or non-Euclidean manifolds can be achieved; see ^[35,10] and the references therein for more relevant discussion. In our new approach, the shape generator enables us to train the learning dataset via the algebraic operations in the shape space directly without dealing with the deformation of the manifold meshes between geometric shapes.

The rest of the paper is organized as follows. In section 2, we provide rigorous mathematical formulations of characteristic values and shape space. Section 3 introduces the notion of shape generator via the inverse source scattering associated with the Helmholtz system. In section 4, we present the mathematical setup of the geometric body generation from a machine-learning perspective. Section 5 is devoted to the development of the new method for the shape generation. In section 6, we present several two- and three-dimensional numerical examples to show the effectiveness and efficiency of our method. The paper is concluded in section 7 with some relevant discussion.

2.   Preliminary knowledge on shape manifold theory

In this section, we present some preliminary knowledge on the shape manifold theory that shall be needed in our subsequent study of body generation. Generally speaking, a geometric shape or a geometric body is a topological $n$ -manifold, $n\in\mathbb{N}$ , equipped with certain shape descriptors, which give the full information to describe the geometric shape. We call such shape descriptors as characteristic values. We have the following formal definition.

Definition 2.1. Let $D$ be a topological $n$ -manifold with $n\in\mathbb{N}$ . Let $\Lambda_D: = \{\lambda^{(j)}\}_{j\in\mathscr{C}}$ be a set of parameters associated with $D$ that are invariant with respect to isometric deformations and are independent to the parametrizations of $D$ . Here, the cardinality $\mathscr{C}$ might be finite or infinite. $\Lambda_D$ is said to be a characteristic set of $D$ if it uniquely determines $D$ . $D$ and its characteristic set $\Lambda_D$ , written as the $(D, \Lambda_D)$ is referred to as a geometric shape or a geometric body.

Clearly, Definition 2.1 includes much general geometric objects. However, for the present study, we are mainly concerned with the case that $D$ can be embedded into $\mathbb{R}^d$ , $d = 2, 3$ , as a bounded domain. That means, we exclude some interesting cases such as $D$ is a Riemannian surface with boundary in $\mathbb{R}^3$ . Nevertheless, our study is general enough to include the human body as a specific case.

In Definition 2.1, the set of characteristic values is typically a set of measurements which gives a systematic characterization of the size, shape and composition of a geometric object for us to determine the shape of the object. For example, when considering a rectangular object, once can introduce a set of characteristic values containing its height, width and length, which provide all details to determine a unique rectangular shape. Expanding the same idea to human body shapes, one could also use characteristic sets to represent them. There are many different ways to represent a human shape. We would try to group those characteristic values into four main catagories, including Eucidean distance, geodesic distance, circumference and ratio. The Eucidean or geodisic distance is linear or curvilinear distance between two points on the human model, such as stature, crotch length, arm length, shoulder breadth etc. The circumference can be computed by the horizontal girth of the body, such as neck girth, chest/bust girth, under-bust girth, waist girth, hip girth, etc. The ratio can be information of weight, Body Mass Index, muscle and fat rate. In spite of the above characteristic values, one can also consider some pure measurements such as age or gender as characteristic values.

The full set of characteristic values gives the complete information of a geometric shape without lossing any information. It is easy to imagine that the cardinality of a set of characteristic values depends on the complexity of a shape. Hence, the number of characteristic values required can be considered as the dimensionality of the geometric shape. For those complicated objects, like human shapes, it may require infinite set of characteristic values for accurate formulations. Due to practical reasons, one can consider a reasonable truncation of an infinite characteristic set into a finite one for a complicated geometric shape. In doing so, we can consider our study in the following product space

where $\mathscr{D}$ is composed of all the bounded domains in $\mathbb{R}^d$ and $\mathscr{V}$ is an $M$ -dimensional vector space containing the characteristic values. In fact, in the present study, the characteristic values are usually real numbers and one can take $\mathscr{V} = \mathbb{R}^{M}$ with $M\in\mathbb{N}$ . $\mathcal{S}$ is referred to as the geometric shape space. According to the (approximate) one-to-one correspondence between a geometric shape and its characteristic values in Definition 2.1, we readily see that all the shape information can be obtained by a single point of this $M$ -dimensional vector space $\mathscr{V}$ . By adjusting the characteristic values, we can obtain new geometric shapes and this is a key ingredient in our human body interpolation.

3.   Shape generators via inverse source scattering

In the previous section, we introduce the important notion of shape space for our study. We proceed to introduce another critical ingredient, shape generator, for our subsequent study of the geometric body generation. In fact, the generation of a new geometric shape shall be based on algebraic interpolation of exemplar models from the shape space. If the algebraic operations are to be conducted directly in the shape space, dealing with geometric deformations of manifolds, one would certainly encounter very complicated and tedious calculations and manipulations because of the lack of global parametrizations for the non-Eucidean shapes involved. The shape generator can overcome this challenge by bridging the geometric shape space and the function space. To that end, we next introduce the inverse scattering problem in finding an active source from its generated far-field pattern.

Let $f:\mathbb{R}^d\mapsto\mathbb{C}$ be a function having a compact support, $f = \chi_D \varphi$ , where $D\subset\mathbb{R}^d$ is a bounded domain and $\varphi\in L^\infty(\mathbb{R}^d)$ . The set $D$ is the external shape of $f$ while $\varphi$ describes the intensity of the source at various points in $D$ . We assume that $\varphi$ and $D$ do not depend on the wavenumber $k\in\mathbb{R}_+$ . In other words we are considering monochomatic scattering. The source $f$ produces a scattered wave $u\in H^2_{loc}(\mathbb{R}^d)$ given by the unique solution to

where $r = |x|$ for $x\in\mathbb{R}^d$ . The limit in (3.1) is known as the Sommerfeld radiation condition which characterizes the outgoing nature of the radiating wave. By the limiting absorption principle (cf. ^[21]), the solution to (3.1) can be computed as follows,

where

signifies the Fourier transform of $f$ . Inverting the Fourier transform in (3.2), one has the following integral representation,

where $H_{(d-2)/2}^{(1)}$ is the first-kind Hankel function of order $(d-2)/2$ . Stationary phase applied to (3.4) yields that

where $\hat x: = x/|x|\in\mathbb{S}^{d-1}$ , $x\in\mathbb{R}^d\backslash\{0\}$ , and

The far-field pattern of $u$ is given by

It is obvious that $u_\infty$ is (real) analytic in both $\hat x$ and $k$ . Hence, if $u_\infty(\hat x, k)$ is known on any open portion of $\mathbb{S}^{d-1}\times\mathbb{R}_+$ , then it is known on the whole set by analytic continuation.

The inverse source scattering problem is concerned with the recovery of $f = \varphi\chi_D$ by knowledge of $u_\infty(\hat x, k; f)$ for $(\hat x, k)\in\Sigma$ , where $\Sigma$ is an open subset of $\mathbb{S}^{d-1}\times\mathbb{R}_+$ . According to our discussion above, without loss of generality, we always assume that $\Sigma = \mathbb{S}^{d-1}\times\mathbb{R}_+$ in what follows. The inverse source problem arises in a variety of important applications including detection of hazardous chemicals, medical imaging, photoacoustic and thermoacoustic tomography, brain imaging, artificial intelligence in gesture computing and others. We refer to the recent articles ^{[25,26,39,40]} by two of the authors of this article for some recent developments on the inverse source problem.

Next, let us consider a specific case by assuming a source supported in a domain $D$ with a constant density $1$ . Then clearly by (3.6), there is a one-to-one correspondence between $D$ and $u_\infty(\Sigma; D): = \{u_\infty(\hat x, k; 1\cdot\chi_D)\}_{(\hat x, k)\in\Sigma}\in L^2(\mathbb{S}^{d-1}\times\mathbb{R}_+)$ in the sense that for two domains $D_1$ and $D_2$ ,

Based on (3.7), we next introduce

Definition 3.1. For a geometric shape $(D, \Lambda_D)\in\mathcal{S}$ ,

defined via the Helmholtz system (3.1) is called a shape generator for $D$ .

Remark 3.1. By Definition 3.1, a geometric body $D$ can be completely determined by a shape generator $u_\infty(\Sigma)$ . Since $u_\infty(\Sigma)$ is from a function space, this paves the way for the new body generation through function interpolations.

Remark 3.2. By (3.6), we know the far-field pattern is actually the Fourier transform of the source density up a dimensional constant. However, introducing the shape generator via the inverse scattering approach shall provide more physical insights in our study, and moreover it enables us to borrow ideas from the inverse scattering literature of recovering the geometric shape $D$ from the associated far-field pattern. This also paves the way of extending the idea by using other inverse scattering models that have such one-to-one correspondence between geometric shapes and far-field patterns; see more relevant discussion in section 7.

4.   Mathematical setup for the geometric body generation

In this section, we introduce the mathematical formulation of the geometric body generation for our study from a machine learning perspective. For a geometric shape $(D, \Lambda_{D})$ with the associated shape generator $u_\infty(\Sigma; D)$ , the pair of the high dimensional variables, written as $\{(\Lambda_{D}, u_\infty(\Sigma; D))\}$ , is referred to as an input-output pair. Let $\{(\Lambda_{D_i}, u_\infty(\Sigma; D_i))\}_{i\in \mathscr{N}}$ with $\mathscr{N} = \{0, 1, \dots, N_{pair}\}$ be a set of input-output pairs associated with the characteristic sets $\{\Lambda_{D_i}\}_{i\in \mathscr{N}}$ . Here the input characteristic sets $\Lambda_{D_i}$ are introduced as

with $\mathscr{C} = \{1, \dots, M\}$ . In (4.1), the notation $\lambda^{(j)}_{i}$ represents the characteristic value of the $i$ -th geometric shape $D_i$ in the $j$ -th direction of its characteristic set. Here, the cardinality $\mathscr{N}$ is finite. The training dataset of the geometric body generation is introduced to be

with

The training dataset consists of certain pre-sampled geometric shapes with statistically well selected characteristic values. The corresponding shape generator of a specific body in the training dataset can also be pre-calculated and stored. The main goal of our study is to first infer a learning model from the training dataset, $T_\mathbf{Z}: \mathbb{R}^M\rightarrow L^2(\mathbb{S}^{d-1}\times\mathbb{R}_+)$ that fulfils the following requirements:

1. It fits the training data well in the sense that

2. It can be used to infer the shape generator for a given new shape with prescribed characteristic values, namely,

and with a statically well selected training dataset, it is justifiable to expect that

where $u_\infty(\Sigma; D_{\mathrm{new}})$ is the shape generator for $D_{\mathrm{new}}$ .

If a learning model can be achieved that fulfils the two requirements as described above, then the body generation can be proceeded as follows. For a given new set of characteristic values, one first generates the learned shape generator as in (4.5). By a certain inverse scattering approach, one can then reconstruct the (approximate) shape $D_{\mathrm{new}}$ from the corresponding shape generator $\hat{u}_\infty(\Sigma; D_{\mathrm{new}})$ . In the next section, we shall develop the two critical ingredients in the body generation procedure described above, namely, the learning model and the reconstruction method. To be more definite and specific, we first introduce the following definition from a machine learning perspective.

Definition 4.1. (Body learning model) Given a training dataset

Let $\mathcal{H}$ be a compact subset of $L^2(\mathbb{S}^{d-1}\times\mathbb{R}_+)$ . $T_\mathbf{Z}\in\mathcal{H}$ (with specified coefficients $C$ ) is said to be the best fit learning model associated with the training dataset $\mathbf{Z}$ if it is the minimizer of the following optimization problem,

According to Definition 4.1, the choice of the learning subspace $\mathcal{H}$ plays a critical role. However, we note that the shape generator is actually (real) analytic in all of its arguments. Hence, instead of solving the computationally costly optimization problem (4.8), we can make use of the functional interpolation to produce a well-rounded shape learning model. This is one of the main advantages of introducing the shape generator through the inverse scattering model. In the next section, for a given training dataset as in (4.7), we shall derive a learning model using the cubic B-spline interpolation through the use of the high-dimensional data-points (4.4). For the reconstruction of the approximate body shape from the shape generator obtained through the learning model, we shall make use of a multiple-frequency Fourier method, and it can also produce an efficient and stable recovery. Throughout, we assume that the characteristic values in the training dataset is statistically well selected and it is not the focus of the present article.

5.   A scheme for geometric body generation

In this section, we develop the details of our scheme for the geometric body generation following the general discussion made in the previous section. We first derive the learning model through the functional interpolation of the high-dimensional data in the training dataset. To that end, we present some preliminary knowledge on the cubic B-spline, and we also refer to ^[2,19,36,37] for more relevant discussion on the cubic B-spline.

5.1.   Preliminary knowledge on the cubic B-spline

Consider the training dataset (4.2). Let the $M$ sets of unique grids in the directions of $\{\lambda^{(j)}\}_{j\in \mathscr{C}}$

define on the intervals $[a_j, b_j]$ as $M$ sets of points $\lambda_{g_j}^{(j)}\in[a_j, b_j]\subset \mathbb{R}$ , where $g_j\in \{0, \dots, k_j\}$ and $a_j = \lambda_{0}^{(j)} < \lambda_{1}^{(j)} < \dots < \lambda_{k_j}^{(j)} = b_j, j = 1, \dots, M$ . Here, $k_j$ is the greatest number of distinct characteristic values in the $j$ -th direction of the characteristic set. We remark that if the characteristic values of the training dataset are all collected in distinct values, then $k_j$ is actually the last index of the training dataset, $N_{pair}$ . However, the training dataset might be collected in such a way that some body shapes may possess the same characteristic value in the $j$ -th direction, and hence $k_j$ is usually smaller than $N_{pair}$ .

With the above notation, the training dataset stored as the array in (4.2) can be represented as elements on the grid mesh corresponding to the characteristic numbers $\lambda^{(j)}$ as described above. The interpolation data are the corresponding shape generators and are written as

where $g_j = 0, \dots, k_j, j = 1, \dots, M.$ In the subsequent study, we shall stick to the same notation $g_1, g_2, \dots, g_M$ to represent the linear indexing. The following example demonstrates a real application for the human body generation.

Example 5.1. The training dataset consists of 20 bodies with two characteristic values as consideration, say, height and relative weight. Here, the height and relative weight are the two directions of the grids, $\lambda^{(height)}$ and $\lambda^{(weight)}$ . Suppose the heights of the sampled bodies are given by 1.5 m, 1.6 m, 1.7 m, 1.8 m, 1.9 m and the relative weights of the sampled bodies are given by 60%, 80%, 100%, 120%. Then the first grid $\Delta_{height} = \{\lambda_0^{(height)}, \lambda_1^{(height)}, \lambda_2^{(height)}, \lambda_3^{(height)}, \lambda_4^{(height)}\} = \{1.5, 1.6, 1.7, 1.8, 1.9\}$ and the second grid $\Delta_{weight} = \{\lambda_0^{(weight)}, \lambda_1^{(weight)}, \lambda_2^{(weight)}, \lambda_3^{(weight)}\} = \{0.6, 0.8, 1, 1.2\}$ . The interpolation data are actually stored as listed in Table 1; e.g., $U_{0, 0} = u_\infty(\Sigma; D_1), U_{0, 3} = u_\infty(\Sigma; D_4), U_{1, 3} = u_\infty(\Sigma; D_{8})$ .

Let $S_3(\Delta_j), j = 1, \dots, M$ be a function subspace of $C^2([a_j, b_j])$ consisting of one dimensional, complex-valued functions in the direction of $\lambda^{(j)}, j = 1, \dots, M$ on the bounded interval $[a_j, b_j]$ . The function in $S_3(\Delta_j), j = 1, \dots, M$ is piecewise polynomial of degree 3 on every subinterval $[\lambda_{g_j-1}^{(l)}, \lambda_{g_j}^{(j)}]$ , where $g_j = 1, \dots, k_j, j = 1, \dots, M$ . Then we introduce a function subspace of multidimensional and complex-valued $C^2([a_j, b_j])$ functions as

on each rectangular grid

for all $0\leq g_j\leq k_j-1, j = 1, \dots, M$ that are piecewise polynomials of degree $3$ on every interval. For easy reference we provide the definition of B-splines.

Definition 5.1. The sets of $k_j+k, j = 1, \dots, M$ , B-spline basis functions $\{B_{l, k}(\lambda^{(j)})\}_{l = 1}^{k_j+k}$ of degree $k$ of the function space $S_k(\Delta_j)$ are defined based on concurrent boundary knots vectors with Cox-deBoor recurrence^[19],

with

for $l = 1, \dots, k_j+k$ , where $k = 3$ is for the cubic B-spline and $\lambda_l^{(j)}$ are elements of the knot vectors, satisfying the relation $\lambda_l^{(j)} < \lambda_{l+1}^{(j)}$ .

All methods are in the following using splines with a $k + 1$ regular knot vector, and the interior knots are the grid points. Based on Definition 5.1, we next introduce a general learning model for the geometric shape generation through the multidimensional cubic B-spline interpolation.

Body learning model Ⅰ. Given the training dataset $\mathbf{Z}: = \{(\Lambda_{D_i}, u_\infty(\Sigma; D_i))\}_{i\in \mathscr{N}}$ , the learning model $T_\mathbf{Z}\in S_3(\Delta_1, \dots, \Delta_M)$ at $\Lambda = (\lambda^{(1)}, \dots, \lambda^{(M)})$ for the geometric body generation associated with the sets of the grids $\{\Delta_j\}_ {j\in \mathscr{C}}$ is defined as follows

which satisfies the following conditions by (4.4)

where $c_{g_1, g_2, \dots, g_{M}}$ with $g_j = 0, \dots, k_j, j = 1, \dots, M$ are the coefficients to be determined from the training dataset $\mathbf{Z}$ , $B_{g_j, 3}^j(\lambda^{(j)}), j\in \mathscr{C}$ are the B-spline basis functions of degree 3 defined in (5.5), and $k_j, j = 1, \dots, M$ is the number of different characteristic values in each direction.

Remark 5.1. In learning model Ⅰ, (5.7) presents a general form of the learning model $T_\mathbf{Z}$ for the geometric body generation associated with the non-uniform grids $\{\Delta_j\}_ {j\in \mathscr{C}}$ . The learning model eventually generates a B-spline interpolation with the associated spacing for each segment. If the training dataset consists of equidistant grids, we can derive a faster and easier learning model and this shall be provided in the next subsection.

5.2.   Uniform B-spline

In this subsection, we derive a learning model for a special case with the training dataset consisting of equidistant grids. By (5.1), for the $M$ set of equidistant grids $\Delta_j = \{\lambda_0^{(j)}, \cdots, \lambda_{k_j}^{(j)}\}_{j\in \mathscr{C}}$ with additional conditions

the B-spline basis function $\beta^k(t)$ of degree $k$ is a symmetrical, bell-shaped function constructed from $k+1$ times self-convolution of the $\beta^0(t)$ basis function of degree zero which is a centered rectangle around origin ^[37]

The centered symmetric B-spline of degree $k$ has an explicit expression ^[36]

where the function $x_+$ is defined as follows

In this paper, we are particular intereted in the cubic B-spline. By (5.12), the closed-form representation of the cubic B-spline basis function can be also expressed as

which is used for preforming the interpolation. Then we choose the interpolation kernels to be

as the basis of $S_3(\Delta_j)$ in the $\lambda^{(j)}$ -direction such that $L = \{L_1, L_2, \dots, L_{k_j+3}\}$ is a basis of the $(k_j+3)$ -dimensional space $S_3(\Delta)$ and hence, the basis of the $\prod_{j\in\mathscr{C}}(k_j+3)$ -dimensional space $S_3(\Delta_1, \dots, \Delta_M)$ in the directions of $\lambda^{(1)}, \dots, \lambda^{(M)}$ is given by

Based on (5.16), (5.1) and (5.9), we next introduce the learning model for the uniform case.

Body learning model Ⅱ. Given the training dataset $\mathbf{Z}: = \{(\Lambda_{D_i}, u_\infty(\Sigma; D_i))\}_{i\in \mathscr{N}}$ , the learning model $T_\mathbf{Z}\in S_3(\Delta_1, \dots, \Delta_M)$ at $\Lambda = (\lambda^{(1)}, \dots, \lambda^{(M)})$ for the geometric body generation associated with the sets of equidistent grids $\{\Delta_j\}_ {j\in \mathscr{C}}$ defined in (5.9) is defined as follows

which is required to satisfy the following conditions

where $c_{g_1, g_2, \dots, g_{M}}$ with $g_j = 0, \dots, k_j, j = 1, \dots, M.$ are the coefficients to be determined from the training dataset $\mathbf{Z}$ , $L_{g_j}^j(\lambda^{(j)})$ are B-spline basis functions of degree 3 defined in (5.15) and $k_j, j = 1, \dots, M$ . is the number of different characteristic values in each direction.

5.3.   Natural spline

In learning models Ⅰ and Ⅱ, the learning functionals for the non-uniform and uniform case are respectively considered in the $\prod_{j\in \mathscr{C}}(k_j+3)$ -dimensional space $S_3(\Delta_1, \dots, \Delta_M)$ . $\prod_{j\in \mathscr{C}}(k_j+3)$ interpolation conditions are required to determine the coefficients $c_{i_1, i_2, \dots, i_{M}}$ in the training models. However, there are only $\prod_{j\in \mathscr{C}}(k_j+1)$ shape generators to specify $\prod_{j\in \mathscr{C}}(k_j+1)$ conditions in (5.8) or (5.18). To obtain a unique correlation between the characteristic values and the shape generator, we need to add $\prod_{j\in \mathscr{C}}2 = 2^{M}$ conditions, which define the second-order derivatives of the spline function at the boundary $a_j$ and $b_j$ to be equal to 0 and lead to a natural spline.

5.4.   Prediction on shape generator

With the learning models Ⅰ and Ⅱ established in the previous subsections, for an input new set of characteristic values $\Lambda_{D_{new}}$ associated with a new geometric body $D_{new}$ , the unknown shape generator can be generated as follows,

where the coefficients $c_{g_1, g_2, \dots, g_{M}}$ in (5.19) could be determined by solving the natural spline problem.

5.5.   Reconstruction

In this subsection, we briefly outline the Fourier method for the reconstruction of geometry shape $D_{{new}}$ by using the shape generators ${u}_\infty(\Sigma; D_{new})$ .

Define the periodic Sobolev space by

where $\sigma \geq 1$ , $\xi\in \mathbb{Z}^d$ and $\widehat g(\xi)$ denote the Fourier coefficients of $g$ . Suppose that $f\in H^{\sigma}(\mathbb{R}^d)$ has a compact support in domain $V_0 = (-a/2, a/2)^d, \, (a > 0)$ , then the Fourier transform of $f$ is represented by

where the overbar stands for the complex conjugate and the Fourier basis functions are given by

Definition 5.2 (Admissible wavenumbers and observation directions). Let $\mu$ be a sufficiently small positive constant such that $0 < \mu < 1$ and

then the admissible set of wavenumbers is defined by

correspondingly, the admissiable set of observation directions is given by

where $k_0 = 2\pi|\xi_0|/a$ and $\hat x_0 = \xi_0/|\xi_0|$ for $\xi = 0\in \mathbb{Z}^d$ .

Due to $\mathrm{supp}\, S \subset\subset V_0$ , for $\xi\in \mathbb{Z}^{3}\backslash \{0\}$ , the far-field pattern defined in (3.6) can be written as

where $k\in \mathbb{K}$ and $\hat x\in \mathbb{X}$ depend on $\xi$ . Combining (5.20) and (5.21), one has

For $\xi = 0$ , using the Fourier expansion of $f$ , we derive that

which implies

Therefore, the Fourier method is to approximate $f$ by a truncated Fourier expansion

where $N\in \mathbb{N}_+$ denotes the truncation order and the Fourier coefficients are given by (5.22) and (5.23). Hence the domain $D_{new}$ is determined since the set $D_{new}$ is the external shape of $f$ .

Next, we investigate the stability of the proposed Fourier method. In practical computation, there exists some noise between the shape generators $u_\infty(\Sigma; D_{{new}})$ and the predictions $\hat u_\infty(\Sigma; D_{{new}})$ , which satisfies

where $\delta > 0$ denotes the noise level. Noting that $(\hat x, k)\in \mathbb{X}\times \mathbb{K}\subset \Sigma$ , then the approximation of $f$ from predicted shape generators is given by

where

Theorem 5.1. Let $f$ be a compactly supported function in $H^{\sigma}(\mathbb{R}^d)$ , $\sigma \geq 1$ , with $\mathrm{supp} f \subset\subset V_0$ , then we have the following estimate

where $d\leq \tau\in \mathbb{R}_+$ and $C$ is a constant which depends on $f, a, d, \mu$ .

Proof. Using the Plancherel theorem, we have

where $N\in \mathbb{N}_+$ . Due to $f\in H^{\sigma}(\mathbb{R}^d)$ , that is,

It means that both $|\xi|\widehat f_{\xi}$ and $|\xi|\widehat f_{\xi}^{\delta}$ are bounded in $L^2(\mathbb{R}^d)$ , so we can find $N > 0$ , such that

where $C_1 > 0$ is a constant. For $1\leq |\xi|_{\infty}\leq N$ , from (5.22) and (5.24), we have

which implies

for $C_2 = \|f\|_{L^2(V_0)}^2/ a^d$ . Define $\xi = (\xi_{1}, \xi_2) \in \mathbb{Z}^2$ or $\xi = (\xi_{1}, \xi_2, \xi_3) \in \mathbb{Z}^3$ , by a straight forward calculation, one finds that

For $\xi = 0$ , using (5.23), (5.25), (5.27), (5.28) and the last equation, it derives that

where $C_3 = {\mu \pi\|f\|_{L^2(V_0)} }/(a^{d/2}\sin \mu \pi)$ . Hence, substituting (5.27), (5.28) and (5.29) into (5.26), it deduces that

where $C = \max\{2 C_1, 2^{d+1}C_2, C_3^2\}/ a^{2d}$ . Furthermore, if we take $N = \tau\delta^{-\frac{2}{2+d}}$ with $\tau\geq d$ in Theorem 5.1, then it holds that

Let $N = \left[\tau\delta^{-\frac{2}{2+d}}, \, \tau\geq d\right]$ , here $[X]$ denotes the largest integer that is smaller than $X + 1$ . From definition 5, the truncated wavenumbers and observation directions can be written as

Thus, the truncated Fourier expansion of $f$ from the predictions $\{\hat u_\infty(\Sigma; D_{{new}_q})\}_{q\in\mathscr{Q}}$ takes the form

where

5.6.   Summary

Motivated by above discussion, we are ready to present our novel modeling methodology for geometric shape in $\mathbb{R}^d, d = 2, 3$ , see Algorithm 1.

6.   Numerical examples

In this section, several numerical examples of geometric body generation are conducted to show that the proposed method is effective and efficient. Here the geometric body includes general geometric shape and human body shape.

The proposed algorithm is implemented by using Matlab 2016. The shape generator $\{u_\infty(\hat x, k; D_i)\}_{i\in \mathscr{N}}$ is obtained by solving the direct problem of (3.1). To avoid the inverse crime, we use the quadratic finite elements on a truncated spherical domain enclosed by a PML layer. The mesh of the forward solver is successively refined till the relative error of the successive measured scattered data is below $0.1\%$ . Then artificial shape generators are generated by applying the Kirchhoff integral formula to the scattered data. Thus the training dataset is given by

where $j = 1, 2, \cdots, (2N+1)^{d}$ and $i = 1, 2, \cdots, N_t$ denotes the $i$ -th geometry shape. In what follows, we set $\tau = 2$ ($\tau = 3$ ) for $d = 2$ ($d = 3$ ) and $\delta = 1\%$ , then we have $N = 20$ ($N = 19$ ) for $d = 2$ ($d = 3$ ).

Next, we present the implementation of interpolation. The characteristic value set is given by $\{\Lambda_{D_i}\}$ , where $\Lambda_{D_i} = \{\lambda_i^{(1)}, \lambda_i^{(2)}, \cdots, \lambda_i^{({M})}\}$ has ${M}$ variables. For a fixed wavenumber $k_j$ , we use cubic spline interpolation to obtain the coefficients $c_{i_1\dots, i_{{M}}}$ from the characteristic value $\Lambda_{D_i}$ and the shape generator $u_{\infty}(\hat{x}_j, k_j; D_i)$ . Therefore, given a new characteristic value $\Lambda_{D_{new}} = \{\lambda_{D_{new}}^{(1)}, \lambda_{D_{new}}^{(2)}, \cdots, \lambda_{D_{new}}^{(M)}\}$ , we obtain the predicted shape generators $\{\hat u_{\infty}(\hat{x}_j, k_j; D_{new})\}, \, j = 1, 2, \cdots, (2N+1)^d$ .

Finally, we specify details of recovering the geometry. As discussed above, reconstructing the geometry shape is equally to reconstructing the source function $f$ . In the discrete formula, the domain $V_0$ is divided into a uniform mesh with size $100\times 100$ in two dimensions and size $100\times 100\times 100$ in three dimensions. Further, the approximated Fourier series $f_{N}^{\delta}$ are computed at the mesh nodes $\mathcal{T}_j, \, j = 1, \cdots, 100^3$ in (5.30). Thus, the geometry shape $D_{new}$ is approximated by the boundary of the imaging results $f_{N}^{\delta}$ .

6.1.   Kite shaped experiments

In the first example, we aim to reconstruct a kite shaped domain with scale changing. The kite shaped domain is parameterized by

where $\beta_1$ and $\beta_2$ are scale factors (characteristic values) with $\beta_1, \beta_2 \in [0.5, \, 1.8]$ . The training dataset consists of $14 \times 14$ different scale domains, i.e., $\beta_1$ and $\beta_2$ uniformly distributed on $[0.5, \, 1.8]$ with $M = 14^2$ . Next, we consider four sets of different scale factors which are not covered by the training data. In this numerical experiments, the imaging results with different characteristic values are shown in Figure 1, where the black dotted lines denote the exact boundary. It is clear that the reconstructions are very closed to the exact domains.

6.2.   Rounded triangle and apple shaped experiments

In the second example, we aim to recover multi-domain with different scale factors. The apple shaped domain is parameterized by

and the rounded triangle shaped domain is parameterized by

where $\beta_1\in [1,2]$ and $\beta_2\in [0.5, 1.5]$ are scale factors (characteristic values) for different domains. The training dataset consists of $11 \times 11$ different scale domains, that is, $\beta_1$ and $\beta_2$ uniformly distributed on $[1,2]\times [0.5, 1]$ . Similarly, we give four sets of different scale factors which are not covered by the training data. Figure 2 shows the the reconstruction of multi-domain with different characteristic values via contour plots, where the black dotted lines denote the exact boundary. It demonstrates very good imaging performance of the approach.

6.3.   Rectangular solid experiments

In the third example, we verify the proposed method by using a set of artificial experiments on rectangular solid. The training dataset consists of $125$ rectangular solids with different height, width and length. Here the height, width and length are uniformly distributed on $[1,2]$ with $5$ amounts, i.e., $1, 1.25, 1.5, 1.75, 2$ . Here, We consider four sets of different height, width and length of rectangular solids which are not covered by the training data. The imaging results with different characteristic values are shown in Figure 3, where the black dotted lines denote the shadows of the exact cube boundary. Due to discontinuities of the source, there is Gibbs phenomena on the boundary of the rectangular solids. On the whole, given the characteristic values, our proposed method is valid for determining the geometry shape.

6.4.   Body experiments

In last example, we consider a challenging case and verify the proposed method by using a set of synthetic experiments on 3D human body shape. The training dataset consists of $25$ bodies which are generated by the MakeHuman soft ^[42]. MakeHuman soft is an open source software designed to create 3D virtual human models using 3D morphing technology. Providing the main parameters gender, age, muscle mass, weight, height, proportion and ethnicity, etc. of any specific character, the intermediate human shape can be reproduced with its large and own database. In this experiment, we consider two characteristic values, i.e., height and relative weight. Define the exact weight by $EW$ and standard weight by $SW$ , then the relative weight $RW$ is calculated by

Here the height of the body is given by $1.5\; m, 1.6\; m, 1.7\; m, 1.8\; m, 1.9\; m$ and the relative weight of the body is given by $60\%, 80\%, 100\%, 120\%, 140\%$ . Some human body shapes in the training dataset are presented in Figure 4. In addition, we choose two characteristic values of human body which are not covered by the training data. The first body's height is $1.55\; m$ and the relative weight is $130\%$ . The second body's height is $1.85\; m$ and the relative weight is $110\%$ . Figure 5(a) and Figure 6(a) present the exact body shape with the given characteristic value. Figure 5(b) and Figure 6(b) show the prediction of the human body shape with the given characteristic value. The results show that our method is efficient to predict the human body shape.

7.   Concluding remarks

In this paper, we develop a machine-learning method in generating a geometric body shape through prescribing a set of characteristic values of the body. The generation is mainly based on a given training dataset consisting of certain pre-selected body shapes with statistically well-sampled characteristic values. A major novelty and critical ingredient of our study is the borrowing of inverse scattering techniques in the theory of wave propagation to the geometric shape generation. We introduce the notion of shape generator which establishes a one-to-one correspondence between the geometric shape space and the function space consisting of the multiple-frequency far-field patterns associated with the time-harmonic source scattering problem. The shape generator plays an intermediate role in the geometric shape generation. First, the training dataset of geometric shapes is converted into a subset of the function space consisting of the corresponding shape generators. Then a learning model is derived through a functional interpolation of the aforementioned shape generators. For a given set of characteristic values, one then uses the learning model to obtain the shape generator of the underlying geometric body and finally reconstructs it through a multiple-frequency Fourier method.

In the current article, the shape generator is introduced through an inverse source scattering model where we make use of the one-to-one correspondence between a geometric shape and the multiple-frequency far-field pattern associated with a compactly-supported acoustic source. One may consider to introduce the shape generator through other inverse scattering models, e.g., the inverse acoustic obstacle scattering model (cf. ^[28,29,31]) or the inverse electromagnetic scattering model (cf. ^[27,30]). In doing so, one may achieve other geometric shape generation schemes that are suitable for different applications.

Acknowledgments

The work of H. Liu was supported by the FRG and startup grants from Hong Kong Baptist University, Hong Kong RGC General Research Funds, 12302017, 12301218 and 12302919.

Conflict of interest

The authors declare no conflict of interest.