Three-dimensional human pose estimation is a key technology in many computer vision tasks. Regressing a 3D pose from 2D images is a challenging task, especially for applications in natural scenes. Recovering the 3D pose from a monocular image is an ill-posed problem itself; moreover, most of the existing datasets have been captured in a laboratory environment, which means that the model trained by them cannot generalize well to in-the-wild data. In this work, we improve the 3D pose estimation performance by introducing the attention mechanism and a calibration network. The attention model will capture the channel-wise dependence, so as to enhance the depth analysis ability of the model. The multi-scale pose calibration network adaptively learns body structure and motion characteristics, and will therefore rectify the estimation results. We tested our model on the Human 3.6M dataset for quantitive evaluation, and the experimental results show the proposed methods with higher accuracy. In order to test the generalization capability for in-the-wild applications, we also report the qualitative results on the natural scene Leeds Sports Pose dataset; the visualization results show that the estimated results are more reasonable than the baseline model.
Citation: Longkui Jiang, Yuru Wang, Xinhe Ji. Calibrated deep attention model for 3D pose estimation in the wild[J]. Electronic Research Archive, 2023, 31(3): 1556-1569. doi: 10.3934/era.2023079
[1] | Janthip Jaiprasert, Pattrawut Chansangiam . Exact and least-squares solutions of a generalized Sylvester-transpose matrix equation over generalized quaternions. Electronic Research Archive, 2024, 32(4): 2789-2804. doi: 10.3934/era.2024126 |
[2] |
Jin Wang, Jun-E Feng, Hua-Lin Huang .
Solvability of the matrix equation |
[3] | Yimou Liao, Tianxiu Lu, Feng Yin . A two-step randomized Gauss-Seidel method for solving large-scale linear least squares problems. Electronic Research Archive, 2022, 30(2): 755-779. doi: 10.3934/era.2022040 |
[4] | Zehua Wang, Jinrui Guan, Ahmed Zubair . A structure-preserving doubling algorithm for the square root of regular M-matrix. Electronic Research Archive, 2024, 32(9): 5306-5320. doi: 10.3934/era.2024245 |
[5] | Hankang Ji, Yuanyuan Li, Xueying Ding, Jianquan Lu . Stability analysis of Boolean networks with Markov jump disturbances and their application in apoptosis networks. Electronic Research Archive, 2022, 30(9): 3422-3434. doi: 10.3934/era.2022174 |
[6] | Ping Yang, Xingyong Zhang . Existence of nontrivial solutions for a poly-Laplacian system involving concave-convex nonlinearities on locally finite graphs. Electronic Research Archive, 2023, 31(12): 7473-7495. doi: 10.3934/era.2023377 |
[7] | ShinJa Jeong, Mi-Young Kim . Computational aspects of the multiscale discontinuous Galerkin method for convection-diffusion-reaction problems. Electronic Research Archive, 2021, 29(2): 1991-2006. doi: 10.3934/era.2020101 |
[8] | Yang Song, Beiyan Yang, Jimin Wang . Stability analysis and security control of nonlinear singular semi-Markov jump systems. Electronic Research Archive, 2025, 33(1): 1-25. doi: 10.3934/era.2025001 |
[9] | Lot-Kei Chou, Siu-Long Lei . High dimensional Riesz space distributed-order advection-dispersion equations with ADI scheme in compression format. Electronic Research Archive, 2022, 30(4): 1463-1476. doi: 10.3934/era.2022077 |
[10] | Zhengmao Chen . A priori bounds and existence of smooth solutions to a Lp Aleksandrov problem for Codazzi tensor with log-convex measure. Electronic Research Archive, 2023, 31(2): 840-859. doi: 10.3934/era.2023042 |
Three-dimensional human pose estimation is a key technology in many computer vision tasks. Regressing a 3D pose from 2D images is a challenging task, especially for applications in natural scenes. Recovering the 3D pose from a monocular image is an ill-posed problem itself; moreover, most of the existing datasets have been captured in a laboratory environment, which means that the model trained by them cannot generalize well to in-the-wild data. In this work, we improve the 3D pose estimation performance by introducing the attention mechanism and a calibration network. The attention model will capture the channel-wise dependence, so as to enhance the depth analysis ability of the model. The multi-scale pose calibration network adaptively learns body structure and motion characteristics, and will therefore rectify the estimation results. We tested our model on the Human 3.6M dataset for quantitive evaluation, and the experimental results show the proposed methods with higher accuracy. In order to test the generalization capability for in-the-wild applications, we also report the qualitative results on the natural scene Leeds Sports Pose dataset; the visualization results show that the estimated results are more reasonable than the baseline model.
Matrix equations of the form AXB=C are important research topics in linear algebra. They are widely used in engineering and theoretical studies, such as in image and signal processing, photogrammetry and surface fitting in computer-aided geometric design [1,2]. In addition, the equation-solving problems are also applied to the numerical solutions of differential equations, signal processing, cybernetics, optimization models, solid mechanics, structural dynamics, and so on [3,4,5,6,7]. So far, there is an abundance of research results on the solutions of matrix equation AXB=C, including the existence [8], uniqueness [9], numerical solution [10], and the structure of solutions [11,12,13,14,15,16]. Moreover, [17] discusses the Hermitian and skew-Hermitian splitting iterative method for solving the equation. The authors of [18] provided the Jacobi and Gauss-Seidel type iterative method to solve the equation.
However, in practical applications, ordinary matrix multiplication can no longer meet the needs. In 2001, Cheng and Zhao constructed a semi-tensor product method, which makes the multiplication of two matrices no longer limited by dimension [19,20]. After that, the semi-tensor product began to be widely studied and discussed. It is not only applied to problems such as the permutation of high-dimensional data and algebraization of non linear robust stability control of power systems [22], it also provides a new research tool for the study of problems in Boolean networks [23], game theory [24], graph coloring [25], fuzzy control [26] and other fields [27]. However, some of these problems can be reduced to solving linear or matrix equations under the semi-tensor product. Yao et al. studied the solution of the equation under a semi-tensor product (STP equation), i.e., AX=B, in [28]. After that, the authors of [29,30,31] studied the solvability of STP equations AX2=B,A∘lX=B and AX−XB=C, respectively.
To date, the application of the STP equation AXB=C has also been reflected in many studies using the matrix semi-tensor product method. For example, in the study of multi-agent distributed cooperative control over finite fields, the authors of [32] transformed nonlinear dynamic equations over finite fields into the form of STP equation Z(t)=˜LZ(t+1), where ˜L=ˆLQM and Q is the control matrix. Thus, if we want to get the right control matrix to realize consensus regarding L, we need to solve the STP equation AXB=C. Recently, Ji et al. studied the solutions of STP equation AXB=C and gave the necessary and sufficient conditions for the equation to have a solution; they also formulated the specific solution steps in [33]. Nevertheless, the condition that STP equation AXB=C has a solution is very harsh. On the one hand, parameter matrix C needs to have a specific form; particularly, it should be a block Toeplitz matrix and, even if the matrix C meets certain conditions, the equation may not have a solution. This brings difficulties in practical applications. On the other hand, there is usually a certain error in the data that we measure, which will cause the parameter matrix C of the equation AXB=C to not achieve the required specific form; furthermore, the equation at this time will have no exact solutions.
Therefore, this paper describes a study of the approximate solutions of STP equation AXB=C. The main contributions of this paper are as follows: (1) The least-squares (LS) solution of STP equation AXB=C is discussed for the first time. Compared with the existing results indicating that the equation has a solution, it is more general and greatly reduces the requirement of matrix form. (2) On the basis of Moore-Penrose generalized inverse operation and matrix differentiation, the specific forms of the LS solutions under the conditions of the matrix-vector equation and matrix equation are derived.
The paper is organized as follows. First, we study the LS solution problem of the matrix-vector STP equation AXB=C, together with a specific form of the LS solutions, where X is an unknown vector. Then, we study the LS solution problem when X is an unknown matrix and give the concrete form of the LS solutions. In addition, several simple numerical examples are given for each case to verify the feasibility of the theoretical results.
This study applies the following notations.
● R: the real number field;
● Rn: the set of n-dimensional vectors over R;
● Rm×n: the set of m×n matrices over R;
● AT: the transpose of matrix A;
● ∥A∥: the Frobenius norm of matrix A;
● tr(A): the trace of matrix A;
● A+: the Moore-Penrose generalized inverse of matrix A;
● lcm{m,n}: the least common multiple of positive integers m and n;
● gcd{m,n}: the greatest common divisor of positive integers m and n;
● ab: the formula b divides a;
● a∣b: b is divisible by a;
● ∂f(x)∂x: differentiation of f(x) with respect to x.
Let A=[aij]∈Rm×n and B=[bij]∈Rp×q. We give the following definitions:
Definition 2.1. [34] The Kronecker product A⊗B is defined as follows:
A⊗B=[a11Ba21B…a1nBa21Ba22B…a2nB⋮⋮⋱⋮am1Bam2B…amnB]∈Rmp×nq. | (2.1) |
Definition 2.2. [20] The left semi-tensor product A⋉B is defined as follows:
A⋉B=(A⊗It/n)(B⊗It/p)∈R(mt/n)×(qt/p), | (2.2) |
where t=lcm{n,p}.
Definition 2.3. [21] For a matrix A∈Rm×n, the mn column vector Vc(A) is defined as follows:
Vc(A)=[a11⋯am1a12⋯am2⋯a1n⋯amn]T. | (2.3) |
Proposition 2.1. [33,34] When A,B are two real-valued matrices and X is an unknown variable matrix, we have the following results about matrix differentiation:
∂tr(AX)∂X=AT,∂tr(XTA)∂X=A,∂tr(XTAX)∂X=(A+AT)X. |
In this subsection, we will consider the following matrix-vector STP equation:
AXB=C, | (2.4) |
where A∈Rm×n,B∈Rr×l, C∈Rh×k are given matrices, and X∈Rp is the vector that needs to be solved.
With regard to the requirements of the dimensionality of the matrices in the STP equation (2.4), we have the following properties:
Proposition 2.2. [33] For matrix-vector STP equation (2.4),
1) when m=h, the necessary conditions for (2.4) with vector solutions of size p are that kl,nr should be positive integers and kl∣nr,p=lnrk;
2) when m≠h, the necessary conditions for (2.4) with vector solutions of size p are that hm,kl should be positive integers and β=gcd{hm,r},gcd{kl,β}=1,gcd{hm,kl}=1 and p=nhlmrk hold.
Remark: When Proposition 2.2 is satisfied, matrices A,B, and C are said to be compatible, and the sizes of X are called permissible sizes.
Example 2.1 Consider matrix-vector STP equation AXB=C with the following coefficients:
A=[1201], B=[01], C=[111020]. |
It is easy to see that m=1,n=4,r=2,l=1,h=2, and k=3. Although m∣h,l∣k,β=gcd{hm,r},gcd{kl,β}=1, and gcd{hm,kl}=1, nbak is not a positive integer. So, A,B, and C are not compatible. At this time, matrix-vector STP equation (2.4) has no solution.
For the case that m=h, let X=[x1x2⋯xp]T∈Rp, As be the s-th column of A, and ˇA1,ˇA2,⋯,ˇAp∈Rm×np=Rm×rkl be the p equal block of the matrix A, i.e., A=[ˇA1ˇA2⋯ˇAp], and
ˇAi=[A(i−1)rkl+1A(i−1)rkl+2⋯Airkl],i=1,⋯,p. |
Let t1=lcm{n,p},t2=lcm{t1p,r}; comparing the relationship of dimensions, we can get that t1=n and t2=rkl. Then
A⋉X⋉B=(A⊗It1n)(X⊗It1p)⋉B=[ˇA1ˇA2⋯ˇAp]⋉[x1x2⋮xp]⋉B=(x1ˇA1+x2ˇA2+⋯+xpˇAp)⋉B=x1ˇA1⋉B+x2ˇA2⋉B+⋯+xpˇAp⋉B=x1(ˇA1⊗It2lrk)(B⊗It2r)+x2(ˇA2⊗It2lrk)(B⊗It2r)+⋯+xp(ˇAp⊗It2lrk)(B⊗It2r)=x1ˇA1(B⊗Ikl)+x2ˇA2(B⊗Ikl)+⋯+xpˇAp(B⊗Ikl)=C∈Rm×k. |
Denote
ˇBi=ˇAi(B⊗Ikl)=[A(i−1)rkl+1A(i−1)rkl+2⋯Airkl](B⊗Ikl)=[A(i−1)rkl+1⋯A((ji−1)r+1)kl](B1⊗Ikl)+⋯+[A(ir−1)kl+1⋯Airkl](Bh⊗Ikl)∈Rm×k,i=1,⋯,p. |
It is easy to see that when the matrices A and C have the same row dimension, the STP equation (2.4) has a better representation.
Proposition 2.3. Matrix-vector STP equation (2.4), given m=h, can be rewritten as follows:
x1ˇB1+x2ˇB2+⋯+xpˇBp=C. | (2.5) |
Obviously, it can also have the following form:
[ˇB1,jˇB2,j⋯ˇBp,j]X=Cj,i=1,⋯,p,j=1,⋯,k, |
and ˇBi,j is the j-th column of ˇBi.
At the same time, applying the Vc operator to both sides of (2.5) yields
xlVc(ˇB1)+x2Vc(ˇB2)+⋯+xpVc(ˇBp)=[Vc(ˇB1)Vc(ˇB2)⋯Vc(ˇBp)]X=Vc(C). |
We get the following proposition.
Proposition 2.4. When m=h, matrix-vector STP equation (2.4) is equivalent to the linear form equation under the traditional matrix product:
¯BX=Vc(C), |
where
¯B=[Vc(ˇB1)Vc(ˇB2)⋯Vc(ˇBp)]=[ˇB1,1ˇB2,1⋯ˇBp,1ˇB1,2ˇB2,2⋯ˇBp,2⋮⋮⋱⋮ˇB1,kˇB2,k⋯ˇBp,k]. | (2.6) |
In this subsection, we will consider the following matrix STP equation:
AXB=C, | (2.7) |
where A∈Rm×n,B∈Rr×l,C∈Rh×k are given matrices, and X∈Rp×q is the matrix that needs to be solved.
For matrix STP equation (2.7), the dimensionality of its matrices has the following requirements:
Proposition 2.5. [33] For matrix STP equation (2.7),
1) when m=h, the necessary conditions for (2.7) with a matrix solution with size p×q are that kl,nr should be positive integers and p=nα,q=rklα, where α is a common factor of n and rkl;
2) when m≠h, the necessary conditions for (2.7) with a matrix solution of size p×q are that hm,kl should be positive integers, gcd{hmβ,αβ}=1,gcd{hm,kl}=1,β∣r,p=nhmα,q=rklα, α is the common factor of nhm and rkl, and β=gcd{hm,α}.
Remark: When Proposition 2.5 is satisfied, matrices A,B, and C are said to be compatible, and the sizes of X are called permissible sizes.
Example 2.2 Consider matrix STP equation AXB=C with the following coefficients:
A=[10110−100], B=[21], C=[315020]. |
We see that m=2,n=4,r=2,l=1,h=2, and k=3, so A,B and C are compatible. At this time, matrix STP equation (2.7) may have a solution X∈R2×3 or R4×6. (In fact, by Corollary 4.1 of [33], this equation has no solution.)
When m=h, let As be the s-th column of A and denote ˇA1,ˇA2,⋯,ˇAp∈Rm×α as p blocks of A with the same size, i.e., A=[˜A1ˇA2⋯ˇAp], where
ˇAi=[A(i−1)α+1A(i−1)α+2⋯Aiα],i=1,⋯,p. |
Denote
ˉA=[Vc(ˇA1),Vc(ˇA2),⋯,Vc(ˇA2)]=[A1Aα+1⋯A(p−1)α+1A2Aα+2⋯A(p−1)α+2⋮⋮⋱⋮AαA2α⋯Apα], |
we will have the following proposition.
Proposition 2.6. [33] When m=h, STP equation (2.7) can be rewritten as follows:
(BT⊗Ikml)(Iq⊗ˉA)Vc(X)=Vc(C). | (2.8) |
In this subsection we will consider the LS solutions of the following matrix-vector STP equation:
AXB=C, | (3.1) |
where A∈Rm×n,B∈Rr×l,C∈Rm×k are given matrices, and X∈Rp is the vector that needs to be solved. By Proposition 2.2, we know that when k∣l,n∣r, and kl∣nr, all matrices are compatible. At this time, the matrix-vector STP equation (3.1) may have solutions in Rlnrk.
Now, assuming that k∣l,n∣r, and kl∣nr hold and we want to find the LS solutions of matrix-vector STP equation (3.1) on Rlnrk, that is, given A∈Rm×n,B∈Rr×l, and C∈Rm×k, we want to find X∗∈Rlnrk such that
∥A⋉X∗⋉B−C∥2=minX∈Rlnrk∥A⋉X⋉B−C∥2. | (3.2) |
According to Proposition 2.3, matrix-vector equation (2.4) under the condition that m=h can be rewritten in the column form as follows:
[ˇB1,jˇB2,k+j⋯ˇBp,(p−1)k+j]X=Cj,j=1,⋯,k. |
So, we have
∥A⋉X⋉B−C∥2=kΣj=1∥[ˇB1,jˇB2,k+j⋯ˇBp,(p−1)k+j]X−Cj∥2=kΣj=1tr[([ˇB1,jˇB2,k+j⋯ˇBp,(p−1)k+j]X−Cj)T([ˇB1,jˇB2,k+j⋯ˇBp,(p−1)k+j]X−Cj)]=kΣj=1tr(XT[ˇB1,jˇB2,k+j⋯ˇBp,(p−1)k+j]T[ˇB1,jˇB2,k+j⋯ˇBp,(p−1)k+j]X−XT[ˇB1,jˇB2,k+j⋯ˇBp,(p−1)k+j]TCj−CTj[ˇB1,jˇB2,k+j⋯ˇBp,(p−1)k+j]X+CTjCj). |
Since ∥A⋉X⋉B−C∥2 is a smooth function for the variables of X, X∗ is the minimum point if and only if X∗ satisfies the following equation:
ddX∥A⋉X⋉B−C∥2=0. |
Then, we derive the following:
ddX∥A⋉X⋉B−C∥2=kΣj=1(2[ˇB1,j⋯ˇBp,(p−1)k+j][ˇB1,j⋯ˇBp,(p−1)k+j]TX−2[ˇB1,j⋯ˇBp,(p−1)k+j]TCj). |
Taking
ddX∥A⋉X⋉B−C∥2=0, |
we have
kΣj=1[ˇB1,j⋯ˇBp,(p−1)k+j]T[ˇB1,j⋯ˇBp,(p−1)k+j]X=kΣj=1[ˇB1,j⋯ˇBp,(p−1)k+j]TCj. | (3.3) |
Hence, the minimum point of linear equation (3.2) is given by
X∗=(kΣj=1[ˇB1,j⋯ˇBp,(p−1)k+j]T[ˇB1,j⋯ˇBp,(p−1)k+j])+⋅(kΣj=1[ˇB1,j⋯ˇBp,(p−1)k+j]TCj). |
And, it is also the LS solution of (3.1).
Meanwhile, we can draw the following result:
Theorem 3.1. If ˇB1,ˇB2,⋯,ˇBp are linearly independent and ¯B of (2.6) is full rank, then the LS solution of matrix-vector STP equation (3.1) is given by
X∗=(¯BT¯B)−1¯BTVc(C); |
If ˇB1,ˇB2,⋯,ˇBp are linearly related and ¯B is not full rank, then the LS solution of matrix-vector STP equation (3.1) is given by
X∗=(¯BT¯B)+¯BTVc(C). |
Proof. According to Proposition 2.4, (3.1) is equals to the following system of linear equations with a traditional matrix product
¯BX=Vc(C). | (3.4) |
Therefore, we only need to study the LS solutions of matrix-vector STP equation (3.4). From the conclusion in linear algebra, the LS solutions of matrix-vector STP equation (3.4) must satisfy the following equation:
¯BT¯BX=¯BTVc(C). | (3.5) |
When ¯B is full rank, ¯BT¯B is invertible and the LS solution of matrix-vector STP equation (3.4) is given by
X∗=(¯BT¯B)−1¯BTVc(C); |
When ¯B is not full rank, ¯BT¯B is nonsingular and the LS solution of matrix-vector STP equation (3.4) is given by
X∗=(¯BT¯B)+¯BTVc(C). |
Comparing (3.3) and (3.5), we can see that
¯BT¯B=kΣj=1[ˇB1,jˇB2,k+j⋯ˇBp,(p−1)k+j]T[ˇB1,jˇB2,k+j⋯ˇBp,(p−1)k+j],¯BTVc(C)=kΣj=1([ˇB1,jˇB2,k+j⋯ˇBp,(p−1)k+j]TCj, |
and
∥¯BX−Vc(C)∥2=kΣj=1∥[ˇB1,jˇB2,k+j⋯ˇBp,(p−1)k+j]X−Cj∥2. |
Therefore, the two equations are the same, and the LS solution obtained via the two methods is consistent. Obviously, the second method is easier to employ. Below, we only use the second method to find the LS solutions.
Example 3.1 Now, we shall explore the LS solution of the matrix-vector STP equation AXB=C with the following coefficients:
A=[10110−1001001],B=[21200−1],C=[1110201−10]. |
By Example 2.1(1), it follows that the matrix-vector STP equation AXB=C has no exact solution. Then, we can investigate the LS solutions of this equation.
First, because A,B, and C are compatible, the matrix-vector equation may have LS solutions on R2. Second, divide A into 2 blocks; we have
ˇA1=[100−110],ˇA2=[110001],ˇB1=ˇA1(B⊗I1)=[212001212],ˇB2=ˇA2(B⊗I1)=[21100000−1]. |
Then, we can get
¯B=[22002011001021102−1],Vc(C)=[101102−110]. |
Because ¯B is full rank, the LS solution of this matrix-vector STP equation is given by
X∗=(¯BT¯B)−1¯BTVc(C)=[0.29630.0741]. |
In this subsection we will explore the LS solutions of the following matrix-vector STP equation:
AXB=C, | (3.6) |
where A∈Rm×n,B∈Rr×l and C∈Rh×k are given matrices and X∈Rp is the vector that needs to be solved. By Proposition 2.2, we know that, when m|h,k|l,nl∣rk,β=gcd{hm,r},gcd{kl,β}=1, and gcd{hm,kl}=1, A,B, and C are compatible. At this time, STP equation (3.6) may have a solution belonging to Rnhlmrk.
In what follows, we assume that matrix-vector STP equation (3.6) always satisfies the compatibility conditions, and we will find the LS solutions of matrix-vector STP equation (3.6) on Rnhlmrk. Since hm is a factor of the dimension nhlmrk of X, it is easy to obtain the matrix-vector STP equation given by
A⋉X⋉B=(A⊗Ihm)⋉X⋉B, |
according to the multiplication rules of semi-tensor products. Let A′=A⊗Ihm; then matrix-vector STP equation (3.6) is transformed into the case of m=h, and, from the conclusion of the previous section, one can easily obtain the LS solution of matrix-vector STP equation (3.6).
Below, we give an algorithm for finding the LS solutions of matrix-vector STP equation (3.6):
● Step one: Check whether A,B, and C are compatible, that is, whether m|h andk|l hold, and whether gcd{hm,kl}=1. If not, we can get that the equation has no solution.
● Step two: Let X∈Rp,p=nhlmrk, and A′=A⊗Ihm∈Rh×nhm. Take ˇA′1,ˇA′2,⋯,ˇA′p∈Rm×nhmp=Rm×rkl to have p equal blocks of the matrix A′:
ˇA′i=[A′(i−1)rkl+1A′(i−1)rkl+2⋯A′irkl],i=1,⋯,p, |
A′s is the s-th column of A′. Let
ˇB′1,ˇB′2,⋯,ˇB′p∈Rm×k, |
where
ˇB′i=ˇA′i(B⊗Ikl)=[A′(i−1)rkl+1A′(i−1)rkl+2⋯A′irkl](B⊗Ikl),i=1,⋯,p. |
● Step three: Let
¯B′=[ˇB1,1ˇB2,1⋯ˇBp,1ˇB1,2ˇB2,2⋯ˇBp,2⋮⋮⋱⋮ˇB1,kˇB2,k⋯ˇBp,k], |
and calculate Vc(C).
● Step four: Solve the equation ¯B′T¯B′X=¯B′TVc(C); if ¯B′ is full rank and ¯B′T¯B′ is reversible, at this time, the LS solution of matrix-vector STP equation (3.6) is given by
X∗=(¯B′T¯B′)−1¯B′TVc(C); |
If ¯B′ is not full rank, the LS solution of matrix-vector STP equation (3.6) is given by
X∗=(¯B′T¯B′)+¯B′TVc(C). |
Example 3.2 Now, we shall explore the LS solutions of the matrix-vector STP equation AXB=C with the following coefficients:
A=[1011],B=[20],C=[110210]. |
According to Example 2.1(2), we know that this matrix-vector STP equation has no exact solution. Then, we can investigate the LS solutions of this STP equation.
Step one: m|h,k|l,gcd{hm, and kl}=1, so A,B, and C are compatible; we proceed to the second step.
Step two: The matrix-vector STP equation may have an LS solution X∈R3, and
A′=A⊗I3=[100000100100010000010010001000001001]. |
Let
ˇA′1=[100001000010],ˇA′2=[001000010000],ˇA′3=[010000101001], |
be three equal blocks of the matrix A′. We have
ˇB′1=ˇA′1(B⊗I2)=[200200],ˇB′2=ˇA′2(B⊗I2)=[000000],ˇB′3=ˇA′3(B⊗I2)=[020020]. |
Step three: Let
¯B′=[200000002002200000],Vc(C)=[101120]. |
Step four: Because ¯B′ is not full rank, the LS solution of this matrix-vector STP equation is given by
X∗=(¯B′T¯B′)+¯B′TVc(C)=[0.750000.5000]. |
In this subsection we will explore the LS solutions of the following matrix STP equation
AXB=C, | (4.1) |
where A∈Rm×n,B∈Rr×l, and C∈Rm×k are given matrices and X∈Rp×q is the matrix that needs to be solved. By Proposition 2.5, we have that, when l∣k, all matrices are compatible. At this time, matrix STP equation (4.1) may have solutions in Rnα×rklα, and α is a common factor of n and rkl.
Now, we assume that l∣k, and we want to find the LS solutions of matrix STP equation (4.1) on Rnα×rklα; the problem is as follows: Given A∈Rm×n,B∈Rr×l, and C∈Rm×k, we want to find X∗∈Rp×q such that
∥A⋉X∗⋉B−C∥2=minX∈Rp×q∥A⋉X⋉B−C∥2. | (4.2) |
By Proposition 2.6, matrix STP equation (4.1) can be rewritten as
(BT⊗Ikml)(Iq⊗ˉA)Vc(X)=Vc(C). | (4.3) |
So, finding the LS solution of (4.1) is equivalent to finding X∗∈Rp×q such that
∥(BT⊗Ikml)(Iq⊗ˉA)Vc(X)−Vc(C)∥2=minX∈Rp×q∥(BT⊗Ikml)(Iq⊗ˉA)Vc(X∗)−Vc(C)∥2. | (4.4) |
Then, we have the following theorem.
Theorem 4.1. When B″A″=(BT⊗Ikml)(Iq⊗ˉA) is full rank and B″A″ is invertible, the LS solution of matrix STP equation (4.1) is given by
Vc(X∗)=(B″A″)+C″; |
When B″A″ is not full rank and B″A″ is nonsingular, the LS solution of matrix STP equation (4.1) is given by
Vc(X∗)=(B″A″)−1C″. |
Proof. Let B″=BT⊗Ikml,A″=Iq⊗ˉA,X″=Vc(X), and C″=Vc(C); then (4.4) can be rewritten as
∥(BT⊗Ikml)(Iq⊗ˉA)Vc(X)−Vc(C)∥2=∥B″A″X″−C″∥2=tr[(B″A″X″−C″)T(B″A″X″−C″)]=tr[(X″TA″TB″T−C″T)(B″A″X″−C″)]=tr[(X″TA″TB″TB″A″X″)−(X″TA″TB″TC″)−(C″TB″A″X″)+(C″TC″)]. |
Because ∥A⋉X⋉B−C∥2 is a smooth function for the variables of X, it follows that X∗ is the minimum point if and only if X∗ satisfies
ddX∥(BT⊗Ikml)(Iq⊗ˉA)Vc(X)−Vc(C)∥2=0. |
Given that
ddX∥(BT⊗Ikml)(Iq⊗ˉA)Vc(X)−Vc(C)∥2=2A″TB″TB″A″X″−2A″TB″TC″, |
let
ddX∥(BT⊗Ikml)(Iq⊗ˉA)Vc(X)−Vc(C)∥2=0. |
Then, we have
A″TB″TB″A″X″=A″TB″TC″. |
Thus, the minimum point of linear equation (4.2) is also the LS solution of matrix STP equation (4.1). That is to say, ∥A⋉X⋉B−C∥2 is the smallest if and only if ∥(BT⊗Ikml)(Iq⊗ˉA)Vc(X∗)−Vc(C)∥2 gets the minimum value. And, the statement is naturally proven.
Now, we shall examine the relationship between the LS solutions of different compatible sizes. Let p1×q1,p2×q2 be two different compatible sizes and 1<p1q1=p2q2∈Z. If X∈Rp1×q1, we should have that X⊗Ip2p1∈Rp2×q2; we can get the following formula:
minX∈Rp2×q2∥A⋉X⋉B−C∥2≤minX∈Rp1×q1∥A⋉X⋉B−C∥2. |
Therefore, if we consider (4.1) to take the LS solutions among all compatible sizes of matrices, then it should be the LS solutions of the equation on Rn×k.
Example 4.1 Now, we shall explore the LS solutions of matrix STP equation AXB=C with the following coefficients:
A=[10110−100],B=[21],C=[315020]. |
Example 2.2(1) shows that the matrix STP equation AXB=C has no exact solution. Now, we can investigate the LS solutions of this equation.
First, given that A,B, and C are compatible, the matrix STP equation may have LS solutions on R2×3 or R4×6.
(1) The case that α=2,X∈R2×3:
Let
ˇA1=[A1A2]=[100−1],ˇA2=[A3A4]=[1100]. |
Then, we have
ˉA=[110001−10],A″=I3⊗ˉA=[110000000000010000−10000000110000000000010000−10000000110000000000010000−10]. |
Let
B″=BT⊗I6=[200000100000020000010000002000001000000200000100000020000010000002000001],C″=Vc(C)=[315020]. |
Because B″A″ is full rank, the LS solution of this matrix STP equation satisfies
Vc(X∗)=(B″A″)−1C″=[01.1667−1.00000.666702.6667],thenX∗=[01.1667−1.00000.666702.6667]. |
(2) The case that α=1,X∈R4×6:
Let
ˉA=[10110−100], |
A″=I6⊗ˉA=[100000000000100000100000010000000000010000010000001000000000001000001000000100000000000100000100000010000000000010000010000001000000000001000001000000−1000000000000000000000000−1000000000000000000000000−1000000000000000000000000−1000000000000000000000000−1000000000000000000000000−1000000000000]. |
Let
B″=BT⊗I6=[200000100000020000010000002000001000000200000100000020000010000002000001],C″=Vc(C)=[315020]. |
Because B″A″ is full rank, the LS solution of this matrix STP equation satisfies
Vc(X∗)=(B″A″)+C″=[0.46150.15380.769200.30770−0.2308−0.0769−0.38460−0.153800.46150.15380.769200.307700.46150.15380.769200.30770],thenX∗=[0.46150.15380.769200.30770−0.2308−0.0769−0.38460−0.153800.46150.15380.769200.307700.46150.15380.769200.30770]. |
This section focuses on the LS solutions of the following matrix STP equation:
AXB=C, | (4.5) |
where A∈Rm×n,B∈Rr×l and C∈Rh×k are given matrices and X∈Rp×q is the matrix that needs to be solved. By Proposition 2.5, we have that, when m|h,l|k,gcd{hmβ,αβ}=1,gcd{β,kl}=1, and β|r, where β=gcd{hm,α}, all matrices are compatible. At this time, the matrix-vector equation (4.5) may have solutions in Rnhmα×rklα and α is a common factor of nhm and rkl.
Now, we assume that matrix STP equation (4.5) always satisfies the compatibility conditions. Since hm is a factor of the row dimension nhmα of X, it is easy to obtain the matrix STP equation
A⋉X⋉B=(A⊗Ihm)⋉X⋉B, |
according to the multiplication rules of STP. Let A′=A⊗Ihm: then (4.5) can be transformed into the case of m=h, and we can easily obtain the LS solution of matrix STP equation (4.5).
Below, we give an algorithm for finding the LS solutions of matrix STP equation (4.5):
● Step one: Check whether m|h and l|k hold. If not, we can get that the equation has no solution.
● Step two: Find all values of α that satisfy that gcd{rm,h}=1,gcd{hmβ,αβ}=1,gcd{β,kl}=1, and β|r,β=gcd{hm,α}; correspondingly, find all compatible sizes p×q and perform the following steps for each compatible size.
● Step three: Let A′=A⊗Ihm∈Rh×nhm. We have
¯A′=[Vc(ˇA′1),Vc(ˇA′2),⋯,Vc(ˇA′2)]=[A′1A′α+1⋯A′(p−1)α+1A′2A′α+2⋯A′(p−1)α+2⋮⋮⋱⋮A′αA′2α⋯A′pα], |
where ˇA′1,ˇA′2,⋯,ˇA′p∈Rm×α are p blocks of A′ of the same size, and A′i is the i-th column of A′. Let B″=BT⊗Ikhl,A″=Iq⊗¯A′,X″=Vc(X), and C″=Vc(C).
● Step four: Solve the equation A″TB″TB″A″X″=A″TB″TC″; if B″A″ is full rank and B″A″ is reversible, the LS solution of matrix STP equation (4.5) is given by
Vc(X∗)=(B″A″)−1C″; |
if B″A″ is not full rank, the LS solution of matrix STP equation (4.5) is given by
Vc(X∗)=(B″A″)+C″. |
Example 4.2 Now, we shall explore the LS solutions of matrix STP equation AXB=C with the following coefficients:
A=[10],B=[1010−1010100−1],C=[315020]. |
According to Example 2.2(2), matrix STP equation AXB=C has no exact solution. Now, we can investigate the LS solutions of this equation:
Step one: m|r and l|k hold.
Step two: gcd{rm,h}=1,gcd{hmβ,αβ}=1,gcd{β,kl}=1,β|r, and β=gcd{hm,α} hold. The matrix STP equation AXB=C may have the solution X∈R2×2 or R4×4.
Step three: (1) The case that α=2:
Let
A′=A⊗I2=[10000100]. |
Then, we have
¯A′=[Vc(ˇA′1),Vc(ˇA′2)]=[10000010]. |
Let
B″=BT⊗I2=[100010000100010000−100000000−10000100010−100100010−1],A″=I2⊗¯A′=[10000000000010000010000000000010],C″=Vc(C)=[315020]. |
(2) The case that α=1:
Let
A′=A⊗I2=[10000100]. |
Then, we have
¯A′=[Vc(ˇA′1),Vc(ˇA′2),Vc(ˇA′3),Vc(ˇA′4)]=[10000100]. |
Let
B″=BT⊗I2=[100010000100010000−100000000−10000100010−100100010−1],A″=I4⊗¯A′=[1000000000000000000001000000000000000000001000000000000000000001000000000000000000001000000000000000000001000000000000000000001000000000000000000001000000000000],C″=Vc(C)=[315020]. |
Step four: Because B″A″ is not full rank, the LS solution of this matrix STP equation is obtained as follows:
(1) The case that α=2:
Vc(X∗)=(B″A″)+C″=[1.000001.00000]⟹X∗=[1.000001.00000]. |
(2) The case that α=1:
Vc(X∗)=(B″A″)+C″=[1.50000.5000−5.000001.50000.50001.00001.000000000000]⟹X∗=[1.50000.5000−5.000001.50000.50001.00001.000000000000]. |
In this paper, we applied the semi-tensor product to solve the matrix equation AXB=C and studied the LS solutions of the matrix equation under the semi-tensor product. By appling the definition of semi-tensor products, the equation can be transformed into the matrix equation under the ordinary matrix product and then combined with the Moore-Penrose generalized inverse operation and matrix differentiation. The specific forms of the LS solutions under the conditions of the matrix-vector equation and matrix equation were also respectively derived. Investigating the solution of Sylvester equations under a semi-tensor product, as well as the LS solution problem, will be future research work.
No artificial intelligence tools were usded in the creation of this article.
The work was supported in part by the National Natural Science Foundation (NNSF) of China under Grant 12301573 and in part by the Natural Science Foundation of Shandong under grant ZR2022QA095.
No potential conflict of interest was reported by the author.
[1] | J. Wu, T. Xue, J. J. Lim, Y. Tian, J. B. Tenenbaum, A. Torralba, et al., Single image 3D interpreter network, in European Conference on Computer Vision, (2016), 365–382. https://doi.org/10.1007/978-3-319-46466-4_22 |
[2] | C. H. Chen, D. Ramanan, 3D human pose estimation = 2D pose estimation+ matching, in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2017), 5759–5767. https://doi.org/10.1109/CVPR.2017.610 |
[3] | D. Mehta, H. Rhodin, D. Casas, P. Fua, O. Sotnychenko, W. Xu, et al., Monocular 3D human pose estimation in the wild using improved CNN supervision, in 2017 International Conference on 3D Vision (3DV), (2017), 506–516. https://doi.org/10.1109/3DV.2017.00064 |
[4] | D. Pavllo, C. Feichtenhofer, D. Grangier, M. Auli, 3D human pose estimation in video with tem- poral convolutions and semi-supervised training, in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2019), 7753–7762. https://doi.org/10.1109/CVPR.2019.00794 |
[5] | Y. Cheng, B. Yang, B. Wang, W. Yan, R. Tan, Occlusion-aware networks for 3D human pose estima- tion in video, in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), (2019), 723–732. https://doi.org/10.1109/ICCV.2019.00081 |
[6] | G. Moon, J. Y. Chang, K. M. Lee, Camera distance-aware top-down approach for 3D multi- person pose estimation from a single RGB image, in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), (2019). https://doi.org/10.1109/ICCV.2019.01023 |
[7] | I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. WardeFarley, S. Ozair, et al., Generative adversarial nets, in Proceedings of the 27th International Conference on Neural Information Processing Systems (NIPS), 2 (2014), 2672-2680. |
[8] | W. Yang, W. Ouyang, X. Wang, J. Ren, H. Li, X. Wang, 3D human pose estimation in the wild by adversarial learning, in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, (2018), 5255–5264. https://doi.org/10.1109/CVPR.2018.00551 |
[9] | B. Wandt, B. Rosenhahn, RepNet: Weakly supervised training of an adversarial reprojection network for 3D human pose estimation, in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2019), 7774–7783. https://doi.org/10.1109/CVPR.2019.00797 |
[10] | B. Wandt, M. Rudolph, P. Zell, H. Rhodin, B. Rosenhahn, CanonPose: Self-supervised monocular 3D human pose estimation in the wild, in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2021), 13289–13299. https://doi.org/10.1109/CVPR46437.2021.01309 |
[11] | S. Amin, M. Andriluka, M. Rohrbach, B. Schiele, Multi-view pictorial structures for 3D human pose estimation, in British Machine Vision Conference, (2013), 1–12. |
[12] | H. Qiu, C. Wang, J. Wang, N. Wang, W. Zeng, Cross view fusion for 3D human pose estimation. in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), (2019), 4342–4351. https://doi.org/10.1109/ICCV.2019.00444 |
[13] | H. Ci, C. Wang, X. Ma, Y. Wang, Optimizing network structure for 3D human pose estimation, in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), (2019), 2262–2271. https://doi.org/10.1109/ICCV.2019.00235 |
[14] | X. Ma, J. Su, C. Wang, H. Ci, Y. Wang, Context modeling in 3D human pose estimation: A unified perspective, in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2021), 6234–6243. https://doi.org/10.1109/CVPR46437.2021.00617 |
[15] | X. Zhou, Q. Huang, X. Sun, X. Xue, Y. Wei, Towards 3D human pose estimation in the wild: A weakly-supervised approach, in 2017 IEEE International Conference on Computer Vision (ICCV), (2017), 398–407. https://doi.org/10.1109/ICCV.2017.51 |
[16] |
Q. Zhou, B. Zhong, X. Liu, R. Ji, Attention-based neural architecture search for person re- identification, IEEE Trans. Neural Networks Learn. Syst., 33 (2022), 6627–6639. https://doi.org/10.1109/TNNLS.2021.3082701 doi: 10.1109/TNNLS.2021.3082701
![]() |
[17] | H. Guo, Z. Ren, Y. Wu, G. Hua, Q. Ji, Uncertainty-based spatial-temporal attention for online action detection, in European Conference on Computer Vision (ECCV), (2022), 69–86. https://doi.org/10.1007/978-3-031-19772-7_5 |
[18] | X. Chu, W. Yang, W. Ouyang, C. Ma, A. L Yuille, X. Wang, Multi-context attention for human pose estimation, in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2017), 5669–5678. https://doi.org/10.1109/CVPR.2017.601 |
[19] | K. Su, D. Yu, Z. Xu, X. Geng, C. Wang, Multi-person pose estimation with enhanced channel-wise and spatial information, in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2019), 5667–5675. https://doi.org/10.1109/CVPR.2019.00582 |
[20] | Z. Cui, T. Song, Y. Wang, Q. Ji, Knowledge augmented deep neural networks for joint facial expression and action unit recognition, in Proceedings of the 34th International Conference on Neural Information Processing Systems, (2020), 14338–14349. |
[21] | Z. Cui, P. Kapanipathi, K. Talamadupula, T. Gao, Q. Ji, Type-augmented relation prediction in knowledge graphs, in Proceedings of the AAAI Conference on Artificial Intelligence, 35 (2021), 7151–7159. https://doi.org/10.1609/aaai.v35i8.16879 |
[22] | M. R. Ronchi, O. M. Aodha, R. Eng, P. Perona, It's all relative: Monocular 3D Human Pose Estimation from weakly supervised data, preprint, arXiv: 1805.06880. |
[23] |
V. Belagiannis, S. Amin, M. Andriluka, B. Schiele, N. Navab, S. Ilic, 3D pictorial structures revisited: multiple human pose estimation, IEEE Trans. Pattern Anal. Mach. Intell., 38 (2016), 1929–1942. https://doi.org/10.1109/TPAMI.2015.2509986 doi: 10.1109/TPAMI.2015.2509986
![]() |
[24] | A. Newell, K. Yang, J. Deng, Stacked Hourglass Networks for human pose estimation, in European Conference on Computer Vision, 9912 (2016), 483–499. https://doi.org/10.1007/978-3-319-46484-8_29 |
[25] | K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2016), 770–778. https://doi.org/10.1109/CVPR.2016.90 |
[26] | S. Ioffe, C. Szegedy, Batch normalization: Accelerating deep network training by reducing internal covariate shift, in Proceedings of the 32nd International Conference on International Conference on Machine Learning, 37 (2015). 448–456. |
[27] | R. Arora, A. Basu, P. Mianjy, A. Mukherjee, Understanding deep neural networks with rectified linear units, preprint, arXiv: 1611.01491. |
[28] | Q. Wang, B. Wu, P. Zhu, P. Li, W. Zuo, Q. Hu, ECA-Net: Efficient channel attention for deep convolutional neural networks, in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2020), 11531–11539. https://doi.org/10.1109/CVPR42600.2020.01155 |
[29] |
M. Schuster, K. K. Paliwal, Bidirectional recurrent neural networks, IEEE Trans. Signal Process., 45 (1997), 2673–2681. https://doi.org/10.1109/78.650093 doi: 10.1109/78.650093
![]() |
[30] | A. J. Haug, Bayesian Estimation and Tracking: A Practical Guide, Wiley, Hoboken, 2012. |
[31] |
J. Pearl, Fusion, propagation, and structuring in belief networks, Artif. Intell., 29 (1986), 241–288. https://doi.org/10.1016/0004-3702(86)90072-X doi: 10.1016/0004-3702(86)90072-X
![]() |
[32] |
H. T. Ma, Z. Yang, J. F. Griffith, P. C. Leung, R. Y. W. Lee, A new method for determining lumbar spine motion using bayesian belief network, Med. Biol. Eng. Comput., 46 (2008), 333–340. https://doi.org/10.1007/s11517-008-0318-y doi: 10.1007/s11517-008-0318-y
![]() |
[33] | D. Tome, C. Russell, L. Agapito, Lifting from the deep: Convolutional 3D pose estimation from a single image, in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2017), 5689–5698. https://doi.org/10.1109/CVPR.2017.603 |
[34] | J. Martinez, R. Hossain, J. Romero, J. J. Little, A simple yet effective baseline for 3D human pose estimation, in 2017 IEEE International Conference on Computer Vision (ICCV), (2017), 2659–2668. https://doi.org/10.1109/ICCV.2017.288 |
[35] | D. Drover, M. v Rohith, C. H. Chen, A. Agrawal, A. Tyagi, P. H. Cong, Can 3D pose be learned from 2D projections alone?, in European Conference on Computer Vision, (2018), 78–94. https://doi.org/10.1007/978-3-030-11018-5_7 |