We deal with overdetermined system of linear equations AX≈B , which is common in linear parameter estimation problem [9]. If the data matrix A and observation matrix B are contaminated with errors, and all the errors are uncorrelated and have equal variances, then the total least squares (TLS) technique is appropriate for solving this system [9]. Kukush and Van Huffel [5] showed the statistical consistency of the TLS estimator Xˆtls as the number m of rows in A grows, provided that the errors in [A,B] are row-wise i.i.d. with zero mean and covariance matrix proportional to a unit matrix; the covariance matrix was assumed to be known up to a factor of proportionality; the true input matrix A0 was supposed to be nonrandom. In fact, in [5] a more general, element-wise weighted TLS estimator was studied, where the errors in [A,B] were row-wise independent, but within each row, the entries could be observed without errors, and, additionally, the error covariance matrix could differ from row to row. In [6], an iterative numerical procedure was developed to compute the elementwise-weighted TLS estimator, and the rate of convergence of the procedure was established.

In a univariate case where B and X are column vectors, the asymptotic normality of Xˆtls was shown by Gallo [4] as m grows. In [7], that result was extended to mixing error sequences. Both [4] and [7] utilized an explicit form of the TLS solution.

In the present paper, we extend the Gallo’s asymptotic normality result to a multivariate case, where A, X, and B are matrices.

Now a closed-form solution is unavailable, and we work instead with the cost function. More precisely, we deal with the estimating function, which is a matrix derivative of the cost function. In fact, we show that under mild conditions, the normalized estimator converges in distribution to a Gaussian random matrix with nonsingular covariance structure. For normal errors, the latter structure can be estimated consistently based on the observed matrix [A,B] . The results can be used to construct the asymptotic confidence ellipsoid for a vector Xu , where u is a column vector of the corresponding dimension.

The paper is organized as follows. In Section 2, we describe the model, refer to the consistency result for the estimator, and present the objective function and corresponding matrix estimating function. In Section 3, we state the asymptotic normality of Xˆtls and provide a nonsingular covariance structure for a limit random matrix. The latter structure depends continuously on some nuisance parameters of the model, and we derive consistent estimators for those parameters. Section 4 concludes. The proofs are given in Appendix. There we work with the estimating function and derive an expansion for the normalized estimator using Taylor’s formula. The expansion holds with probability tending to 1.

Throughout the paper, all vectors are column ones, E stands for the expectation and acts as an operator on the total product, cov(x) denotes the covariance matrix of a random vector x, and for a sequence of random matrices {Xm,m≥1} of the same size, the notation Xm=Op(1) means that the sequence {‖Xm‖} is stochastically bounded, and Xm=op(1) means that ‖Xm‖⟶P0 . By Ip we denote the unit matrix of size p.

Introduce further assumptions to state the asymptotic normality of Xˆtls . We need a bit higher moments compared with conditions (ii) and (iii) in order to use the Lyapunov CLT. Recall that z˜i satisfies condition (i).

Introduce a random element in the space of systems consisting of five matrices: (3.1) Wi=(a0ia˜iT,a0ib˜iT,a˜ia˜iT−σ2In,a˜ib˜iT,b˜ib˜iT−σ2Id).

Hereafter ⟶d stands for the convergence in distribution.

Now, we state the asymptotic normality of Xˆtls .

In [7], Section 2, we can find a discussion of importance of the asymptotic normality result for Xˆtls . It is claimed there that the formula for the asymptotic covariance structure of Xˆtls is computationally useless, but in case where the limit distribution is nonsingular, we can use the block-bootstrap techniques when constructing confidence intervals and testing hypotheses.

However, in the case of normal errors z˜i , we can apply Theorem 8(b) to construct the asymptotic confidence ellipsoid, say, for X0u , u∈Rd×1 , u≠0 . Indeed, relations (3.1)–(3.4) show that the nonsingular matrix Su:=cov(VA−1Γ(X0)u) is a continuous function Su=Su(X0,VA,σ2) of unknown parameters X0 , VA , and σ2 . (It is important here that now the components Γj of Γ are independent, and the covariance structure of each Γj depends on σ2 and VA , not on some other limit characteristics of A0 ; see Lemma 6.) Once we possess consistent estimators VˆA and σˆ2 of VA and σ2 , the matrix Sˆu:=Su(Xˆtls,VˆA,σˆ2) is a consistent estimator for the covariance matrix Su .

Hereafter, a bar means averaging for rows i=1,…,m , for example, abT‾=m1m∑i=1maibiT .

Finally, for the case z˜1∼N(0,σ2In+d) , based on Lemma 10 and the relations 1m(Xˆtlsu−X0u)⟶dN(0,Su),Su>0,Sˆu⟶PSu, 0,\hspace{2.5pt}\hat{S}_{u}\stackrel{\mathrm{P}}{\longrightarrow }S_{u},\]]]> we can construct the asymptotic confidence ellipsoid for the vector X0u in a standard way. Remark 12.

In a similar way, a confidence ellipsoid can be constructed for any finite set of linear combinations of X0 entries with fixed known coefficients.

We extended the result of Gallo [4] and proved the asymptotic normality of the TLS estimator in a multivariate model AX≈B . The normalized estimator converges in distribution to a random matrix with quite complicated covariance structure. If the error distribution is symmetric around the origin, then the latter covariance structure is nonsingular. For the case of normal errors, this makes it possible to construct the asymptotic confidence region for a vector X0u , u∈Rd×1 , where X0 is the true value of X.

In future papers, we will extend the result for the elementwise weighted TLS estimator [5] in the model AX≈B , where some columns of the matrix [A,B] may be observed without errors, and, in addition, the error covariance matrix may differ from row to row.