100 Multi-innovation stochastic gradient algorithms for multi-input multi-output systems

2. The multi-innovation SG algorithm for the MIMO systems
Let us introduce some notations first. The symbol I stands for an identity matrix of appropriate sizes; 1n×m is an n × m matrix whose elements are 1; the superscript T denotes the matrix transpose; E denotes the expectation operator; the norm of a matrix X is defined as X 2 = tr[X X T] = tr[X T X ].
Digital Signal Processing 19 (2009) 545–554
Contents lists available at ScienceDirect
Digital Signal Processing
/locate/dsp
Hale Waihona Puke Multi-innovation stochastic gradient algorithms for multi-input multi-output systems
Earlier work on parameter estimation for multi-input multi-output (MIMO) systems exists: the previous least squares based algorithms adopted the idea of decomposing a system into several subsystems and then estimating the parameters of each subsystem, respectively [6,7]. Since such algorithms require computing many covariance matrices, they have the drawback of having heavy computational load. Subspace state-space identification methods can estimate the parameter matrices of the state space models of MIMO systems [8–11] by determining the extended observability matrix from the singular value decomposition or RQ factorization of an input–output information matrix but the difficulty and complexity in computation increase with the size of the information matrix growing [12]. The hierarchical least squares algorithm is to decompose an MIMO system into two subsystems to identify, but the reducing computational load still is limited [13].
© 2008 Published by Elsevier Inc.
1. Introduction
Signal processing, filtering and prediction, and system parameter estimation have received much attention, e.g. [1–5]. For example, Malik and Salman discussed the least mean square algorithm for state space models [1]; Abrahamsson et al. studied the parameter estimation problem of the bilinear models for submarine systems [2]; Zrguine analyzed the steadystate performances of the normalized least mean fourth algorithms [3].
* Corresponding author. Fax: +86 510 85910633.
E-mail address: fding@ (F. Ding).
1051-2004/$ – see front matter © 2008 Published by Elsevier Inc.
Consider an MIMO system described by a multivariable autoregressive model with exogenous input (ARX model) [15],
A(z) y(t) = B(z)u(t) + v(t),
(1)
where u(t) ∈ Rr is the system input vector, y(t) ∈ Rm the system output vector, v(t) ∈ Rm a stochastic white noise vector with zero mean, A(z) and B(z) are polynomial matrices in the unit backward shift operator [z−1 y(t) = y(t − 1)], and
doi:10.1016/j.dsp.2008.12.002
546
L. Han, F. Ding / Digital Signal Processing 19 (2009) 545–554
not involve computing covariance matrices, and uses not only the current innovation but also the past innovation at each iteration, and repeatedly utilizes the available input–output data, thus the parameter estimation accuracy is improved.
Keywords: Stochastic gradient Parameter estimation Filtering Multi-innovation identification Multi-input multi-output systems
abstract
This paper develops a multi-innovation stochastic gradient (MISG) algorithm for multiinput multi-output systems by expanding the innovation vector to an innovation matrix. The convergence analysis shows that the parameter estimates by the MISG algorithm consistently converge to the true parameters under the persistent excitation condition. The MISG algorithm uses not only the current innovation but also the past innovation at each iteration and repeatedly utilizes the available input–output data, thus the parameter estimation accuracy can be improved. The simulation example confirms the theoretical results.
Briefly, the paper is organized as follows. Section 2 develops the MISG algorithm for the MIMO systems and Sections 3 studies the convergence properties of the proposed MISG algorithm by using the martingale convergence theorem. Section 4 provides an illustrative example. Finally, concluding remarks are given in Section 5.
Lili Han, Feng Ding ∗
Control Science and Engineering Research Center, Jiangnan University, Wuxi 214122, PR China
article info
Article history: Available online 25 December 2008
The stochastic gradient (SG) algorithm requires less computational effort than the recursive least squares (RLS) algorithm, but has slower convergence rate than the RLS algorithm [14–16]. In order to improve the convergence rate of the SG algorithm, Ding and Chen presented the multi-innovation identification theory [16,17] and multi-innovation SG algorithms for linear regression models [16] and auxiliary model based multi-innovation least squares algorithms for multi-input singleoutput output error systems [17,18] by extending the scalar innovation to an innovation vector. In this paper, the multiinnovation identification theory is used to deal with the identification problems of MIMO systems and the multi-innovation stochastic gradient (MISG) algorithm is developed for the MIMO systems by extending the single innovation vector to an innovation matrix. This is the main differences from the previous work in [16]. The proposed MIMO MISG algorithm does