A Discrete-Time Projection Neural Network for Solving

A Discrete-Time Projection Neural Network for Solving Degenerate Convex Quadratic OptimizationZican Zhang1·Chuandong Li1·Xing He1·Tingwen Huang2Received:24June2015/Revised:21March2016/Accepted:23March2016©Springer Science+Business Media New York2016Abstract This paper presents a discrete-time neural network to solve convex degen-erate quadratic optimization problems.Under certain conditions,it is shown that the proposed neural network is stable in the sense of Lyapunov and globally convergent to an optimal pared with the existing continuous-time neural networks for degenerate quadratic optimization,the proposed neural network in this paper is more suitable for hardware implementation.Results of two experiments of this neural network are given to illustrate the effectiveness of the proposed neural network.Keywords Degenerate quadratic optimization·Discrete-time neural network·Convex optimization·Algorithm convergence1IntroductionOptimization problem plays an important role in science and engineering applications, such as regression analysis[14],signal and image processing[29],manufacturing,B Chuandong Lilicdswu@Zican Zhangzhangzican123@Xing Hehexingdoc@Tingwen Huangtingwen.huang@1School of Electronics and Information Engineering,Southwest University,Chongqing400715, People’s Republic of China2Department of Mathematics,Texas A&M University at Qatar,P.O.Box23874,Doha,Qataroptimal control[3,15,16,30,34]and system identification.In many cases,the real-time solutions of optimization problems are required.But traditional algorithms are not efficient because the computing time is greatly dependent on the structure and dimension of the problems.One possible promising approach to real-time optimiza-tion is to utilize artificial neural networks,which is much faster than the traditional optimization algorithms.In[10],Tank and Hopfieldfirst proposed a neural network for solving linear optimization problems.Their heuristic work has inspired many researchers to investigate neural networks for solving optimization problems[19]. Soon after that,Kennedy and Chua[12]presented a recurrent neural network for solv-ing constraint nonlinear programming problems,which used both gradient method and penalty parameter method,but its equilibrium points correspond to approximate optimal solutions only.Since then,the advent of the neural network method for opti-mization problems already has profound influence,for instance the primal–dual neural networks[28],projection neural networks[2,4,6,7,18,20–22,33,35],one-layer neural networks[8,9]and non-smooth neural networks[5,17,24,25].Recently,the projection method is widely used when building the neural networks in constraint optimization,such as[23]and[26].In particular,the condition that using neural networks to solve a category of non-degenerate quadratic optimization problems [35]has been widely studied.But few people utilize neural network to solve this class of degenerate quadratic optimization problems,and the coefficient of the quadratic term is a degenerate matrix,which also can be said the non-full rank matrix.However, degenerate quadratic programming problems are more general.Specially,Tao and Cao [27]developed a projection neural network to solve the degenerate convex quadratic programming problems.Xue and Bian[35]developed a continuous-time projection neural network for solving degenerate convex quadratic problems with general linear constraints.There exist a few discrete-time neural networks for solving constraint optimization problems[23,26,27,36].The discrete-time projection neural networks with global exponential stability in[23]and[26]were proposed for solving the strictly convex quadratic optimization problems.In[36],a discrete-time primal–dual neural network that derived from its corresponding continuous-time neural network was proposed for solving nonlinear convex optimization problems.For this kind of problems,many researchers focused on the methods of continuous-time neural network.In this paper, we have solved the degenerate quadratic optimization problems using discrete-time neural network approach,and,to the best of our knowledge,few(if any)publica-tions have reported this method in the literature.Continuous-time neural network can hardly simulate on hardware circuits,while the discrete-time neural network can eas-ily simulate on hardware circuits.Motivated by the advantage of discrete-time neural network,we constructed the hardware circuit for simulating the iterative calculation. It means a great significance in engineering application,and in many cases,the real-time solutions of optimization problems are necessary.Moreover,based on Lyapunov function theory,and under certain conditions,it is shown that the proposed neural network is Lyapunov stable and mightfind the optimization solution effectively for many optimization problems.The rest of this paper is organized as follows:In Sect.2,the discrete-time projection neural network is described.In Sect.3,we will prove the asymptotic stability of theproposed neural network.In Sect.4,two examples are given to illustrate the good performance of the proposed neural network.Conclusions are found in Sect.5.2Problem Formulation and Model DescriptionConsider the following quadratic optimization problem:minimize 12x T Qx+c T xsubject to b1≤Bx≤b2d0≤x≤h0(1)where Q∈R n×n,B∈R m×n,b1,b2∈R m,d0,h0∈R n,and c∈R n.Q is a symmetric and semi-positive definite matrix.We only investigate the degenerate quadratic optimization problems in this paper.In order to facilitate the calculation, method is similar as[32].First,letE=I nB,d=d0b1,h=h0b2,where I n∈R n×n is an unit matrix.Then,problem(1)can be rewritten asminimize 12x T Qx+c T xsubject to d≤Ex≤h,(2)where d=[d1,d2,...,d n+m]T and h=[h1,h2,...,h n+m]T.The Lagrange function of(2)is defined as:L(x,u,η)=12x T Qx+c T x−u T(Ex−η),(3)where u∈R m+n is the Lagrange multiplier andη∈X={u∈R n+m|d≤u≤h}. According to the well-known saddle point theorem[1],x∗is an optimal solution of (2)if and only if there exist u∗andη∗satisfyingL(x∗,u,η∗)≤L(x∗,u∗,η∗)≤L(x,u∗,η).(4)Then the derivation process is similar as that in[23],from the left inequality in(4), we haveEx∗=η∗(5) And from the right inequality in(4),we can getf(x∗)−f(x)≤0(6)and(u ∗)T (η−η∗)≥0∀η∈X (7)According to the projection formulation [13],it is equivalent to thatη∗=P X (η∗−u ∗),(8)where P X (u )=[P X (u 1),P X (u 2),...,P X (u n +m )]T is a projection function,for i =1,2,...,n +mP X (u i )=⎧⎪⎨⎪⎩d i ,u i <d i ,u i ,d i ≤u i ≤h i ,h i ,u i >h iwhile,in convex optimization,extreme point must be the optimal point,we already have f (x ∗)≤f (x ),∀x ∈R n ,and therefore∇f (x ∗)=Qx ∗+c −E T u ∗=0(9)So,if there exists (x ∗,u ∗,η∗),it certainly satisfiesQx ∗+c −E T u ∗=0,Ex ∗=P X Ex ∗−u ∗ .(10)Based on the above analysis,the projection equation can be designed as follows:x ∗=P X x ∗−(Qx ∗+c −E T u ∗) ,Ex ∗=P X Ex ∗−u ∗ .(11)that is I n 0n ×m0n ×m ⎛⎝x ∗v ∗w ∗⎞⎠=P X ⎡⎣ I n 0n ×n 0n ×m ⎛⎝x ∗v ∗w ∗⎞⎠−⎛⎝ Q −I n −B T ⎛⎝x ∗v ∗w ∗⎞⎠+c ⎞⎠⎤⎦(12)andI n0n×n0n×mB0m×n0m×m ⎛⎝x∗v∗w∗⎞⎠=P X ⎡⎣I n−I n0n×mB0m×n−I m⎛⎝x∗v∗w∗⎞⎠⎤⎦(13)where x∗=⎛⎜⎝x1...x n⎞⎟⎠,u∗=((v∗)T,(w∗)T)T,v∈Rn,w∈R m and0n×n,0n×m,0m×mare all zero matrix,I n,I m are unit matrix,then let us combine the right and the left side of equation(12)(13),respectively,we can get a new projection equation as follows:⎛⎝I n0n×n0n×mB0m×n0m×mI n0n×n0n×m⎞⎠⎛⎝xvw⎞⎠=P Y⎡⎣⎛⎝I n−I n0n×mB0m×n−I mI n0n×n0n×m⎞⎠⎛⎝xvw⎞⎠−⎛⎝⎛⎝0n×n0n×n0n×m0m×n0m×n0n×mQ−I n−B T⎞⎠⎛⎝xvw⎞⎠+⎛⎝0n×10m×1c⎞⎠⎞⎠⎤⎦(14)where Y={u∈R2n+m|´d≤u≤´h},´d=d0+∞,´h=h0+∞,let y∗=⎛⎝x∗v∗w∗⎞⎠,M=⎛⎝I n0n×n0n×mB0m×n0m×mI n0n×m0n×m⎞⎠,W=⎛⎝I n−I n0n×mB0m×n−I mI n0n×n0n×m⎞⎠,A=⎛⎝0n×n0n×n0n×m0m×n0m×n0m×mQ−I n−B T⎞⎠and C=⎛⎝0n×10m×1c⎞⎠,then the upward projection equation can be transformed into thisform:My(k)=P Y[W y(k)−(Ay(k)+C)],(15) y∗is an equilibrium point of the upward projection equation,thenx∗=I n0n×n0n×my∗is an optimal solution of(2).Next,the proposed neural network for solving problem (2)can be designed as:State equation:y(k+1)=y(k)+λ{P Y[W y(k)−(Ay(k)+C)]−My(k)}(16)Fig.1The principle diagram of model(16)Output equation:x(k)=I n0n×n0n×my(k)(17)where H∈R(2n+m)×(2n+m)is a matrix,λ>0is a positive constant,and the asymp-totic stability of the network is depend on this constant,and we will give the proof later.And y(k)∈R2n+m is the state variable,x(k)∈R n is the output variable,H=⎛⎝I n+Q(I n+Q)B T(I n+Q)2I n0n×m−I n0m×n I m−B⎞⎠Since Q is a degenerate matrix,then Q+I n is positive matrix,hence,H is a non-singular matrix.Therefore,y(k+1)=y(k)if and only if(15)holds.Remark1Figure1shows the principle diagram of model(16).Figure2shows the detailing of the module W1y(k),in Fig.2,R i1,R i2,…,R in,and R s1,R s2,…,R sn are all variable resistors,the input signal of R i1is y(k)1,first element of the column vector. While the value of R s1R i1represents the magnification of amplifier A1,the resistance canbe changed as the optimization problems variety,and then,we let W11=R s1Ri1,whileW11is thefirst row andfirst column element of matrix W.There are some other modules of the coefficients matrix,for example the module of matrix H and module of A,and the principle is the same.We can accommodate all the elements in the coefficient matrix corresponding to the ratio of resistance.The triggered subsystem is Dflip-flop,and it is rising edge trigger.Clock signal is square wave which is controllable as required,Fig.2Module of the matrix Wand when the rising edge arrives,triggered subsystem outputs a signal which activatean iteration,after W1y(k)passes through the Amplifier1,the signal will change intoW1y(k)−[(A1y(k)+C1)],then the signal passes the Projection Module,the Amplifier 2,the module of matrix H,after a cycle of operation,the signal feedback representingy(k+1).Iterative operation will go on along with the rising edge comes.3Stability AnalysisIn this section,the stability of the proposed neural network will be analyzed andproved.Lemma1[35](2007)Let E P be the set consisted of the solutions of(16),and supposethat y∗∈E P is one of the equilibrium points of(16),letδ=W y(k)−(Ay(k)+c), then we can get the following inequality:(y(k)−y∗)T(2M−W+A)T[P Y[δ]−My(k)]≤− P Y[δ]−My(k) 2−(y(k)−y∗)T(M−W+A)T M(y(k)−y∗)(18)Theorem1For model(16),if the constantλsatisfies0<λ≤2H 2λmax(P),then theneural network is asymptotically stable,whereP=⎛⎝(I n+Q)−10n×n0n×m0n×n I n0n×m0m×n0m×n0m×m⎞⎠,λmax(P)is the maximum eigenvalue of matrix P. H 2is the square of the two-norm of H.Proof See“Appendix”. 4Illustrative ExamplesIn this section,two examples are given to illustrate the effectiveness of the proposed neural network.Example1Consider the following degenerate optimization problem[31]:minimize x21+x22+x23−2x1x2−x3subject to0≤x1,x2,x32≤x1−x2≤30≤x1+x3≤3(19) where x=(x1,x2,x3)T,and this problem has a solution setΩ∗={(x1,x2,0.5)|x1−x2=2}.In this problem,the coefficients are as follows Q=⎛⎝2−20−220002⎞⎠,B=1−10101,d0=⎛⎝⎞⎠,b1=2,b2=33,c=⎛⎝−1⎞⎠,the maximal eigenvalue of matrix Pis1,and then,we can obtain the feasible region of the constantλ:0<λ≤2H 2=0.0028,in this example,the value ofλis0.0023.The random initial points and the corre-sponding limit points are given in Table1,and the convergence result is shown in Fig.3.If we let the step sizeλ=0.0328,which beyond its feasible region,and with the initial point(5,1,3,6,7,9,2,4),then the system will be unstable in Fig.4.The initial point The limit point The optimal solution(0,0,0,1,0,0,0,0)T(2.00,0.00,0.50,0.00,0.00,0.00,4.00,0.00)T(2.00,0.00,0.50)T (1,2,3,4,5,6,7,8)T(2.00,0.00,0.50,0.00,0.00,0.00,4.00,0.00)T(2.00,0.00,0.50)T (8,2,2,2,2,3,4,7)T(2.00,0.00,0.50,0.00,0.00,0.00,4.00,0.00)T(2.00,0.00,0.50)T (5,1,3,6,7,9,2,4)T(2.00,0.00,0.50,0.00,0.00,0.00,4.00,0.00)T(2.00,0.00,0.50)T (3,1,2,8,4,6,5,7)T(2.00,0.00,0.50,0.00,0.00,0.00,4.00,0.00)T(2.00,0.00,0.50)T −(1,1,2,3,4,6,5,7)T(2.00,0.00,0.50,0.00,0.00,0.00,4.00,0.00)T(2.00,0.00,0.50)TFig.3Convergence result of Example1Fig.4Behavior of the network under the unsuitable step sizeThe initial point The limit point The optimal solution(0,0,0,3,1,0,0,0)T(0.00,0.00,1.00,0.00,0.00,0.00,0.00,0.00)T(0.00,0.00,1.00)T (1,2,3,4,5,6,7,8)T(0.00,0.00,1.00,0.00,0.00,0.00,0.00,0.00)T(0.00,0.00,1.00)T (0,1,2,4,2,3,6,1)T(0.00,0.00,1.00,0.00,0.00,0.00,0.00,0.00)T(0.00,0.00,1.00)T (4,2,5,6,2,3,4,7)T(0.00,0.00,1.00,0.00,0.00,0.00,0.00,0.00)T(0.00,0.00,1.00)T −(1,1,2,3,4,6,5,7)T(0.00,0.00,1.00,0.00,0.00,0.00,0.00,0.00)T(0.00,0.00,1.00)T (5,1,3,6,7,9,2,4)T(0.00,0.00,1.00,0.00,0.00,0.00,0.00,0.00)T(0.00,0.00,1.00)TExample2Consider this degenerate quadratic programming problem[11]:minimize 14x21+14x23+12x3subject to−1≤x1,x2,x3≤13≤x1+x2+4x3≤51≤x1−x2+2x3≤3(20) x=(x1,x2,x3)T,this problem has a solution set⎧⎪⎨⎪⎩x∗1=2x∗2−2/3<x∗1<2/3x∗3=1.In this problem,the coefficients are as follows Q=⎛⎝0.5−10−120000.5⎞⎠,B=1141−12,d0=⎛⎝−1−1−1⎞⎠,h0=⎛⎝111⎞⎠,b1=31,b2=53,c=⎛⎝−0.5⎞⎠,while the maximaleigenvalue of matrix P is1,then we can obtain the feasible region of the constantλin this example:0<λ≤2H 2=0.0106,and the value ofλis0.0073.The random initial points and the corresponding limit points are as Table2shows,and the convergence result in Fig.5shows that the neural network canfind the optimization solution effectively.If we let the step sizeλ=0.041,beyond its feasible region,and with the initial point-(1,1,2,3,4,6,5,7),the network will be unstable in Fig.6. Optimal solution cannot be found.Fig.5Convergence result of Example2Fig.6Behavior of the network under the unsuitable step size5ConclusionsIn this paper,a discrete-time neural network has been proposed for solving convex degenerate quadratic optimization problems,and the asymptotic stability can be guar-anteed if the parameterλis in its feasible region.Simulation results on numerical examples demonstrate the effectiveness of the proposed neural network for solving degenerate quadratic optimization problems.The circuit we build has made good preparation for the hardware implementation.In the near future,we will try to build neural networks of discontinuous activation functions for solving optimization prob-lems.Acknowledgments This work was supported by Natural Science Foundation of China (Grant Nos.61403313,61374078),Chongqing Research Program of Basic Research and Frontier Technology (No.cstc2015jcyjBX0052)and also supported by the Natural Science Foundation Project of Chongqing CSTC (No.cstc2014jcyjA40014).This publication was made possible by NPRP Grant No.NPRP 4-1162-1-181from the Qatar National Research Fund (a member of Qatar Foundation).AppendixProof of Theorem 1First,we can construct a Lyapunov functionV (k )=[y (k )−y ∗]T P [y (k )−y ∗]k ∈R +nwhere y ∗is one equilibrium point.Then calculating the differenceV (k )=V (k +1)−V (k )= y (k +1)−y ∗ T P y (k +1)−y ∗ − y (k )−y ∗ T P y (k )−y ∗=y (k +1)T Py (k +1)−y (k +1)Py ∗−(y ∗)T y (k +1)+(y ∗)T y ∗− y (k )T Py (k )−y (k )T Py ∗−(y ∗)T Py (k )+(y ∗)Py∗ =y (k +1)T Py (k +1)−y (k +1)T Py ∗−(y ∗)T Py (k +1)−y (k )T Py (k )+y (k )T Py ∗+(y ∗)T Py (k )=y (k +1)T Py (k +1)−y (k )T Py (k )−[y (k +1)−y (k )]T Py ∗−(y ∗)P [y (k +1)−y (k )]={y (k )+λH [P Y [δ]−My (k )]}T P {y (k )+λH [P Y [δ]−My (k )]}−y (k )T Py (k )−{λH [P Y [δ]−My (k )]}T Py ∗−(y ∗)T P {λH [P Y [δ]−My (k )]}=y (k )T Py (k )+y (k )T P λH {P Y [δ]−My (k )}+{λH [P Y [δ]]}T Py (k ){λH [P Y [δ]−My (k )]}T P {λH [P Y [δ]−My (k )]}−y (k )T y (k )−{λH [P Y [δ]−My (k )]}T Py ∗−(y ∗)T P {λH [P Y [δ]−My (k )]}=y (k )T P λH [P Y [δ]−My (k )]+{λH [P Y [δ]−My (k )]}T Py (k )+{λH [P Y [δ]−My (k )]}T P {λH [P Y [δ]−My (k )]}−{λH [P Y [δ]−My (k )]}T Py ∗−(y ∗)T P {λH [P Y [δ]−My (k )]}=λ y (k )−y ∗ T P {H [P Y [δ]−My (k )]}+λ{H [P Y [δ]−My (k )]}T P y (k )−y ∗+λ2{H [P Y [δ]−My (k )]}T P {H [P Y [δ]−My (k )]}=2λ y (k )−y ∗ T P H [P Y [δ]−My (k )]+λ2{H [P Y [δ]−My (k )]}T P {H [P Y [δ]−My (k )]}(21)and then,by simple calculation,P H =⎛⎝I n B T I n +Q I n 0n ×m −I n 0m ×n I m −B⎞⎠=2M −W +A Using the inequality in Lemma 1,it follows that2λ y (k )−y ∗ T P H [P Y [δ]−My (k )]+λ2{H [P Y [δ]−My (k )]}T P {H [P Y [δ]−My (k )]}≤−2λ P Y [δ]−My (k ) 2−2λ y (k )−y ∗ T (M −W +A )T M y (k )−y ∗+λ2{H [P Y [δ]−My (k )]}T P {H [P Y [δ]−My (k )]}≤−2λ P Y [δ]−My (k ) 2−2λ y (k )−y ∗ T (M −W +A )T M y (k )−y ∗+λ2 H 2λmax (P ) P Y [δ]−My (k ) 2= −2λ+λ2 H 2λmax (P ) P Y [δ]−My (k ) 2−2λ y (k )−y ∗ T (M −W +A )T M y (k )−y ∗(22)While(M −W +A )T M =⎛⎝Q 0n ×n 0n ×m 0n ×n0n ×n 0n ×m 0m ×n0m ×n 0m ×m ⎞⎠is a semi-positive definite matrix.Therefore,if the constant λsatisfies−2λ+λ2 H 2λmax (P )≤0(23)That is0<λ≤2 H 2λmax (P )(24)We obtainV (k )=2λ y (k )−y ∗ T P H [P Y [δ]−My (k )]+λ2{H [P Y [δ]−My (k )]}T P {H [P Y [δ]−My (k )]}≤−2λ P Y [δ]−My (k ) 2−2λ y (k )−y ∗ T (M −W +A )T M y (k )−y ∗+λ2 H 2λmax (P ) P Y [δ]−My (k ) 2=−2λ+λ2 H 2λmax(P)P Y[δ]−My(k) 2−2λy(k)−y∗T(M−W+A)T My(k)−y∗≤0(25)therefore,the network(16)we present is globally convergent.This proof is complete.References1.M.S.Bazaraa,C.M.Shetty,Nonlinear programming:theory and algorithms.J.Oper.Res.Soc.(1979).doi:10.2307/30096962.H.Che et al.,An intelligent method of swarm neural networks for equalities-constrained nonconvexoptimization.Neurocomputing.167,569–577(2015)3. B.Chen et al.,Distributed fusion estimation with communication bandwidth constraints.IEEE Trans.Autom.Control.60(5),1398–1403(2015)4.L.Cheng et al.,A neutral-type delayed projection neural network for solving nonlinear variationalinequalities.IEEE Trans.Circuits Syst.II Express Briefs55(8),806–810(2008)5.L.Cheng et al.,Recurrent neural network for non-smooth convex optimization problems with appli-cation to the identification of genetic regulatory networks.IEEE Trans.Neural Netw.22(5),714–726 (2011)6.X.He et al.,An inertial projection neural network for solving variational inequalities.IEEE Trans.Cybern.(2016).doi:10.1109/TCYB.2016.25235417.X.He et al.,Neural networks for solving Nash equilibrium problem in application of multiuser powercontrol.Neural Netw.57(9),73–78(2014)8.X.He et al.,A recurrent neural network for solving bilevel linear programming problem.IEEE Trans.Neural Netw.Learn.Syst.25(4),824–830(2014)9.X.He et al.,Neural network for solving convex quadratic bilevel programming problems.Neural Netw.51(3),17–25(2014)10.Tank Hopfield,“Neural”computation of decisions in optimization problems.Biol.Cybern.52(3),141–152(1985)11.M.J,Improvement of the convergence speed of a discrete-time recurrent neural network for quadraticoptimization with general linear constraints.Neurocomputing144,493–500(2014)12.Chua Kennedy,Neural networks for nonlinear programming.IEEE Trans.Circuits Syst.35(5),554–562(1988)13. D.Kinderlehrer,G.Stampacchia,An introduction to variational inequalities and their applications.SIAM.31,7–22(1980)14.S.Lee et al.,Landslide susceptibility analysis and its verification using likelihood ratio,logistic regres-sion,and artificial neural network models:case study of ndslides4(4),327–338(2007) 15.H.Li et al.,Control of nonlinear networked systems with packet dropouts:interval type-2fuzzy model-based approach.IEEE Trans.Cybern.(2014).doi:10.1109/TCYB.237181416. C.Li et al.,Distributed event-triggered scheme for economic dispatch in smart grids.IEEE Trans.Ind.Inf.(2015).doi:10.1109/TII.247955817. C.Li et al.,A generalized Hopfield network for nonsmooth constrained convex optimization:Liederivative approach.IEEE Trans.Neural Netw.Learn.Syst.27(2),308–321(2016)18. C.Li,X.Yu,W.Yu,Efficient computation for sparse load shifting in demand side management.IEEETrans.Smart Grid.(2016).doi:10.1109/TSG.252137719.Q.Liu,J.Wang,A one-layer recurrent neural network with a discontinuous activation function forlinear programming.Neural Comput.20(5),1366–1383(2008)20.Q.Liu,J.Wang,A projection neural network for constrained quadratic minimax optimization.IEEETrans.Neural Netw.Learn.Syst.26(11),2891–2900(2015)21.Q.Liu,J.Wang,A second-order multi-agent network for bound-constrained distributed optimization.IEEE Trans.Autom.Control.60(12),3310–3315(2015)22.Q.Liu et al.,One-layer continuous-and discrete-time projection neural networks for solving variationalinequalities and related optimization problems.IEEE Trans.Neural Netw.Learn.Syst.25(7),1308–1318(2014)23.Q.Liu,J.Cao,Global exponential stability of discrete-time recurrent neural network for solvingquadratic programming problems subject to linear constraints.Neurocomputing74(17),3494–3501 (2011)24.S.Qin et al.,Neural network for constrained nonsmooth optimization using Tikhonov regularization.Neural Netw.63(3),272–281(2015)25.S.Qin,X.Xue,A two-layer recurrent neural network for nonsmooth convex optimization problems.IEEE Trans.Neural Netw.Learn.Syst.26(6),1149–1160(2015)26.K.C.Tan et al.,Global exponential stability of discrete-time neural networks for constrained quadraticoptimization.Neurocomputing56(3),399–406(2004)27.Q.Tao,J.Cao,D.Sun,A simple and high performance neural network for quadratic programmingput.124(2),251–260(2001)28.J.Wang,Primal and dual neural networks for shortest-path routing.IEEE Trans.Syst.Man Cybern.Part A Syst.Hum.28(6),864–869(1998)29.W.E.Weideman,Neural network image processing system.U.S.Patent4941,122(1990)30.Z.Wu et al.,Sampled-data fuzzy control of chaotic systems based on T–S fuzzy model.IEEE Trans.Fuzzy Syst.22(1),153–163(2014)31.Y.Xia,J.Wang,A general methodology for designing globally convergent optimization neural net-works.IEEE Trans.Neural Netw.9(6),1331–1343(1998)32.Y.Xia et al.,A recurrent neural network with exponential convergence for solving convex quadraticprogram and related linear piecewise equations.Neural Netw.17(7),1003–1015(2004)33.Y.Xia et al.,A projection neural network and its application to constrained optimization problems.IEEE Trans.Circuits Syst.I Regul.Pap.49(4),447–458(2002)34.Y.Xu et al.,Asynchronous dissipative state estimation for stochastic complex networks with quantizedjumping coupling and uncertain measurements.IEEE Trans.Neural Netw.Learn.Syst.(2015).doi:10.1109/TNNLS.250377235.X.Xue,W.Bian,A project neural network for solving degenerate convex quadratic program.Neuro-computing70(13),2449–2459(2007)36.M.Yashtini,A.Malek,A discrete-time neural network for solving nonlinear convex problems withhybrid put.195(2),576–584(2008)。