Reduction for Imaging and Communication Systems (ISNIC) A ROBUST CLUSTERING TECHNIQUE FOR M

A ROBUST CLUSTERING TECHNIQUE FORMULTI-SPECTRAL SATELLITE IMAGESTatsuya YamazakiATR Adaptive Communications Research Laboratories2-2 Hikaridai, Seika-cho, Soraku-gun,Kyoto 619-0288 JapanE-mail yamazaki@acr.atr.co.jpAbstract A robust clustering technique is developped for multi-spectral satellite images. The observed image data are assumed to come from a mixture of multi-variate normal densities and the number of densities is assumed to be known. In the clustering technique the parameters of the densities are tentatively estimated by a multi-dimensional histogram and the minimum distance clustering method. Using the tentative estimates as the initial values, the EM (Expectation and Maximization) algorithm improves the parameters of the mixture of densities recursively. Image classification is carried out by the conventional maximum likelihood method with the parameters estimated by the EM algorithm. The proposed method is robust for noises and gives stable classi-fication results, because no random selection is used for the initial parameter estimation. The per-formance of the algorithm is demonstrated by computer simulations and an experimental result of Landsat image classification is presented.1 IntroductionMulti-spectral satellite image classification methods can be categorized into supervised classifica-tion and unsupervised classification. The former uses sample data for training and usually results in better classification than the latter. The sample data are not always available, however, it is impor-tant and challenging to develop unsupervised classification methods for multi-spectral satellite images. Unsupervised classification is also called clustering.The real multi-spectral images may be degraded by noises in the process of sensing, transmis-sion, or recording. Also they may include unexpected areas like shadows. One of the famous and wide-spread clustering techniques is the k-means algorithm [1], that works well for uncontami-nated data but may show ambiguous results under noisy environment because of the random selec-tion of initial estimates.In this manuscript a novel clustering technique is developped, which is robust for such obstruc-tions as noises or unexpected areas. In the proposed technique the observed data are assumed tocome from a mixture of multi-variate normal density functions and a multi-dimensional histogram and the minimum distance classification method are used to specify the initial values of estimates.This histogram selection is more stable than the random selection used in the k-means algorithm.Then the proposed technique uses the Expectation and Maximization (EM) algorithm to improve the estimates of mixture density parameters. The multi-spectral satellite image data are classified by the conventional maximum likelihood (ML) method with the estimated parameters.2 Robust Clustering TechniqueThe multi-spectral image data y are observed through K bands, namely a K -dimensional feature vector y ij is observed at the ij th pixel. It is assumed that the number of cluster N is known and y isa realization from a mixture of multi-variate normal density functions whose parameters (mean vectors and covariance matrices) are unknown. The form of a mixture of multi-variate normal density functions is(1)where p n (y ij ; µn , σn ) denotes a multi-variate normal density with a mean vector µn and a covariance matrix σn . αn is the proportion of the n -th normal density in the mixture and .The proposed clustering algorithm is depicted in Fig. 1. In STEP 1, tentative values of the mean vectors are estimated using a multi-dimensional histogram. Then all of observed data are clustered by the minimum distance classification in STEP 2. In STEP 3, initial estimates of the mean vectors and covariance matrices are calculated using the clustering result of STEP 2, then they are used as the initial values for the EM algorithm in STEP 4. The EM algorithm is an iterative algorithm for calculating the maximum-likelihood estimates for µn , σn , and αn(n =1,..., N ). In the EM algorithm the observed data can be viewed as incomplete-data , whereas each element of the complete-data set can be defined to be a two-component vector consisting of an observation and a label specifying which density of the mixture has generated the observation.Each iteration of the EM algorithm consists of an expectation step followed by a maximization step. Details can be found in [2].STEPs 1 to 4 of the algorithm are used for estimation of mean vectors and covariance matrices of the density functions. In STEP 5 the images are classified by the conventional ML method.3 SimulationThe effectiveness of the proposed technique is evaluated by computer simulations. In the simula-tions, data are observed as two-dimensional vectors (K =2) generated from five two-variant normal distributions. The parameters used for generation of the observed data are in Table 1. All of covari-p p ij n n ij n n n Ny y ()==∑αµσ(;,),1Figure 1. Clustering algorithm.ances were set to 0. Since Class 5 emulates a unexpected area, the number of elements in Class 5 is less than other classes and clustering was carried out subject to N=4.The final estimates by STEPs 1 to 4 in the proposed method are shown in Table 2, whereas the estimates by the k-means algorithm are shown in Table 3. In Table 3, two cases of estimation by the different random initial values are shown. It is seen that estimates by the k-means algorithm depend on selection of initial values and they are easily influenced by obstructions. On the other hand, since the proposed technique uses the multi-dimensional histogram and finds dense sections to specify the initial values, it is more robust for obstructions than the k-means algorithm.After the parameter estimation, the observed data are classified by the conventional ML method using the estimated parameters. The rates of accurate class labeling (RACL) defined by (2) are 88.75% for the proposed technique, 88.39% for Case 1 of the k-means algorithm and 61.96% for Case 2 of the k-means algorithm.(2) It is also seen that RACL depends on the initial selection of estimates in the k-means algorithm.4 Landsat MSS Image ClusteringThe proposed clustering technique was applied to real remotely sensed data. Experimental data were acquired by Landsat Multi-spectral Scanner (MSS). The number of observational channels is 4 (K =4) and the size of image is 256x256 pixels. Fig. 2 shows the observed image of Band 7. The RACL The number of elements whose estimated labels are identical with the true ones.2800 (the total number of elements of class 1 to 4.)=.Table 1: Parameters used for generation of the observed data in the first simulation.1s s a l C 2s s a l C 3s s a l C 4s s a l C 5s s a l C µ)08,07()021,021()031,061()001,531()09,001(σ1020*******σ2020*******r e b m u N 007007007007002Table 2: Estimates by the proposed clustering technique. Initial is after STEP 3 and EM is after STEP 4.1s s a l C 2s s a l C 3s s a l C 4s s a l C l a i t i n I µ)2.39,2.38()1.131,5.421()6.821,5.161()2.79,6.031(M E µME )6.28,3.87()4.221,2.711()4.821,5.851()5.201,5.331(σ1ME 5.224.98.514.9σ2M E 7.918.93.517.01Table 3: Estimates by the k-means algorithm. Two cases with different initial values are shown.1s s a l C 2s s a l C 3s s a l C 4s s a l C 1e s a C µ1s n a e m -k )0.97,9.17()9.021,7.611()0.231,8.261()1.99,2.531(σ11s n a e m -k 6.816.112.318.01σ21s n a e m -k 9.716.011.316.012e s a C µ2s n a e m -k )9.47,1.16()9.011,5.721()7.031,5.261()9.88,1.29(σ12s n a e m -k 7.410.212.315.21σ22s n a e m -k 2.813.415.417.71number of clusters N was set to 5, because the data are expected to include five classes: water, coniferous trees, deciduous trees, open fields, and epidemics. The clustering is shown in Fig. 3. The point is that Fig. 3 is classified without using any sample data. It can be improved by using contextual classification methods using spatial information, e.g. Markov Random Fields [3],[4].5 ConclusionA novel clustering technique for multi-spectral remotely sensed images was proposed. It uses a multi-dimensional histogram and the minimum distance classification to estimate the initial values for the normal distribution parameters. The EM algorithm is used to improve the initial estimates. The classification is carried out by the ML method.The proposed technique is robust for obstructions in image data like noises or unexpected areas compared with the clustering method that selects the initial values randomly. Although it is sup-posed that use of spatial information can improve the classification result, the proposed technique is useful to give an initial classification without using any sample data.6 AcknowledgmentThe author would like to thank Dr. Bokuji Komiyama and Dr. Jun Matsuda of ATR Adaptive Communications Research Laboratories for their guidance and support.References[1] G. A. Carpenterer and S. Grossberg, Pattern Recognition by Self-Organizing Neural Net-works, Cambridge, MA: MIT Press, 1991.[2] A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum Likelihood from Incomplete Datavia the EM Algorithm,” J. Roy. Statistics Soc., vol. 39, no. 1, pp. 1-38, 1977.[3] T. Yamazaki and D. Gingras, “Image Classification Using Spectral and Spatial InformationBased on MRF Models,” IEEE Trans. Image Processing, vol. 4, no. 9, Sep. 1995, pp. 1333-1339.[4] T. Yamazaki and D. Gingras, “A Contextual Classification System for Remote Sensing Usinga Multivariate Gaussian MRF Model,’’ in Proceedings of the 1996 IEEE International Sym-posium on Circuits and Systems, vol. 2, pp. 648-651.Figure 3. Clustering result of Landsat MSS data (N=5).Figure 2. Landsat MSS image. Band 7 is shown.。