Natural Object Classification using Artificial Neural Networks

NATURAL OBJECT CLASSIFICATION USINGARTIFICIAL NEURAL NETWORKS 1SAMEER SINGH 1, MARKOS MARKOU 1 & JOHN HADDON 2PANN RESEARCH 1, DEPARTMENT OF COMPUTER SCIENCE, UNIVERSITY OF EXETER, EXETER, UK 2DEFENCE EVALUATION AND RESEARCH AGENCY, FARNBOROUGH, UK{s.singh@, m.markou@, jf_haddon@ }1 © British Crown Copyright 1999/DERA; Published with the permission of the controller of Britannic Majesty's Stationary OfficeABSTRACTIn this paper we apply artificial neural networks for classifying texture data of various natural objects found in FLIR images. Hermite functions are used for texture feature extraction from segmented regions of interest in natural scenes taken as a video sequence. A total of 2680 samples for a total of twelve different classes are used for object recognition. The results on correctly identifying twelve natural objects in scenes are compared across ten folds of the cross-validation study. Neural networks are found to be extremely effective in robust classification of our data giving an average recognition rate of 91.8%.1. INTRODUCTIONNatural scene analysis is important for developing autonomous systems that are capable of recognising and interacting with their environment [1]. Natural scenes contain objects of interest that we would ideally want a computer to recognise automatically. In FLIR (Forward Looking Infrared) images, we have only gray scale data without any colour information. Hence, object recognition is based on shape and textural characteristics of regions of interest. These images are typically used in defence applications or otherwise where a cheap but robust method of image acquisition is required [2]. In our study, FLIR images have been taken from a low flying aircraft. In such images, hot areas appear white while cool areas are dark. The images are typical of the sort of scenes that the techniques are designed to analyse, although in general, image quality will be considerably poorer. A typical scene analysis system involves, in order, image acquisition, image segmentation, region classification and performance analysis steps. These are explained in greater detail in the next section. Image segmentation is based on using region splitting and merging techniques once textural features have been computed for the image. A number of methods for quantifying textural changes within FLIR images are possible including spatial and spectral techniques. In this paper we will measure texture using Hermite function coefficients obtained from the analysis of co-occurrence matrices. The objective of this paper is two-fold; (i) to measure the effectiveness of using Hermite functions for characterising image object texture; and (ii) to find the effectiveness of neural networks on FLIR image data recognition. In the following sections we present a description of our data, details of the classifier and results of our analysis.2. IMAGE SEGMENTATION AND FEATURE EXTRACTIONImage segmentation describes the process of separating regions of interest in an image, mostly objects, from the background. The process of segmenting images and extracting relevant data from them for object recognition can be described in brief as follows:! Segmentation : The FLIR image has been segmented using edge based co-occurrence techniques[3,4] that are able to both segment the key regions of the image, and to detect boundaries. A temporal component to the analysis has been incorporated into the co-occurrence matrix generation so that the quality of image segmentation is consistent between different images in the sequence.! Texture Description : each of the large (>500 pixels) segmented regions have been subjected to detailed texture analysis. A co-occurrence matrix [5] of a region of a single texture is widely recognised as having a characteristic texture that can be described using a variety of techniques [6]. In this research, the segmented regions are described using edge based co-occurrence matrices [7], an extension of the normal grey level co-occurrence matrix that allows greater flexibility in the description of the texture. The predominant form of these matrices is Gaussian with an overlaid higher level structure. The underlying Gaussian is due to Gaussian noise in the original imagery while the higher level structure is due to the texture of the originating region. It is this structure which most co-occurrence based texture measures seek to describe.An edge co-occurrence matrix is generated for each region using an operator appropriate to the scale and orientation of the texture. The physical aspects of the imaging system are taken into account in the definition of this operator. The resultant edge co-occurrence matrix is decomposed using a set of discrete orthogonalHermite functions defined on a lattice [8]. The zeroth order Hermite describes the underlying Gaussian in the matrix and hence the Gaussian noise in the image, while the higher orders describe the higher order structure of the matrix and hence the texture of the region. The definition of the Hermite functions and the decomposition of the matrix is discussed in greater detail in [7].3. IMAGE DATAThe co-occurrence matrix decomposition techniques defined above provide a low order feature vector descriptive of the texture of a region. The zeroth order coefficient describes the Gaussian noise while the higher orders describe the texture and are used as the ‘raw’ features in the classification analysis described in this paper.A total of twelve different objects were manually labelled in the FLIR video sequence. The total data consists of 2680 samples for these class labels- the individual breakdown of samples for each class label is shown in Table1. The object regions in the images are labelled by an expert in the area. Most class labels represent vegetation of some form. River reflecting sky occupies most of the image giving a large number of samples. For each sample,there are 121 feature measurements. These are reduced to a total of 50 most important selected by observing their discriminatory power with the discriminant analysis method. In our study, the data is scaled for all features to be within the [0,1] range.4. CLASSIFICATION SYSTEMThe recognition of natural objects using texture data is not a trivial task. Figure 1 shows the overlap between different classes. For each object data, we plot their overlap by calculating their principal components. The first two principal components represent the maximum variance. For each class, the mean and standard deviation for each principal components is calculated and a circle is plotted to show principal component score distributions across the different classes. For each class, its circle is represented by its means on the first two components defining the centre of the circle and if σ1 and σ1 denote the standard deviations on the components, then the circle radius r is given by:2.1σσ=r As shown, the distributions of different classes do overlap making the classification task difficult.The recognition of natural scene data is based on a ten fold cross-validation study [11,13]. A total of ten trials are conducted. In each trial, a training and a test set are constructed from the original data. The training data takes a partition of the whole data which represents 90% of the total, and the test data is made of the remaining 10%partition of total data. In each trial, a different partition is created so that training and test data are different from other trials. For each trial, called a fold, the classifier is shown training data and asked to recognise unseen test data. The recognition performance is measured in terms of recognition rate R in percentage that represents the ratio of total number of samples correctly recognised to the total number of samples tested.In our previous studies, we have widely used nearest neighbour methods for scene analysis data classification[9,12]. In this study, we have selected an artificial neural network. The three layer architecture (including the input layer) is fully connected and is trained using standard backpropagation with momentum (learning rate η=.1and momentum µ=.9) to a maximum of 2000 epochs. The neural network for each fold needs to be optimised to determine the correct number of hidden nodes. The process for this involves stepwise increase in model complexity by increasing hidden nodes and for each number selected recording of the generalisation error on unseen data. The model giving the least generalisation error is chosen as the best model. The generalisation and training errors for various number of hidden nodes for the ten different folds are shown in the Appendix. The training error is defined over the recognition of training data and generalisation error over the unseen test data. It can be observed that training errors are in general smaller than test or generalisation errors, and also they decrease more steeply in comparison as the number of hidden nodes is increased.5. REULTSThe results are shown for the ten folds in Table 2. In particular, the first and the last folds are particularly poor as shown by their models in the appendix. These models show saturation in performance with a small number of hidden nodes and the test performance does not improve no matter how much more the size of the hidden layer is increased. This can be attributed to the poor quality data available in these folds. The only method to tackle this poor quality data is to either remove sufficient outliers till the performance improves or filter noise using Fourier analysis. The other eight folds give very good results. Their average recognition result is 94.5%.However averaged over all ten folds, we get recognition rates of 91.8% correct. These are extremely good results considering the fact that the texture data for FLIR images is prone to picking up noise contamination and based on the large number of classes we are considering.6. CONCLUSIONIn this paper we have shown the performance of an artificial neural network on natural object recognition task. Excellent results have been obtained on discriminating textural data from twelve classes found in images used for this study. The results show that Hermite functions provide an effective methodology for texture measurement in FLIR imagery. These texture measurements are ideally suited inputs to neural network applications in autonomous scene analysis.REFERENCES[1] DC Becalick, Natural Scene Classification using a Weightless Neural Network, PhD Thesis, Imperial College, Department o Electrical and Electronic Engineering, 1996.[2] O. Firschein, Defence applications of image understanding, IEEE Expert, vol. 10, issue 5, pp. 11-17, (1995).[3] J.F. Haddon, J.F. Boyce, Integrating spatio-temporal information in image sequence analysis for the enforcement of consistency of interpretation, Special Issue of Digital Signal Processing, Oct 1998.[4] J.F. Haddon, J.F. Boyce, Image segmentation by unifying region and boundary information, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 12, No 10, pp 929-948, (1990).[5] R.M. Haralick, K. Shanmugan, I. Dinstein, Texture features for image classification, IEEE SMC-3, pp. 610-621, (1973).[6] R.M. Haralick, Image texture survey, in Handbook of Statistics, vol. 2, P R Krishnaiah, L N Kanal, Eds., pp. 399-415, 1982.[7] J.F. Haddon, J.F. Boyce, Co-occurrence matrices for image analysis, IEE Electronics and Communications Engineering Journal, vol. 5, No 2, pp. 71-83, (1993).[8] J.F. Haddon, J.F. Boyce, Spatio-temporal relaxation labelling applied to segmented infrared image sequence, Proceedings of 13th International Conference on Pattern Recognition, IEEE Press, Austria.[9] S. Singh, J.F. Haddon and M. Markou. Nearest Neighbour Strategies for Image Understanding, Proc. Workshop on Advanced Concepts for Intelligent Vision Systems (ACIVS'99), Baden-Baden, (2-7 August, 1999).[10] S. Singh, A Single Nearest Neighbour Fuzzy Approach for Pattern Recognition, International Journal of Pattern Recognition and Artificial Intelligence, vol. 13, no. 1, pp. 49-54, 1999.[11] M. Stone, Cross-Validatory Choice and Assessment of Statistical Predictions. Journal of the Royal Statistical Society.36 (1), 111-147, 1974.[12] S. Theodoridis and K. Koutroumbas, Pattern Recognition, Academic Press, 1999.[13] S. M. Weiss and C. A. Kulikowski,Computer Systems that Learn, Kauffman, CA, 1991.Figure 1. Image object class distributions:X-axis represents PC2 mean and Y axis representsPC1 mean. The radius of the circles are equal to1.96*s/d.Table 1 Recognition rates in percentage for a ten fold cross-validation study Neural networks have an architecture of 50x h x12Fold Hidden Nodes h Success%11982.222295.131093.242595.851390.962596.271997.082295.592592.410780.0Average -91.8Appendix: Training and generalisation error for the ten fold cross-validation- results for the 10 trials are shown。