FEATURE BASED REGISTRATION OF MULTISPECTRAL DATA-CUBES

FEATURE BASED REGISTRATION OF MULTISPECTRAL DATA-CUBESA.Broersen and R.van Liere Visualization and 3D InterfacesCWIKruislaan 413,1098SJ Amsterdam,The Netherlands {a.broersen,robert.van.liere }@cwi.nlABSTRACTImaging spectroscopy can be used to scan the structure of chemical elements on the surface materials.Multiple three-dimensional datasets are created that contain the chemi-cal properties on each location of the scanned surface.A manual pixel-based registration method was used to align the data-cubes to create one large mosaic of data-cubes.In this paper,we present a feature-based method for the registration of multispectral data-cubes.A feature is de-fined as a combination of important correlated chemical el-ements in the spectral distribution.One advantage of our feature-based registration is that the presence of noise in a data-cube is greatly reduced and,therefore,will result in a more robust registration.Another advantage is that mul-tiple attempts for registration can be performed using sev-eral extracted features to improve robustness.This paper describes our registration method and the conditions to in-dicate if extracted features contain sufficient information.We also compare the performance of our method with a pixel-based registration method.KEY WORDSRegistration,PCA,and multispectral data-cubes1IntroductionAn imaging mass spectroscope is a device that can acquire spectral distributions of chemical elements on the surface of a material.The output of the mass spectroscope is a multispectral data-cube;a three-dimensional dataset with spatial properties of the surface in the xy-dimension and the spectral distribution of a combination of chemical ele-ments in the z-dimension.In contrast to a volume consist-ing of 3D points of scalar values,a spectral dataset consists of two spatial dimensions and mass-to-charge ratio in the third dimension.Each scalar value in the volume is inter-preted as the intensity on a mass-to-charge ratio at a 2D position on the surface of a material.Material scientists of-ten refer to a spectral volumetric dataset as a multispectral data-cube .Compared to a high resolution microscopic image (see Figure 1a),a spectroscopic Total Ion Count -image (TIC-image)mosaic discloses the chemical composition of surface material on each location.This TIC-image in Fig-(a)(b)Figure 1.(a)Microscopic image and (b)spectrometric TIC-image mosaic of a droplet-deposition on a silicon substrateure 1b is created by taking the sum of all spectral intensities on a single two-dimensional location.Instead of the black background in between the crystallized droplets in the mi-croscopic image,the spectroscopic image provides spectra of the material in between the crystals as false color infor-mation in the TIC-image.Recently,more advanced tech-niques for data acquisition are able to sample a larger sur-face area by capturing a number of data-cubes on a higher resolution.State-of-the-art data-cube registration methods use TIC-images to create one large mosaic of data-cubes.These methods are very similar to pixel-based registration techniques found in many image processing handbooks,for instance [1].In this paper,we present a feature-based method for the registration of multispectral data-cubes.A feature is de-fined to be one of the many chemical elements in the spec-tral distribution.Since chemical elements have a unique and known spectral profile,scientists can use spectroscopy to investigate which elements are present on the surface of a material provided their spectral profile can be extracted from the data-cube.In a previous paper,we have shown how Principle Component Analysis (PCA)can be used for extracting specific spectral profiles from multispectral data-cubes [2].In the next section,we describe the methods for se-lecting features from a data-cube,registering these features,adding additional weights to areas of high entropy,and the complete algorithm.Section 3compares our feature reg-istration method with the pixel-based method on two test collections.Finally,we conclude with a discussion of the541-148543pros and cons of our method.2MethodOur registration method can be divided into three parts.In thefirst part,the data is reduced by selecting and extract-ing the most important features from the data-cubes using PCA.Then,by applying the mean squares metric to corre-sponding features in adjacent cubes,a minimalization land-scape is constructed.This landscape represents the’fit’of the feature when two adjacent data-cubes are aligned with each other.The minima provide the regions with the most similarities.Under various conditions,however,the land-scape does not provide sufficient information to robustly assume that the lowest value in the landscape is indeed the desired solution.These conditions can be described in the third step by analyzing the entropy of the landscape and by adding a weight to all of the possible solutions.The result-ing algorithm in subsection2.4gives an overview of how a solution is found in thefinal search space.2.1Principal Component AnalysisFirst,PCA is applied to the data-cubes to reduce the data-cubes by preserving the most important features into a number of spectral and their corresponding image compo-nents.There are many methods for decomposing multidi-mensional data or dimension reduction(see[3,4,5]),but these are better suited for decomposing multi-dimensional data in many independent features instead of in only a few components that describe most of the information.PCA can be used to compress the thousands of image planes into a few image components.It selects correlated spectral and spatial features in the data-cubes which results in different components.When sorted,thefirst few components de-scribe the most variance in the data and therefore provide the most contrast in the image components.PCA is used by unfolding a non-preprocessedλby x by y data-cube in such a way that a2Dλby x×y matrix X is constructed. The standard PCA model is used to compute a sorted list of principle components in an orthonormal matrix P(see Equation1)using eigenvector decomposition.Y=P·X T(1) Thefirst principle components in P describe those spatial regions in the data-cube with the greatest spectral variance.The original data-cube is projected using the principle components as bases resulting in a matrix Y with the spatial(Y images)or spectral(Y spectra)score vectors. Both these matrices are extracted and combined to be able to select similar spectral components from two data-cubes. Once two similar spectral components can be found in both pieces of the same collection,the corresponding image components(see Figure2)can be used for registration.For example,Figure2a shows an area in yellow at the bottom left corner which contains some high intensity values.A(a)(b)Figure2.(a)One image component and(b)an other spectrally matching image componentsimilar area can be found in Figure2b in the top left corner. The similarity between both regions could be measured us-ing an appropriate similarity metric.2.2Mean SquaresAlthough many metrics and applications already exist for the registration of images[6,7,8,etc.],the most basic ap-proach is used in this method.In most cases,there are no clear defined edges or distinguishing regions present in the resulting image ndmarking or region-based registration could be problematic when applied on this type of datasets.Most of these approaches use a metric to mea-sure the difference between intensity values between two regions in two images and an optimizer to transform one of the images according to afitness value tofind a betterfit. One image is called thefixed image and the image that is being transformed the moving image.Because these spec-tral data-cubes do not contain many spatial features and could contain some noise,the complete search space has to be considered tofind a suitable minimum and a correct offset between the two images.An optimizer is no longer needed,but this is only possible when neither image is too large and-even more important-when the moving image does not need any rotational,scaling or warping transfor-mations.The most simple pixel-based similarity measure is the Mean Squares(MS)measure.The mean squared pixel-wise difference is calculated using Equation2in which A is a region in thefixed image,B a region in the moving image,N the number of pixels in these regions and i the pixel position.MS(A,B)=1Figure3.Shifting two image components of256x256pixels to create a MS landscape of511x511pixels of each overlapping re-gion tofind the correct minimum represented by the dark colored pointsresult in high values of the metric with well-defined minima and sharp peaks.The metric is sensitive to clutter,occlu-sion and non-linear changes in contrast.It is not used in this case,because it does not perform considerably better than the MS metric on these multispectral datasets.2.3EntropyLooking at the complete MS landscape on the top right in Figure3,there are still many-black-areas with low val-ues.This is mainly caused by the fact that the intensities in certain regions do not have enough contrast and/or a high amount of randomness and just simply do not contain enough information to accurately use the difference metric. If the subregions consist of only one pixel(at the corners of the MS landscape),it is practically impossible tofind an accurate metric for the similarity.Image characteristics can provide some statistical information about the information in an image(see[9]).They can give an indication offit-ness of a complete image of a region in an image relative to the other image regions.This characteristic can be used to provide a weight for the MS landscape and create a more realistic search space in order tofind the most appropriate minimum.Some commonly used image characteristics are con-trast,correlation,energy and homogeneity,but in this con-text the measure for entropy in[10]is the most suitable to act as a weight.It provides an indication of the’random-ness’of intensities in an image using its histogram.The entropy H of each region in one image component is cal-culated using Equation3,in which A is the region in an image component and A p the histogram values of A.H(A)=− p A·log p A(3) The entropy of each region of thefixed image com-ponent can be calculated and put into a landscape corre-sponding with the MS landscape from Figure3.The en-tropy landscape of the moving image component can be(a)(b)Figure4.(a)Entropy space of all regions in the image compo-nent of Figure2a and(b)the standard deviationfitted entropy of the same image componentcreated similarly and combined with thefixed entropy land-scape using the entry-by-entry product of both entropy val-ues.The resulting combined entropy landscape of the im-age components in Figure2can be found in Figure4a.This combined entropy landscape contains some outliers, mostly located at the corners where only a few pixels are being considered.An additionalfit on the histogram is made to remove unwanted outliers.This is done according to the normal distribution,by using only the mean values including the standard deviation in the histogram.The val-ues in the histogram left of this area are set to zero and the values to the right of this area are set to the remaining max-imum.The remaining landscape(see Figure4b)is applied as a weight for the MS landscape.The black regions in entropy spaces indicate that a minimum found in the corresponding regions of the MS landscape most likely will not contain enough information to be considered as a solution.Thefinal weighted search space can be created by dividing all values in the MS land-scape by thefitted entropy landscape.This results in a MSentropysearch spaces can be compared in order tofind the correct offset.1.PCA on two data-cubes from one collection of mea-surements which results in a decomposition of eachdata-cube with a number of corresponding spectraland image components.2.Select matching spectral components of both data-cubes by comparing of spectral peaks,starting fromthe components describing the most variance.3.Registration of the corresponding selected imagecomponents from both data-cubes by creating a mean 545(a)(b)Figure5.The relative locations of(a)four pieces of the crystal dataset and(b)five pieces of the kneecap datasetsquares landscape of all possible combinations of re-gions in the image components.4.Add a weight function to the MS landscape using anentropy image characteristic applied to each region of both the image components to exclude regions that contain not enough spatial information to be consid-ered as a solution.5.Select the lowest points in thefirstfive regions with lo-cal minima in the MSentropy metric which is used in the PCA-based approachto be able to quantitatively compare both approaches.The crystal dataset consists of four data-cubes resulting in six combinations,in which there is an overlap with every other data-cube.The kneecap dataset consists offive data-cubes resulting in ten combinations,in which seven combinations do have an overlap and three of them do not.The results of this registration can be found in the next subsection.All combinations in each dataset are being registered using this new PCA-based method.First,a PCA is con-ducted on each data-cube using the algorithm from subsec-tion2.4.Similar spectral components are being matched and each corresponding image component(see Figure2)is used for the registration.This results in several MS land-scapes(see Figure6)for each matching spectral component in which minima have to be found to get a most appropriate solution for a registration.The entropy space(see Figure4)of each component image is calculated and combined into the MSentropy search space of thefirst spec-trally matched components of two data-cubes.And Fig-ure7b shows the MS(a)(b)Figure6.Mean Squares landscape from comparing the(a)first and the(b)second principal component of C and E of the kneecap dataset in which the location of the correct solution for the regis-tration is marked(a)(b)Figure7.MSentropy search space could be considered for eachcombination of data-cubes.So the global minimum in the search space was used as a solution for a bestfit.ThereTIC-based array of crystalsfound foundglobalfit02-3 Table1.Summary of the results from the TIC-based approachPCA-based array of crystalsfound foundcorrectfit55-1-0 Table2.Summary of the results from the PCA-based approachwere only two correct offsets successfully found within the kneecap dataset using the global minimum in the MSentropysearch space was used and its global minimum was considered as a solution.The three com-binations in the kneecap dataset without any overlap were correctly found as indicated in the row with the’incorrect fit’.Two offsets that did exist could unfortunately not be found.This was caused by the lack of entropy in regions that should provide for a correctfit.The remaining four of six existing offsets were found and one was found,but was incorrect.This solution was found by comparing two regions without much intensity information.The joint en-tropy was not low enough to dismiss it as a possible solu-tion.4ConclusionThe results show that the PCA-based approach for the reg-istration of a collection of multispectral data-cubes is supe-rior to a traditional TIC-based method.Pixel-based regis-tration of selected image components using the MS metric with a complete coverage of the search space and an ad-ditional entropy weighting is able to correctly register two data-cubes if a solution exists.We canfind a more robust solution with the multiple minima from the landscapes of several extracted features instead of using only one pair of images.Some remarks can be made about the metrics used in this method with the possibility to improve and/or opti-mize the algorithm.4.1DiscussionThere was only one case in the kneecap dataset in which the solution found by the algorithm was incorrect.This547was mainly caused by the fact that the overlapping regions with the found minimum did not contain enough inten-sity information.Unfortunately,the entropy did not con-tribute enough as a weight to eliminate it as a solution in the MSentropy searchspace.The MS metric was not able tofind an unambiguous offset to register the TIC-images,while they can be found with the PCA-based approach.So even if some of the data-cubes do not have much overlap or useful spatial features in those overlapping regions,it is still possible to create a complete mosaic when the complete MS。