自然语言处理技术在生物信息学中的应用_英文

自然语言处理技术在生物信息学中的应用徐继伟(哈尔滨工业大学计算机科学与技术学院,150001)摘要:从信息处理的角度来看,生物信息学与自然语言处理中的许多问题是非常相似的,因此,可以将一些自然语言处理中的经典方法应用到生物信息学文字中。

本文介绍了自然语言处理和生物信息学中共有的问题,如比对、分类、预测等,以及这些问题的解决方法。

通过对两个领域形似问题的分析可知,优秀的自然语言处理技术也可用来解决生物信息学方面的问题,并且一些还未在生物信息学领域得到应用的自然语言理解技术也有其潜在的应用价值。

最后给出了一个分类问题的解决方案,演示了如何在生物数据上应用算法进行实验。

关键词:生物信息学;自然语言处理;比对;分类中图分类号:TP312,Q811.4 文献标识码:A 文章编号:1672-5565(2006)-01-041-04收稿日期:2005-01-21;修回日期:2005-02-25作者简介:徐继伟,哈尔滨工业大学计算机科学与技术学院,e -mail :jiweixu @Applications of natural language processing techniques to bioinformaticsX U Ji -wei(Department o f Computer Science ,Harbin Institute o f Technology ,Harbin 150001,China )Abstract :Many problems of bioin formatics are very similar to those of Natural Language Processing (N LP )from the view point of In formation Processing ,thus classic methods of N LP can be applied to Bioin formatics problems.Several comm on problems of Bioin formatics and N LP and their s olutions are introduced in this paper ,such as alignment ,classification and prediction problems.The discussion of the similarities suggest the usefulness of N LP methods in Bioin formatics problems and its potential applications in Bioin formatics.A step -by -step s olution of classifi 2cation is given to explain how to per form experiment with biological data.K ey Words :Bioin formatics ;Natural Language Processing ;Alignment ;Classification1　IntroductionBioin formatics is a novel research field of interdisci 2plinary sciences ,which integrates biology ,com puter sci 2ence ,applied mathematics and in formation technology in 2to a single discipline.With the development of Bioin for 2matics ,scientists can understand the protocol of biology w orld and s olve many current problems in biology.H owev 2er ,with the rapid accumulation of biological sequence da 2ta ,there is an exigent need to w ork out scientific com pu 2tational methods and advanced Bioinformatics techniques for reliable and large -scale biological data processing and analysis.Biological sequence processing is similar to text processing in N LP ,as shown in Figure 1,if biologi 2cal sequences being processed and analyzed are treated as strings ,regardless of their physical 3D structures.S ome Bioin formatics scientists have already applied N LP tech 2niques to their research ,and the performances of these methods show that many N LP techniques are applicable in Bioin formatics.2　Basic C onceptsAlthough people in Bioin formatics process biological data such as DNA ,RNA ,protein ,etc.and people in N LP process real -w orld texts from news ,reports ,pa 2pers ,speeches ,etc.,the real “objects ”to be manipulat 2ed in both fields are same.It is reas onable to believe that生物信息学　China Journal of Bioinformatics 专论与综述Figure1:M apping betw een hum an langu age andProtein Sequences(http:ΠΠ)there are s ome comm on problems and methods in both re2 search fields;however,different idioms exist in these tw o fields,for the convenience of readers,s ome frequently used concepts and notations are recalled here.2.1　N-gramA consecutive substring of n symbols,with n≤L (here L is the length of the whole string),is called an n -gram(in s ome references Bioinformatics is als o defined as n-w ord or n-tuple).The statistic of n-grams is usually obtained by traversing the string of length L with an n-wide sliding windows,from position1to L-n+1, each content in an n-wide window is an n-gram.There have been a set of tools for extracting both w ord n-grams (Latin lanuage such as English)and character n-grams (Eastern language such as Chinese)from raw corpora,and they are free for downloading at http:ΠΠw w ΠzhangleΠngram.html#download.2.2　Vector space m odelThe perhaps m ost comm only used data representation in in formation processing is the s o called Vector S pace M odel(VS M)(Salton et al.1983),and many machine learning methods are based on VS M,such as K-NN (Masand et al.1992),S VM(Vapnik1999;Joachims 1998),Na1¨ve Bayes(T zeras et al.1993;Decision T rees (Lewis et al.1994),neural netw orks(Wiener et al.1995) and s o on.The idea of VS M is using vector<W1,W2,…W n>to represent sequences or texts,where W i is the weight of ith feature of the sequences.F or exam ple,in se2 quence X=“AT CGG CTT”with trigram,the feature can be any element from{AT C,T CG,CGG,GG C,G CT, CTT}.G iven a collection of sequences and selected fea2 tures,then these sequences can be represented by a fea2 ture-by-sequence matrix M,where each entry repre2 sents the weight of a feature in a sequence.The weight of a feature can be calculated by one of the weighting schemes,such as tf-idf weighting,tfc-weighting and s o on.The tw o basic principles for weighting are(1)the m ore times a feature appears in a sequence,the closer relation it is to the sequence;(2)the m ore times a feature appears in the collection of all sequences,the m ore poorly it dis2 criminates between sequences.3　Similar problemsThere are several similar problems in Bioinformatics and N LP,s o knowing the s olution of one might be helpful to the other;there are als o s ome comm only used methods in both fields,and we can learn to apply the skillfully used methods to new problems.We will introduce these similar topics in this section and give a step-by-step case study of classification in the next section.3.1　AlignmentSequence Alignment in Bioin formatics and Bilingual C orpora Alignment in N LP are tw o problems that are the m ost similar to each other,they are both for helping peo2 ple understand unknown things from the known ones,that is,to help people understand English sentences with the help of the Chinese ones or help people know the function of new protein sequences with the help of known ones.S A can be applied to DNA and protein sequence database searches,m otif searches,gene identification searches and the analysis of multiple regions of similarity in long DNA sequences;it’s very useful for finding the hom ology of known sequence family and providing im portant informa2 tion about the function of newly sequenced genes.On the other hand,BC A plays im portant role in com putational language research,especially in Machine T ranslation, and is believed to be a promising direction for s olving knowledge acquisition-the bottle-neck of N LP.Looking deeply into the tw o problems,we can see other similarities.F or exam ple,S A can be sim ply classi224 生　物　信　息　学 第3卷fied to global alignment and local alignment while BC A includes paragraph alignment,sentence alignment and w ord alignment;the measure methods of S A and BC A are als o similar,s o there remains space for people in Bioin2 formatics to learn BC A methods from machine translation and im prove the quality of sequence alignment.C omm only used sentence alignment methods include length based alignment(Brown et al.1991);Part of S peech based alignment(K ay et al.1993)and biligural dictionary based alignment(Wang1999).Here the func2 tion of bilingural dictionary is similar to that of the matrix of PAM and BLOS UM.Furture m ore,the combination of above methods can im prove the performance(Wu1994; Simard et al.1992)3.2　ClassificationAnother comm on problem in Bioin formatics and N LP is classification.Rapidly increasing data of newspapers, digital libraries and the Internet are in great need of auto2 matic classification systems.The outburst of biological da2 ta,such as DNA and protein sequences,als o need classi2 fication to know their categ ories,we can see from the pro2 tein class tables in http:ΠΠmips.gs f.deΠgenreΠprojΠyeastl that a large percent of proteins are now in unknown class, s o we need to know their categ ories and functions with the help of effective classification systems.Many statistical classification and machine learning techniques have been applied to text class fication,includ2 ing nearest neighbour classifiers,decision trees,Bayesian classifiers,neural netw orks and Support Vector Machines. These methods are suitable for the task of classification. Additionally,people tried to combine several techniques together for im proving classification performance,for ex2 am ple,(Li et al.1991)presented a new method for web-page classification combining S VM and unsupervised clus2 tering and the combining with unsupervised clustering ob2 tained a better classification accuracy.3.3　Other potential applicationOther techniques in N LP,such as W ord Sense Dis2 ambiguation(Ide et al.1998)and Dependency G rammar Analysis(Liu1997),although haven’t been applied in Bioin formatics problems,might be valuable for reference by biologists in future.WS D in N LP is to determine the meaning of w ords that spelled the same but have different meanings in dif2 ferent contexts,for exam ple the w ord bank have different meanings in the following sentences:(1)I take a walk along the bank every m orning.(2)I bank at the National Bank.WS D can help a lot with the problems of natural lan2 guage understanding and machine translation.In Bioin for2 matics,it’s als o possible that snippets with the same structure function quite differently in different protein se2 quences,s o WS D techniques might be applicable in fu2 ture.Dependency G rammar Analysis problem is to find the dependency relationship between w ords and phrases; DG A can find that the w ords according and to have a high dependency with each other,and this dependency are useful in error detection:for exam ple in the sentence with error:According too my watch it is10o’clock. The error detection system can easily find the error with the help of dependency information.In Bioinformatics,a single snippet of sequence might not function correctly without the help of another snippets,that is to say,the snippets might have dependency relationship with each other,and only the co-occurrence of them can lead a biological function,in this way,DG A technique can be applied.There are other similar problems in the tw o research field,such as named entity detection in N LP and drag target detection in Bioinformatics,and we will discuss them in other papers instead of this one due to the space limitation.4　Case studyAs mentioned above,with the accelerated accumula2 tion of biological data,proteins are in great need of clas2 sification systems.We use RNA-binding protein classifi2 cation as the exam ple here to show how to perform experi2 ments with biological data.The RNA-binding proteins are downloaded from protein sequence database S wissport (http:ΠΠw w w.expasy.chΠsprot),and the classifier can be built with any method mentioned in Section3.2.we will34第4期 徐继伟,等:自然语言处理技术在生物信息学中的应用 give description of data set division experimental results and discussion in following paper in detail.The protein classification here is a supervised learn 2ing task ,defined as assigning categ ory labels that are pre -defined to new proteins based on the likelihood ob 2tained by a training set of labeled proteins.Assume pro 2teins P 1,P 2,…and P n consist a training set labeled Pro 2teases and P t is a new protein ,if the comarasion of P t with all P 1,P 2,…and P n get a high score ,then this clas 2sification determines that P t belongs to class Proteases and assigns label Proteases to P t .The architecture of classifi 2cation system isshown in Figure 2:Figure 2:architecture of classification systemIn Figure 2,the features might be the collection of n -grams mentioned above or any other form user defined.Both the labeled sequences and the new sequences are represented as vectors with the same features before learn 2ing and classifying ,the experiment includes these steps :(1)extract features ,such as n -grams ,from the la 2beled RNA -binding proteins and store them.Features do not contribute equally to the function of proteins ,and high dimensional vector space will degrade the system performance ,s o additional process is needed to eliminate useless features and reduce the feature dimension.(2)build the vector representation of labeled proteins with the features after dimension reduction in step (1).(3)learn and build the classifier with the vectors ob 2tained in step (2).(4)build the vector representation of new (testing )proteins with the features after dimension reduction in step (1).(5)classify with the classifier built in step (3)and the vector representation of new proteins built in step(4),and assign labels to the new proteins according to the likelihood obtained.5　C onclusionIn this paper ,s ome similarities in Bioinformatics and Natural Language Processing are discussed ,such as the problems of alignment and classification.Many exper 2imental results show that N LP techniques are applicable for s olving Bioin formatics problems.Further study on N LP techniques will help the development of Bioinformatics ,and we believe that s ome N LP techniques that haven ’t been applied to Bioinformatics will be useful tools for Bioin formatics in future.References :[1]　Brown ,P.F.,Lai ,J.C.and M ercer ,.R.L.Aligning Sentences inparallel C orpora[R].Proc.O f the 29th Annual M eeting of the AC L ,1991169-176.[2]　Ide ,N.and Veronia ,J.Introduction to the special issue on w ord sensedisambiguation:The state of the art [J ].C om putational Linguistics ,1998,24(1):1-40.[3]　Joachims ,T.T ext Categ orization with support vector machines :Learn 2ing with many relevant features[R].In Proc.10th European C on ferenceon M achine Learning (EC M L ),S pringer Verlag 1998.[4]　K ay ,M.and R ocheisen ,M.T ext -T ranslation Alignment [J ].C om pu 2tational Linguistics ,1993,19(1):121-142.[5]　Lewis ,D.D.and Ringuette ,M.C om paris on of tw o learning alg orithmsfor text categ orization[R].In Proceedings of the Third Annual Sym pos 2ium on D ocument Analysis and In formation Retrieval (S DAIR ’94),1994.[6]　Liu ,H.T.“Dependency G rammar and M achine T ranslation.”[J ](in Chinese ),Language Application ,1997,3,89-93.[7]　M asand ,B.,Linoff ,G.and W altz ,D.Classifying news stories usingmem ory based reas oning[R].In 15th Ann Int AC M SIGIR C on ference on Research and Development in In formation Retrieval (SIGIR ’92),1992,59-64.[8]　T zeras ,K.and Hartman ,S.Automatic indexing based on Bayesian in 2ference netw orks[R].In Proc 16th Ann Int AC M SIGIR C on ference on Research and Development in In formation Retrieval (SIGIR ’93),1993,22-34.[9]　W ang ,B.“Chinese -English Biligural C orpora Automatic Alignment Research ”(in Chinese )[D].Ph.D thesis ,1999.[10]　W iener ,E.,Pedersen ,J.O.and W eigend ,A.S.A neural netw ork ap 2proach to topic spotting [R ].In Proceedings of the F ourth Annual Sym posium on D ocument Analysis and In formation Retrieval (S DAIR ’95),1995.[11]　Wu ,D.Aligning a Parallel English -Chinese C orpus S tatistically withLexical Criteria.Proc[R ].O f the 32nd Annual C on ference of AC L ,1994,80-87.44 生　物　信　息　学 第3卷。