模式识别与机器学习综述

Machine Learning and Pattern Recognition1. Basic Introduction1. Machine LearningLearning is a very important feature of intelligent behavior. But the machine learning . H.A.simon believes that learning is adaptive changes made to the system, making the system more effective when completed the same or similar tasks next time. R.s.Michaiski said that learning is to construct or modify things for experienced representation .People engaged in the development if Expert-Systerm think that learning is the acquisition of knowledge. These views have different emphases. The first emphasizes the effect of the external behavior of learning, the second emphasizes the internal processes of learning and the third is mainly from the practical point of departure of knowledge engineering.Machine learning has a very important position in the study of artificial intelligence. An intelligent system that does not have the ability to learn is difficult to be called a real intelligent systems. But in the past generally intelligent systems lack the capacity to learn. For example, they can not be self-correcting when errors are encountered; do not improve their performance through experience; does not automatically obtain and discovery the required knowledge . They are limited to deductive reasoning and lack of induction, so it can only prove the fact and theorem witch have existed, and do not discovery a new theorems, laws and rules. With the development of artificial intelligence, these limitations behave even more prominent. It is in this case, machine learning has gradually become one of the core of artificial intelligence research. Its application has been throughout all branches of artificial intelligence, such as expert systems, automated reasoning Natural language understanding, pattern recognition, computer vision, intelligent robotics and other fields. In particular, the typical expert system knowledge acquisition bottleneck problem, people have been trying to try to use machine learning methods to overcome them.The research of machine learning is based on the understanding of physiology and cognitive science ,build model or models of human understanding of the learning process, develop Various learning theories and learning methods; Learning algorithm research and general theoretical analysis, establish learning system withapplication-specific task-oriented. The goal of this study affect each other and promote each other.2. Pattern RecognitionPattern recognition is a fundamental human intelligence , in everyday life , people often conducting " pattern recognition ." With the riseof the computer artificial intelligence occurred in the 1940s and 1950s , Of course, one also hopes to use the computer to replace or extend the mental part ofhumanity . Pattern Recognition rapid and become a new discipline in the early 1960s . Pattern recognition means ( text and logical relationships between the values of ) the characterization of the various forms of objects or phenomena information processing and analysis to describe phenomena or things , identification , classification and interpretation process , information science and an important part of artificial intelligence .Pattern recognition research focuses on two aspects, One study of objects ( including people ) is how we perceive objects belonging to the scope of scientific knowledge , the second is given the task of how to use a computer to implement the theories and methods of pattern recognition . The former is a research physiologist , psychologists , biologists and neurophysiologists , the latter through the mathematicians , informatics experts and computer scientists ,in recent decades efforts have been made in this research system. Using computer to identify or classify a process.Events or processes may be identified by text, sound, images, and other specific objects, these objects are different from the information in digitalform is called pattern information.The number of classes of pattern recognition classification is determined by the specific identification problems. Sometimes, you can not know the actual number of classes at the start, you need to identify the system after repeated observations to determine the object to be identified.Pattern recognitions have a relationship with statistical pattern recognition, psychology, linguistics, computer science, biology, cybernetics, etc. It also has relations with artificial intelligence research, image processing. Such as adaptive or self-organizing pattern recognition system contains artificial intelligence learning mechanisms; scene understanding of artificial intelligence research, natural language understanding also includes pattern recognition problems. Another example is pattern recognition preprocessing and feature extraction, image processing applicationstechnology sectors; image processing pattern recognition, image analysis techniques are applied.2. The relationshipPattern recognition is derived from engineering, it is a kind of problem (problem); Machine learning derived from mathematics, is a kind of method (methodology). For a specific pattern recognition problems, you can use handcrafted rule - -based approach to solving, but more complicated PR problems often adopt the method of machine learning. 1. The classification of machine learningAccording to the study of different pattern, the machine learning in general can be divided into four categories:Supervised LearningTraining set with all the input of the target value is called supervised learning.This study aims to find the relationship between the input variables and target. According to the target value, supervised learning can be divided into two types of problems: if the target value is a discrete variable, called classification; If it is a continuous variable, called regression.Unsupervised LearningAll input variables without a target value are called unsupervised learning. This study aims to find the internal links between the input variables. According to the specific internal contact type, unsupervised learning also can be divided into a variety of problems, such as clustering, density estimation, the visualization.Semi - supervised LearningInput variables some have the target value and some have no is called semi -supervised learning. In fact, the book did not refer directly to semi - supervised learning.Reinforcement learningThis type of learning is on the basis of supervised learning, it allows the machine to choose training data. At the same time, training in access to information at the same time also can bring the cost or loss, triggeringa tradeoff.2. The basic process of machine learningThe most basic of the machine learning process is:1. Determine the type of model2. To determine the model complexity (i.e., the number of free parameters)3. Make sure all the parameters of each model4. Finally, the comparison between model one of the best choice, also known asmodel selectionTrainning model generally refers to one or a set of analytical expressions, through which can use the analytic method to express knowledge or direct optimization decision. According to the generalization ability is different, trainning model can only face the step 3, also can at the same time to cover all 4 steps. Trainning cannot express for the part of the model, either through conputational algorithm to enumeration or more, or to specific application model assumptions. Over fitting and Model SelectionThe specific meaning of over fitting description is not clear in the book, generally refers to such a phenomenon: sometimes the model error on the training set is very small, but a large error on the training set.If algorithm over fitting phenomena, traditionally to choosing asubset of the training data (called the validation set), and based on the validation set to do the model selection.The difference between the validation data and test data is that the former can be in another part of the run of the training data, different run for the same set of data to take a different training - to divide the validation; While the latter is not used in the process of training, is usually used in the experiment.The validation of the faults are two:1. The validation takes up extra training data, the data in the application are particularly affected. Cross - validation technology is used to alleviate the defects. It in turn to select a small number of data from the training set to do the validation set, and finally to combine multiple results. But cross - validation to introduce several rounds of the validation, increased the amount of calculation.2. As a result of the existence of the validation, training model can not according to the training data parsing model comparison. When need to enumerate compare model complexity parameters change, the validation of miscellaneous complexity index rose.3. Pattern recognition methodDecision theory methodAlso known as statistical method, and it is a way that the develop early. Identify the object first and make digital transformation to fit the computer . A model is often represented with a large amount of information. Many pattern recognition systems in the digital link after preprocessing, also used to remove the interference with information and reduce some deformation and distortion. After, that is followed for feature extraction from digital or after preprocessing of input patterns extracted a set of features. So-called feature is the measure of a selected it for normal deformation and distortion remains the same or almost the same, and only contain redundant information of as little as possible. Feature extraction process map the input mode from the object space to feature space. At this time, the model can be used a point in the feature space or a feature vector representation. This mapping not only compress the amount of information, and easy to classification. In decision theory method, the feature extraction is very important, but there is no general theoretical guidance, only by analyzing specific recognition object the feature selection. Feature extraction can be carried out after the classification, that is, from the feature space and then mapped to the decision space. For introducing the identification function, calculated from characteristic vector corresponding to various other identification function value, the categorized by identifying function values.Syntactic methodsAlso called structure method or linguistics. Its basic idea is to puta model described as the combination of simpler subpattern, sub model and can be described as the combination of simpler subpattern, end up with a tree structure to describe, at the bottom of the simplest primitive subpattern called mode. In syntactic approach selected primitive problem is equivalent to select characteristic problems in decision theory method. Usually requires the selected primitives reflects on model provides a compact description of its structural relationship, and easy to use the syntax method to extract them. Obviously, the primitive itself should not contain important structural information. Model to a set of motifs and to describe their combination relationship, referred to as the model to describe the statement, the same as in the language sentence combination of words and phrases, words with the same character combinations. Primitive rules combined into patterns, by so-called syntax to specify. Once primitives have been identified, the identification process can be through the syntactic analysis, the analysis of the given pattern statement is in line with the specified syntax, meet is of a kind of grammar points in the class. Choice depends on the nature of the problem of pattern recognition method. If the identified object is very complex, and contains rich structural information, general syntax methods; By the object recognition is not very complex or do not contain obvious structural information, generally USES decision theory method. These two methods cannot be separated, in syntactic approach, primitive itself is made of decision theory method to extract. In application, combine the two methods were applied to different levels, often can get good effect.Statistical pattern recognitionStatistical pattern recognition (statistic pattern recognition), the basic principle is: the similarities of the samples in the pattern space close to each other, and form a "group", namely "birds of a feather flock together." Its analysis method based on pattern characteristic vector measured by Xi = (xi1, xi2,..., xid) T (I = 1, 2,..., N), will be under a given pattern C class 1 omega, omega 2,... , omega c, and then according to the distance between the model function to discriminant classification. Among them, T transposed; N as sample points; D characteristic number for sample. Statistical pattern recognition is the main method is: discriminant function method, near neighbour classification, nonlinear mapping method, characteristic analysis, the main factor analysis method, etc.In statistical pattern recognition, bayesian decision rule is theoretically solved of optimal classifier design problem, but its implementation must first solve probability density estimation problem more difficult. BP neural network directly from the observed data (the training sample) to study, is more simple and effective method, thus obtained the widespread application, but it is a kind of heuristic techniques, the lack of a specified engineering practice solidtheoretical basis. Breakthrough in statistical inference theory research in cause of modern theories of statistical learning theory, VC, the theory is not only based on strict mathematical successfully answered the problem of the theory of artificial neural network and deduced a new learning method called support vector machine (SVM).。