Linear Dimensionality Reduction

合集下载

孪生网络入门（上）SiameseNet及其损失函数

孪⽣⽹络⼊门（上）SiameseNet及其损失函数最近在多个关键词（⼩数据集，⽆监督半监督，图像分割，SOTA模型）的范畴内，都看到了这样的⼀个概念，孪⽣⽹络，所以今天有空⼤概翻看了⼀下相关的经典论⽂和博⽂，之后做了⼀个简单的案例来强化理解。

如果需要交流的话欢迎联系我，WX：cyx645016617所以这个孪⽣⽹络⼊门，我想着分成上下两篇，上篇也就是这⼀篇讲解模型理论、基础知识和孪⽣⽹络独特的损失函数；下篇讲解⼀下如何⽤代码来复线⼀个简单的孪⽣⽹络。

1 名字的由来孪⽣⽹络的别名就会死Siamese Net，⽽Siam是古代泰国的称呼，所以Siamese其实是“泰国⼈”的古代的称呼。

为什么Siamese现在在英⽂中是“孪⽣”“连体”的意思呢？这源⾃⼀个典故：⼗九世纪泰国出⽣了⼀对连体婴⼉，当时的医学技术⽆法使两⼈分离出来，于是两⼈顽强地⽣活了⼀⽣，1829年被英国商⼈发现，进⼊马戏团，在全世界各地表演，1839年他们访问美国北卡罗莱那州后来成为“玲玲马戏团” 的台柱，最后成为美国公民。

1843年4⽉13⽇跟英国⼀对姐妹结婚，恩⽣了10个⼩孩，昌⽣了12个，姐妹吵架时，兄弟就要轮流到每个⽼婆家住三天。

1874年恩因肺病去世，另⼀位不久也去世，两⼈均于63岁离开⼈间。

两⼈的肝⾄今仍保存在费城的马特博物馆内。

从此之后“暹罗双胞胎”（Siamese twins）就成了连体⼈的代名词，也因为这对双胞胎让全世界都重视到这项特殊疾病。

2 模型结构这个图有这⼏个点来理解：其中的Network1和Network2按照专业的话来说就是共享权制，说⽩了这两个⽹络其实就是⼀个⽹络，在代码中就构建⼀个⽹络就⾏了；⼀般的任务，每⼀个样本经过模型得到⼀个模型的pred，然后这个pred和ground truth进⾏损失函数的计算，然后得到梯度；这个孪⽣⽹络则改变了这种结构，假设是图⽚分类的任务，把图⽚A输⼊到模型中得到了⼀个输出pred1，然后我再把图⽚B输⼊到模型中，得到了另外⼀个输出pred2，然后我这个损失函数是从pred1和pred2之间计算出来的。

几种降维思想方法总结

几种降维思想方法总结降维（Dimensionality Reduction）是机器学习与数据挖掘领域非常重要的一个技术，其主要目的是剔除无关信息，提取数据的本质特征，并将高维数据映射到低维空间进行处理。

降维方法主要有两大类：特征选择和特征提取。

在本文中，我们将总结几种常见的降维方法思想。

1. 主成分分析（Principal Component Analysis，PCA）PCA是一种最常见的降维方法，其思想是将原始特征通过线性变换，得到一组新的互不相关的特征，这些新特征被称为主成分。

主成分的选择是根据方差来确定，保留方差多的特征，可以更好地保留数据的信息。

通过选择合适的主成分数目，我们可以将原本具有很高维度的数据映射到一个低维的空间中，从而达到降维的目的。

2. 因子分析（Factor Analysis）因子分析是另一种常用的降维方法，它假设观测数据是由一组潜在的因子（Latent Variables）决定的。

通过寻找这些潜在因子，我们可以在保留数据信息的前提下，将原始特征的维度降低。

因子分析可以用于探索数据之间的潜在关系，还可以用于处理带有缺失值的数据，通过估计缺失值进行降维。

3. 独立成分分析（Independent Component Analysis，ICA）ICA是一种基于统计独立性的降维方法，它假设观测数据是由相互独立的成分组成的。

与PCA不同，ICA并不追求数据的方差最大化，而是追求数据之间的独立性。

ICA可以将观测数据分解为多个互不相关的独立成分，从而实现数据的降维。

4. 线性判别分析（Linear Discriminant Analysis，LDA）LDA是一种用于分类问题的降维方法，它通过将原始数据投影到低维空间中，从而最大限度地保留数据的类别信息。

LDA寻找一个最佳投影方向，使得同一类别的数据尽可能接近，不同类别的数据尽可能分开。

通过LDA降维，可以在不降低分类性能的情况下，减少数据的维度。

线性代数合同的英语

线性代数合同的英语Linear algebra is a fundamental branch of mathematics that has numerous applications in various fields such as physics, engineering, computer science, and economics. One of the key aspects of linear algebra is the concept of contracts which are used to represent and manipulate linear relationships between variables. In this essay, we will delve into the intricacies of linear algebra contracts and explore their significance in various contexts.At the core of linear algebra are matrices, which are rectangular arrays of numbers or other mathematical objects. These matrices can be used to represent linear transformations, systems of linear equations, and other mathematical structures. A linear algebra contract is a way of expressing these linear relationships in a concise and organized manner.One of the primary uses of linear algebra contracts is in the representation of systems of linear equations. Consider a set of n linear equations in m variables, where each equation can be expressed in the form:a11x1 + a12x2 + ... + a1mxm = b1a21x1 + a22x2 + ... + a2mxm = b2...an1x1 + an2x2 + ... + anmxm = bnThese equations can be represented using a matrix equation of the form Ax = b, where A is the coefficient matrix, x is the vector of variables, and b is the vector of constants. The linear algebra contract in this case would be the matrix A, which encapsulates the linear relationships between the variables and the constants.Another important application of linear algebra contracts is in the study of linear transformations. A linear transformation is a function that maps vectors in one space to vectors in another space, while preserving the linear structure of the vectors. These transformations can be represented using matrices, and the matrix itself can be considered a linear algebra contract that defines the transformation.For example, consider a linear transformation T: R^n -> R^m, which maps vectors in R^n to vectors in R^m. This transformation can be represented by a matrix A of size m x n, where the (i,j)th element of A represents the coefficient of the jth variable in the ith equation of the transformation. The linear algebra contract in this case would be the matrix A, which fully defines the linear transformation.Linear algebra contracts also play a crucial role in optimization problems, where the goal is to find the optimal solution to a problem subject to a set of constraints. These constraints can often be expressed as linear equations or inequalities, and the linear algebra contracts involved in the problem formulation are essential for solving the optimization problem efficiently.In the field of machine learning, linear algebra contracts are extensively used in the development of various algorithms and models. For instance, in linear regression, the goal is to find a linear relationship between a set of input variables and a target variable. The linear algebra contract in this case would be the coefficient matrix that defines the linear relationship.Similarly, in principal component analysis (PCA), a widely used dimensionality reduction technique, the linear algebra contract is the covariance matrix of the input data, which is used to identify the principal components that capture the most significant variations in the data.The applications of linear algebra contracts extend beyond the academic realm and into the realm of industry and commerce. In the field of finance, linear algebra contracts are used in the pricing and risk management of financial instruments, such as bonds, derivatives,and portfolio optimization.In the engineering field, linear algebra contracts are crucial for the design and analysis of complex systems, such as electrical circuits, mechanical structures, and control systems. These contracts are used to model the relationships between various components and variables, allowing engineers to optimize the performance and reliability of their designs.In conclusion, linear algebra contracts are a fundamental concept in the field of linear algebra, with widespread applications across various domains. These contracts provide a concise and organized way of representing and manipulating linear relationships, enabling researchers, engineers, and decision-makers to solve complex problems efficiently and effectively. As the world becomes increasingly data-driven, the importance of linear algebra contracts will only continue to grow, making it an essential tool for understanding and shaping the world around us.。

深度学习常见的专业术语

深度学习常见的专业术语（部分内容转载⾃⽹络，有修改）1. 激活函数（Activation Function）为了让神经⽹络能够学习复杂的决策边界（decision boundary），我们在其⼀些层应⽤⼀个⾮线性激活函数。

最常⽤的函数包括sigmoid、tanh、ReLU（Rectified Linear Unit 线性修正单元）以及这些函数的变体。

2. 优化器3. 仿射层（Affine Layer）神经⽹络中的⼀个全连接层。

仿射（Affine）的意思是前⾯⼀层中的每⼀个神经元都连接到当前层中的每⼀个神经元。

在许多⽅⾯，这是神经⽹络的「标准」层。

仿射层通常被加在卷积神经⽹络或循环神经⽹络做出最终预测前的输出的顶层。

仿射层的⼀般形式为 y = f(Wx + b)，其中 x 是层输⼊，w 是参数，b 是⼀个偏差⽮量，f 是⼀个⾮线性激活函数。

4.5. AlexnetAlexnet 是⼀种卷积神经⽹络架构的名字，这种架构曾在 2012 年 ILSVRC 挑战赛中以巨⼤优势获胜，⽽且它还导致了⼈们对⽤于图像识别的卷积神经⽹络（CNN）的兴趣的复苏。

它由 5 个卷积层组成。

其中⼀些后⾯跟随着最⼤池化（max-pooling）层和带有最终1000条路径的 softmax (1000-way softmax)的 3个全连接层。

Alexnet 被引⼊到了使⽤深度卷积神经⽹络的 ImageNet 分类中。

6. ⾃编码器（Autoencoder）⾃编码器是⼀种神经⽹络模型，它的⽬标是预测输⼊⾃⾝，这通常通过⽹络中某个地⽅的「瓶颈（bottleneck）」实现。

通过引⼊瓶颈，我们迫使⽹络学习输⼊更低维度的表征，从⽽有效地将输⼊压缩成⼀个好的表征。

⾃编码器和 PCA 等降维技术相关，但因为它们的⾮线性本质，它们可以学习更为复杂的映射。

⽬前已有⼀些范围涵盖较⼴的⾃编码器存在，包括降噪⾃编码器（DenoisingAutoencoders）、变⾃编码器（Variational Autoencoders）和序列⾃编码器（Sequence Autoencoders）。

机器学习与人工智能领域中常用的英语词汇

机器学习与人工智能领域中常用的英语词汇1.General Concepts (基础概念)•Artificial Intelligence (AI) - 人工智能1)Artificial Intelligence (AI) - 人工智能2)Machine Learning (ML) - 机器学习3)Deep Learning (DL) - 深度学习4)Neural Network - 神经网络5)Natural Language Processing (NLP) - 自然语言处理6)Computer Vision - 计算机视觉7)Robotics - 机器人技术8)Speech Recognition - 语音识别9)Expert Systems - 专家系统10)Knowledge Representation - 知识表示11)Pattern Recognition - 模式识别12)Cognitive Computing - 认知计算13)Autonomous Systems - 自主系统14)Human-Machine Interaction - 人机交互15)Intelligent Agents - 智能代理16)Machine Translation - 机器翻译17)Swarm Intelligence - 群体智能18)Genetic Algorithms - 遗传算法19)Fuzzy Logic - 模糊逻辑20)Reinforcement Learning - 强化学习•Machine Learning (ML) - 机器学习1)Machine Learning (ML) - 机器学习2)Artificial Neural Network - 人工神经网络3)Deep Learning - 深度学习4)Supervised Learning - 有监督学习5)Unsupervised Learning - 无监督学习6)Reinforcement Learning - 强化学习7)Semi-Supervised Learning - 半监督学习8)Training Data - 训练数据9)Test Data - 测试数据10)Validation Data - 验证数据11)Feature - 特征12)Label - 标签13)Model - 模型14)Algorithm - 算法15)Regression - 回归16)Classification - 分类17)Clustering - 聚类18)Dimensionality Reduction - 降维19)Overfitting - 过拟合20)Underfitting - 欠拟合•Deep Learning (DL) - 深度学习1)Deep Learning - 深度学习2)Neural Network - 神经网络3)Artificial Neural Network (ANN) - 人工神经网络4)Convolutional Neural Network (CNN) - 卷积神经网络5)Recurrent Neural Network (RNN) - 循环神经网络6)Long Short-Term Memory (LSTM) - 长短期记忆网络7)Gated Recurrent Unit (GRU) - 门控循环单元8)Autoencoder - 自编码器9)Generative Adversarial Network (GAN) - 生成对抗网络10)Transfer Learning - 迁移学习11)Pre-trained Model - 预训练模型12)Fine-tuning - 微调13)Feature Extraction - 特征提取14)Activation Function - 激活函数15)Loss Function - 损失函数16)Gradient Descent - 梯度下降17)Backpropagation - 反向传播18)Epoch - 训练周期19)Batch Size - 批量大小20)Dropout - 丢弃法•Neural Network - 神经网络1)Neural Network - 神经网络2)Artificial Neural Network (ANN) - 人工神经网络3)Deep Neural Network (DNN) - 深度神经网络4)Convolutional Neural Network (CNN) - 卷积神经网络5)Recurrent Neural Network (RNN) - 循环神经网络6)Long Short-Term Memory (LSTM) - 长短期记忆网络7)Gated Recurrent Unit (GRU) - 门控循环单元8)Feedforward Neural Network - 前馈神经网络9)Multi-layer Perceptron (MLP) - 多层感知器10)Radial Basis Function Network (RBFN) - 径向基函数网络11)Hopfield Network - 霍普菲尔德网络12)Boltzmann Machine - 玻尔兹曼机13)Autoencoder - 自编码器14)Spiking Neural Network (SNN) - 脉冲神经网络15)Self-organizing Map (SOM) - 自组织映射16)Restricted Boltzmann Machine (RBM) - 受限玻尔兹曼机17)Hebbian Learning - 海比安学习18)Competitive Learning - 竞争学习19)Neuroevolutionary - 神经进化20)Neuron - 神经元•Algorithm - 算法1)Algorithm - 算法2)Supervised Learning Algorithm - 有监督学习算法3)Unsupervised Learning Algorithm - 无监督学习算法4)Reinforcement Learning Algorithm - 强化学习算法5)Classification Algorithm - 分类算法6)Regression Algorithm - 回归算法7)Clustering Algorithm - 聚类算法8)Dimensionality Reduction Algorithm - 降维算法9)Decision Tree Algorithm - 决策树算法10)Random Forest Algorithm - 随机森林算法11)Support Vector Machine (SVM) Algorithm - 支持向量机算法12)K-Nearest Neighbors (KNN) Algorithm - K近邻算法13)Naive Bayes Algorithm - 朴素贝叶斯算法14)Gradient Descent Algorithm - 梯度下降算法15)Genetic Algorithm - 遗传算法16)Neural Network Algorithm - 神经网络算法17)Deep Learning Algorithm - 深度学习算法18)Ensemble Learning Algorithm - 集成学习算法19)Reinforcement Learning Algorithm - 强化学习算法20)Metaheuristic Algorithm - 元启发式算法•Model - 模型1)Model - 模型2)Machine Learning Model - 机器学习模型3)Artificial Intelligence Model - 人工智能模型4)Predictive Model - 预测模型5)Classification Model - 分类模型6)Regression Model - 回归模型7)Generative Model - 生成模型8)Discriminative Model - 判别模型9)Probabilistic Model - 概率模型10)Statistical Model - 统计模型11)Neural Network Model - 神经网络模型12)Deep Learning Model - 深度学习模型13)Ensemble Model - 集成模型14)Reinforcement Learning Model - 强化学习模型15)Support Vector Machine (SVM) Model - 支持向量机模型16)Decision Tree Model - 决策树模型17)Random Forest Model - 随机森林模型18)Naive Bayes Model - 朴素贝叶斯模型19)Autoencoder Model - 自编码器模型20)Convolutional Neural Network (CNN) Model - 卷积神经网络模型•Dataset - 数据集1)Dataset - 数据集2)Training Dataset - 训练数据集3)Test Dataset - 测试数据集4)Validation Dataset - 验证数据集5)Balanced Dataset - 平衡数据集6)Imbalanced Dataset - 不平衡数据集7)Synthetic Dataset - 合成数据集8)Benchmark Dataset - 基准数据集9)Open Dataset - 开放数据集10)Labeled Dataset - 标记数据集11)Unlabeled Dataset - 未标记数据集12)Semi-Supervised Dataset - 半监督数据集13)Multiclass Dataset - 多分类数据集14)Feature Set - 特征集15)Data Augmentation - 数据增强16)Data Preprocessing - 数据预处理17)Missing Data - 缺失数据18)Outlier Detection - 异常值检测19)Data Imputation - 数据插补20)Metadata - 元数据•Training - 训练1)Training - 训练2)Training Data - 训练数据3)Training Phase - 训练阶段4)Training Set - 训练集5)Training Examples - 训练样本6)Training Instance - 训练实例7)Training Algorithm - 训练算法8)Training Model - 训练模型9)Training Process - 训练过程10)Training Loss - 训练损失11)Training Epoch - 训练周期12)Training Batch - 训练批次13)Online Training - 在线训练14)Offline Training - 离线训练15)Continuous Training - 连续训练16)Transfer Learning - 迁移学习17)Fine-Tuning - 微调18)Curriculum Learning - 课程学习19)Self-Supervised Learning - 自监督学习20)Active Learning - 主动学习•Testing - 测试1)Testing - 测试2)Test Data - 测试数据3)Test Set - 测试集4)Test Examples - 测试样本5)Test Instance - 测试实例6)Test Phase - 测试阶段7)Test Accuracy - 测试准确率8)Test Loss - 测试损失9)Test Error - 测试错误10)Test Metrics - 测试指标11)Test Suite - 测试套件12)Test Case - 测试用例13)Test Coverage - 测试覆盖率14)Cross-Validation - 交叉验证15)Holdout Validation - 留出验证16)K-Fold Cross-Validation - K折交叉验证17)Stratified Cross-Validation - 分层交叉验证18)Test Driven Development (TDD) - 测试驱动开发19)A/B Testing - A/B 测试20)Model Evaluation - 模型评估•Validation - 验证1)Validation - 验证2)Validation Data - 验证数据3)Validation Set - 验证集4)Validation Examples - 验证样本5)Validation Instance - 验证实例6)Validation Phase - 验证阶段7)Validation Accuracy - 验证准确率8)Validation Loss - 验证损失9)Validation Error - 验证错误10)Validation Metrics - 验证指标11)Cross-Validation - 交叉验证12)Holdout Validation - 留出验证13)K-Fold Cross-Validation - K折交叉验证14)Stratified Cross-Validation - 分层交叉验证15)Leave-One-Out Cross-Validation - 留一法交叉验证16)Validation Curve - 验证曲线17)Hyperparameter Validation - 超参数验证18)Model Validation - 模型验证19)Early Stopping - 提前停止20)Validation Strategy - 验证策略•Supervised Learning - 有监督学习1)Supervised Learning - 有监督学习2)Label - 标签3)Feature - 特征4)Target - 目标5)Training Labels - 训练标签6)Training Features - 训练特征7)Training Targets - 训练目标8)Training Examples - 训练样本9)Training Instance - 训练实例10)Regression - 回归11)Classification - 分类12)Predictor - 预测器13)Regression Model - 回归模型14)Classifier - 分类器15)Decision Tree - 决策树16)Support Vector Machine (SVM) - 支持向量机17)Neural Network - 神经网络18)Feature Engineering - 特征工程19)Model Evaluation - 模型评估20)Overfitting - 过拟合21)Underfitting - 欠拟合22)Bias-Variance Tradeoff - 偏差-方差权衡•Unsupervised Learning - 无监督学习1)Unsupervised Learning - 无监督学习2)Clustering - 聚类3)Dimensionality Reduction - 降维4)Anomaly Detection - 异常检测5)Association Rule Learning - 关联规则学习6)Feature Extraction - 特征提取7)Feature Selection - 特征选择8)K-Means - K均值9)Hierarchical Clustering - 层次聚类10)Density-Based Clustering - 基于密度的聚类11)Principal Component Analysis (PCA) - 主成分分析12)Independent Component Analysis (ICA) - 独立成分分析13)T-distributed Stochastic Neighbor Embedding (t-SNE) - t分布随机邻居嵌入14)Gaussian Mixture Model (GMM) - 高斯混合模型15)Self-Organizing Maps (SOM) - 自组织映射16)Autoencoder - 自动编码器17)Latent Variable - 潜变量18)Data Preprocessing - 数据预处理19)Outlier Detection - 异常值检测20)Clustering Algorithm - 聚类算法•Reinforcement Learning - 强化学习1)Reinforcement Learning - 强化学习2)Agent - 代理3)Environment - 环境4)State - 状态5)Action - 动作6)Reward - 奖励7)Policy - 策略8)Value Function - 值函数9)Q-Learning - Q学习10)Deep Q-Network (DQN) - 深度Q网络11)Policy Gradient - 策略梯度12)Actor-Critic - 演员-评论家13)Exploration - 探索14)Exploitation - 开发15)Temporal Difference (TD) - 时间差分16)Markov Decision Process (MDP) - 马尔可夫决策过程17)State-Action-Reward-State-Action (SARSA) - 状态-动作-奖励-状态-动作18)Policy Iteration - 策略迭代19)Value Iteration - 值迭代20)Monte Carlo Methods - 蒙特卡洛方法•Semi-Supervised Learning - 半监督学习1)Semi-Supervised Learning - 半监督学习2)Labeled Data - 有标签数据3)Unlabeled Data - 无标签数据4)Label Propagation - 标签传播5)Self-Training - 自训练6)Co-Training - 协同训练7)Transudative Learning - 传导学习8)Inductive Learning - 归纳学习9)Manifold Regularization - 流形正则化10)Graph-based Methods - 基于图的方法11)Cluster Assumption - 聚类假设12)Low-Density Separation - 低密度分离13)Semi-Supervised Support Vector Machines (S3VM) - 半监督支持向量机14)Expectation-Maximization (EM) - 期望最大化15)Co-EM - 协同期望最大化16)Entropy-Regularized EM - 熵正则化EM17)Mean Teacher - 平均教师18)Virtual Adversarial Training - 虚拟对抗训练19)Tri-training - 三重训练20)Mix Match - 混合匹配•Feature - 特征1)Feature - 特征2)Feature Engineering - 特征工程3)Feature Extraction - 特征提取4)Feature Selection - 特征选择5)Input Features - 输入特征6)Output Features - 输出特征7)Feature Vector - 特征向量8)Feature Space - 特征空间9)Feature Representation - 特征表示10)Feature Transformation - 特征转换11)Feature Importance - 特征重要性12)Feature Scaling - 特征缩放13)Feature Normalization - 特征归一化14)Feature Encoding - 特征编码15)Feature Fusion - 特征融合16)Feature Dimensionality Reduction - 特征维度减少17)Continuous Feature - 连续特征18)Categorical Feature - 分类特征19)Nominal Feature - 名义特征20)Ordinal Feature - 有序特征•Label - 标签1)Label - 标签2)Labeling - 标注3)Ground Truth - 地面真值4)Class Label - 类别标签5)Target Variable - 目标变量6)Labeling Scheme - 标注方案7)Multi-class Labeling - 多类别标注8)Binary Labeling - 二分类标注9)Label Noise - 标签噪声10)Labeling Error - 标注错误11)Label Propagation - 标签传播12)Unlabeled Data - 无标签数据13)Labeled Data - 有标签数据14)Semi-supervised Learning - 半监督学习15)Active Learning - 主动学习16)Weakly Supervised Learning - 弱监督学习17)Noisy Label Learning - 噪声标签学习18)Self-training - 自训练19)Crowdsourcing Labeling - 众包标注20)Label Smoothing - 标签平滑化•Prediction - 预测1)Prediction - 预测2)Forecasting - 预测3)Regression - 回归4)Classification - 分类5)Time Series Prediction - 时间序列预测6)Forecast Accuracy - 预测准确性7)Predictive Modeling - 预测建模8)Predictive Analytics - 预测分析9)Forecasting Method - 预测方法10)Predictive Performance - 预测性能11)Predictive Power - 预测能力12)Prediction Error - 预测误差13)Prediction Interval - 预测区间14)Prediction Model - 预测模型15)Predictive Uncertainty - 预测不确定性16)Forecast Horizon - 预测时间跨度17)Predictive Maintenance - 预测性维护18)Predictive Policing - 预测式警务19)Predictive Healthcare - 预测性医疗20)Predictive Maintenance - 预测性维护•Classification - 分类1)Classification - 分类2)Classifier - 分类器3)Class - 类别4)Classify - 对数据进行分类5)Class Label - 类别标签6)Binary Classification - 二元分类7)Multiclass Classification - 多类分类8)Class Probability - 类别概率9)Decision Boundary - 决策边界10)Decision Tree - 决策树11)Support Vector Machine (SVM) - 支持向量机12)K-Nearest Neighbors (KNN) - K最近邻算法13)Naive Bayes - 朴素贝叶斯14)Logistic Regression - 逻辑回归15)Random Forest - 随机森林16)Neural Network - 神经网络17)SoftMax Function - SoftMax函数18)One-vs-All (One-vs-Rest) - 一对多(一对剩余)19)Ensemble Learning - 集成学习20)Confusion Matrix - 混淆矩阵•Regression - 回归1)Regression Analysis - 回归分析2)Linear Regression - 线性回归3)Multiple Regression - 多元回归4)Polynomial Regression - 多项式回归5)Logistic Regression - 逻辑回归6)Ridge Regression - 岭回归7)Lasso Regression - Lasso回归8)Elastic Net Regression - 弹性网络回归9)Regression Coefficients - 回归系数10)Residuals - 残差11)Ordinary Least Squares (OLS) - 普通最小二乘法12)Ridge Regression Coefficient - 岭回归系数13)Lasso Regression Coefficient - Lasso回归系数14)Elastic Net Regression Coefficient - 弹性网络回归系数15)Regression Line - 回归线16)Prediction Error - 预测误差17)Regression Model - 回归模型18)Nonlinear Regression - 非线性回归19)Generalized Linear Models (GLM) - 广义线性模型20)Coefficient of Determination (R-squared) - 决定系数21)F-test - F检验22)Homoscedasticity - 同方差性23)Heteroscedasticity - 异方差性24)Autocorrelation - 自相关25)Multicollinearity - 多重共线性26)Outliers - 异常值27)Cross-validation - 交叉验证28)Feature Selection - 特征选择29)Feature Engineering - 特征工程30)Regularization - 正则化2.Neural Networks and Deep Learning (神经网络与深度学习)•Convolutional Neural Network (CNN) - 卷积神经网络1)Convolutional Neural Network (CNN) - 卷积神经网络2)Convolution Layer - 卷积层3)Feature Map - 特征图4)Convolution Operation - 卷积操作5)Stride - 步幅6)Padding - 填充7)Pooling Layer - 池化层8)Max Pooling - 最大池化9)Average Pooling - 平均池化10)Fully Connected Layer - 全连接层11)Activation Function - 激活函数12)Rectified Linear Unit (ReLU) - 线性修正单元13)Dropout - 随机失活14)Batch Normalization - 批量归一化15)Transfer Learning - 迁移学习16)Fine-Tuning - 微调17)Image Classification - 图像分类18)Object Detection - 物体检测19)Semantic Segmentation - 语义分割20)Instance Segmentation - 实例分割21)Generative Adversarial Network (GAN) - 生成对抗网络22)Image Generation - 图像生成23)Style Transfer - 风格迁移24)Convolutional Autoencoder - 卷积自编码器25)Recurrent Neural Network (RNN) - 循环神经网络•Recurrent Neural Network (RNN) - 循环神经网络1)Recurrent Neural Network (RNN) - 循环神经网络2)Long Short-Term Memory (LSTM) - 长短期记忆网络3)Gated Recurrent Unit (GRU) - 门控循环单元4)Sequence Modeling - 序列建模5)Time Series Prediction - 时间序列预测6)Natural Language Processing (NLP) - 自然语言处理7)Text Generation - 文本生成8)Sentiment Analysis - 情感分析9)Named Entity Recognition (NER) - 命名实体识别10)Part-of-Speech Tagging (POS Tagging) - 词性标注11)Sequence-to-Sequence (Seq2Seq) - 序列到序列12)Attention Mechanism - 注意力机制13)Encoder-Decoder Architecture - 编码器-解码器架构14)Bidirectional RNN - 双向循环神经网络15)Teacher Forcing - 强制教师法16)Backpropagation Through Time (BPTT) - 通过时间的反向传播17)Vanishing Gradient Problem - 梯度消失问题18)Exploding Gradient Problem - 梯度爆炸问题19)Language Modeling - 语言建模20)Speech Recognition - 语音识别•Long Short-Term Memory (LSTM) - 长短期记忆网络1)Long Short-Term Memory (LSTM) - 长短期记忆网络2)Cell State - 细胞状态3)Hidden State - 隐藏状态4)Forget Gate - 遗忘门5)Input Gate - 输入门6)Output Gate - 输出门7)Peephole Connections - 窥视孔连接8)Gated Recurrent Unit (GRU) - 门控循环单元9)Vanishing Gradient Problem - 梯度消失问题10)Exploding Gradient Problem - 梯度爆炸问题11)Sequence Modeling - 序列建模12)Time Series Prediction - 时间序列预测13)Natural Language Processing (NLP) - 自然语言处理14)Text Generation - 文本生成15)Sentiment Analysis - 情感分析16)Named Entity Recognition (NER) - 命名实体识别17)Part-of-Speech Tagging (POS Tagging) - 词性标注18)Attention Mechanism - 注意力机制19)Encoder-Decoder Architecture - 编码器-解码器架构20)Bidirectional LSTM - 双向长短期记忆网络•Attention Mechanism - 注意力机制1)Attention Mechanism - 注意力机制2)Self-Attention - 自注意力3)Multi-Head Attention - 多头注意力4)Transformer - 变换器5)Query - 查询6)Key - 键7)Value - 值8)Query-Value Attention - 查询-值注意力9)Dot-Product Attention - 点积注意力10)Scaled Dot-Product Attention - 缩放点积注意力11)Additive Attention - 加性注意力12)Context Vector - 上下文向量13)Attention Score - 注意力分数14)SoftMax Function - SoftMax函数15)Attention Weight - 注意力权重16)Global Attention - 全局注意力17)Local Attention - 局部注意力18)Positional Encoding - 位置编码19)Encoder-Decoder Attention - 编码器-解码器注意力20)Cross-Modal Attention - 跨模态注意力•Generative Adversarial Network (GAN) - 生成对抗网络1)Generative Adversarial Network (GAN) - 生成对抗网络2)Generator - 生成器3)Discriminator - 判别器4)Adversarial Training - 对抗训练5)Minimax Game - 极小极大博弈6)Nash Equilibrium - 纳什均衡7)Mode Collapse - 模式崩溃8)Training Stability - 训练稳定性9)Loss Function - 损失函数10)Discriminative Loss - 判别损失11)Generative Loss - 生成损失12)Wasserstein GAN (WGAN) - Wasserstein GAN（WGAN）13)Deep Convolutional GAN (DCGAN) - 深度卷积生成对抗网络（DCGAN）14)Conditional GAN (c GAN) - 条件生成对抗网络（c GAN）15)Style GAN - 风格生成对抗网络16)Cycle GAN - 循环生成对抗网络17)Progressive Growing GAN (PGGAN) - 渐进式增长生成对抗网络（PGGAN）18)Self-Attention GAN (SAGAN) - 自注意力生成对抗网络（SAGAN）19)Big GAN - 大规模生成对抗网络20)Adversarial Examples - 对抗样本•Encoder-Decoder - 编码器-解码器1)Encoder-Decoder Architecture - 编码器-解码器架构2)Encoder - 编码器3)Decoder - 解码器4)Sequence-to-Sequence Model (Seq2Seq) - 序列到序列模型5)State Vector - 状态向量6)Context Vector - 上下文向量7)Hidden State - 隐藏状态8)Attention Mechanism - 注意力机制9)Teacher Forcing - 强制教师法10)Beam Search - 束搜索11)Recurrent Neural Network (RNN) - 循环神经网络12)Long Short-Term Memory (LSTM) - 长短期记忆网络13)Gated Recurrent Unit (GRU) - 门控循环单元14)Bidirectional Encoder - 双向编码器15)Greedy Decoding - 贪婪解码16)Masking - 遮盖17)Dropout - 随机失活18)Embedding Layer - 嵌入层19)Cross-Entropy Loss - 交叉熵损失20)Tokenization - 令牌化•Transfer Learning - 迁移学习1)Transfer Learning - 迁移学习2)Source Domain - 源领域3)Target Domain - 目标领域4)Fine-Tuning - 微调5)Domain Adaptation - 领域自适应6)Pre-Trained Model - 预训练模型7)Feature Extraction - 特征提取8)Knowledge Transfer - 知识迁移9)Unsupervised Domain Adaptation - 无监督领域自适应10)Semi-Supervised Domain Adaptation - 半监督领域自适应11)Multi-Task Learning - 多任务学习12)Data Augmentation - 数据增强13)Task Transfer - 任务迁移14)Model Agnostic Meta-Learning (MAML) - 与模型无关的元学习（MAML）15)One-Shot Learning - 单样本学习16)Zero-Shot Learning - 零样本学习17)Few-Shot Learning - 少样本学习18)Knowledge Distillation - 知识蒸馏19)Representation Learning - 表征学习20)Adversarial Transfer Learning - 对抗迁移学习•Pre-trained Models - 预训练模型1)Pre-trained Model - 预训练模型2)Transfer Learning - 迁移学习3)Fine-Tuning - 微调4)Knowledge Transfer - 知识迁移5)Domain Adaptation - 领域自适应6)Feature Extraction - 特征提取7)Representation Learning - 表征学习8)Language Model - 语言模型9)Bidirectional Encoder Representations from Transformers (BERT) - 双向编码器结构转换器10)Generative Pre-trained Transformer (GPT) - 生成式预训练转换器11)Transformer-based Models - 基于转换器的模型12)Masked Language Model (MLM) - 掩蔽语言模型13)Cloze Task - 填空任务14)Tokenization - 令牌化15)Word Embeddings - 词嵌入16)Sentence Embeddings - 句子嵌入17)Contextual Embeddings - 上下文嵌入18)Self-Supervised Learning - 自监督学习19)Large-Scale Pre-trained Models - 大规模预训练模型•Loss Function - 损失函数1)Loss Function - 损失函数2)Mean Squared Error (MSE) - 均方误差3)Mean Absolute Error (MAE) - 平均绝对误差4)Cross-Entropy Loss - 交叉熵损失5)Binary Cross-Entropy Loss - 二元交叉熵损失6)Categorical Cross-Entropy Loss - 分类交叉熵损失7)Hinge Loss - 合页损失8)Huber Loss - Huber损失9)Wasserstein Distance - Wasserstein距离10)Triplet Loss - 三元组损失11)Contrastive Loss - 对比损失12)Dice Loss - Dice损失13)Focal Loss - 焦点损失14)GAN Loss - GAN损失15)Adversarial Loss - 对抗损失16)L1 Loss - L1损失17)L2 Loss - L2损失18)Huber Loss - Huber损失19)Quantile Loss - 分位数损失•Activation Function - 激活函数1)Activation Function - 激活函数2)Sigmoid Function - Sigmoid函数3)Hyperbolic Tangent Function (Tanh) - 双曲正切函数4)Rectified Linear Unit (Re LU) - 矩形线性单元5)Parametric Re LU (P Re LU) - 参数化Re LU6)Exponential Linear Unit (ELU) - 指数线性单元7)Swish Function - Swish函数8)Softplus Function - Soft plus函数9)Softmax Function - SoftMax函数10)Hard Tanh Function - 硬双曲正切函数11)Softsign Function - Softsign函数12)GELU (Gaussian Error Linear Unit) - GELU（高斯误差线性单元）13)Mish Function - Mish函数14)CELU (Continuous Exponential Linear Unit) - CELU（连续指数线性单元）15)Bent Identity Function - 弯曲恒等函数16)Gaussian Error Linear Units (GELUs) - 高斯误差线性单元17)Adaptive Piecewise Linear (APL) - 自适应分段线性函数18)Radial Basis Function (RBF) - 径向基函数•Backpropagation - 反向传播1)Backpropagation - 反向传播2)Gradient Descent - 梯度下降3)Partial Derivative - 偏导数4)Chain Rule - 链式法则5)Forward Pass - 前向传播6)Backward Pass - 反向传播7)Computational Graph - 计算图8)Neural Network - 神经网络9)Loss Function - 损失函数10)Gradient Calculation - 梯度计算11)Weight Update - 权重更新12)Activation Function - 激活函数13)Optimizer - 优化器14)Learning Rate - 学习率15)Mini-Batch Gradient Descent - 小批量梯度下降16)Stochastic Gradient Descent (SGD) - 随机梯度下降17)Batch Gradient Descent - 批量梯度下降18)Momentum - 动量19)Adam Optimizer - Adam优化器20)Learning Rate Decay - 学习率衰减•Gradient Descent - 梯度下降1)Gradient Descent - 梯度下降2)Stochastic Gradient Descent (SGD) - 随机梯度下降3)Mini-Batch Gradient Descent - 小批量梯度下降4)Batch Gradient Descent - 批量梯度下降5)Learning Rate - 学习率6)Momentum - 动量7)Adaptive Moment Estimation (Adam) - 自适应矩估计8)RMSprop - 均方根传播9)Learning Rate Schedule - 学习率调度10)Convergence - 收敛11)Divergence - 发散12)Adagrad - 自适应学习速率方法13)Adadelta - 自适应增量学习率方法14)Adamax - 自适应矩估计的扩展版本15)Nadam - Nesterov Accelerated Adaptive Moment Estimation16)Learning Rate Decay - 学习率衰减17)Step Size - 步长18)Conjugate Gradient Descent - 共轭梯度下降19)Line Search - 线搜索20)Newton's Method - 牛顿法•Learning Rate - 学习率1)Learning Rate - 学习率2)Adaptive Learning Rate - 自适应学习率3)Learning Rate Decay - 学习率衰减4)Initial Learning Rate - 初始学习率5)Step Size - 步长6)Momentum - 动量7)Exponential Decay - 指数衰减8)Annealing - 退火9)Cyclical Learning Rate - 循环学习率10)Learning Rate Schedule - 学习率调度11)Warm-up - 预热12)Learning Rate Policy - 学习率策略13)Learning Rate Annealing - 学习率退火14)Cosine Annealing - 余弦退火15)Gradient Clipping - 梯度裁剪16)Adapting Learning Rate - 适应学习率17)Learning Rate Multiplier - 学习率倍增器18)Learning Rate Reduction - 学习率降低19)Learning Rate Update - 学习率更新20)Scheduled Learning Rate - 定期学习率•Batch Size - 批量大小1)Batch Size - 批量大小2)Mini-Batch - 小批量3)Batch Gradient Descent - 批量梯度下降4)Stochastic Gradient Descent (SGD) - 随机梯度下降5)Mini-Batch Gradient Descent - 小批量梯度下降6)Online Learning - 在线学习7)Full-Batch - 全批量8)Data Batch - 数据批次9)Training Batch - 训练批次10)Batch Normalization - 批量归一化11)Batch-wise Optimization - 批量优化12)Batch Processing - 批量处理13)Batch Sampling - 批量采样14)Adaptive Batch Size - 自适应批量大小15)Batch Splitting - 批量分割16)Dynamic Batch Size - 动态批量大小17)Fixed Batch Size - 固定批量大小18)Batch-wise Inference - 批量推理19)Batch-wise Training - 批量训练20)Batch Shuffling - 批量洗牌•Epoch - 训练周期1)Training Epoch - 训练周期2)Epoch Size - 周期大小3)Early Stopping - 提前停止4)Validation Set - 验证集5)Training Set - 训练集6)Test Set - 测试集7)Overfitting - 过拟合8)Underfitting - 欠拟合9)Model Evaluation - 模型评估10)Model Selection - 模型选择11)Hyperparameter Tuning - 超参数调优12)Cross-Validation - 交叉验证13)K-fold Cross-Validation - K折交叉验证14)Stratified Cross-Validation - 分层交叉验证15)Leave-One-Out Cross-Validation (LOOCV) - 留一法交叉验证16)Grid Search - 网格搜索17)Random Search - 随机搜索18)Model Complexity - 模型复杂度19)Learning Curve - 学习曲线20)Convergence - 收敛3.Machine Learning Techniques and Algorithms (机器学习技术与算法)•Decision Tree - 决策树1)Decision Tree - 决策树2)Node - 节点3)Root Node - 根节点4)Leaf Node - 叶节点5)Internal Node - 内部节点6)Splitting Criterion - 分裂准则7)Gini Impurity - 基尼不纯度8)Entropy - 熵9)Information Gain - 信息增益10)Gain Ratio - 增益率11)Pruning - 剪枝12)Recursive Partitioning - 递归分割13)CART (Classification and Regression Trees) - 分类回归树14)ID3 (Iterative Dichotomiser 3) - 迭代二叉树315)C4.5 (successor of ID3) - C4.5（ID3的后继者）16)C5.0 (successor of C4.5) - C5.0（C4.5的后继者）17)Split Point - 分裂点18)Decision Boundary - 决策边界19)Pruned Tree - 剪枝后的树20)Decision Tree Ensemble - 决策树集成•Random Forest - 随机森林1)Random Forest - 随机森林2)Ensemble Learning - 集成学习3)Bootstrap Sampling - 自助采样4)Bagging (Bootstrap Aggregating) - 装袋法5)Out-of-Bag (OOB) Error - 袋外误差6)Feature Subset - 特征子集7)Decision Tree - 决策树8)Base Estimator - 基础估计器9)Tree Depth - 树深度10)Randomization - 随机化11)Majority Voting - 多数投票12)Feature Importance - 特征重要性13)OOB Score - 袋外得分14)Forest Size - 森林大小15)Max Features - 最大特征数16)Min Samples Split - 最小分裂样本数17)Min Samples Leaf - 最小叶节点样本数18)Gini Impurity - 基尼不纯度19)Entropy - 熵20)Variable Importance - 变量重要性•Support Vector Machine (SVM) - 支持向量机1)Support Vector Machine (SVM) - 支持向量机2)Hyperplane - 超平面3)Kernel Trick - 核技巧4)Kernel Function - 核函数5)Margin - 间隔6)Support Vectors - 支持向量7)Decision Boundary - 决策边界8)Maximum Margin Classifier - 最大间隔分类器9)Soft Margin Classifier - 软间隔分类器10) C Parameter - C参数11)Radial Basis Function (RBF) Kernel - 径向基函数核12)Polynomial Kernel - 多项式核13)Linear Kernel - 线性核14)Quadratic Kernel - 二次核15)Gaussian Kernel - 高斯核16)Regularization - 正则化17)Dual Problem - 对偶问题18)Primal Problem - 原始问题19)Kernelized SVM - 核化支持向量机20)Multiclass SVM - 多类支持向量机•K-Nearest Neighbors (KNN) - K-最近邻1)K-Nearest Neighbors (KNN) - K-最近邻2)Nearest Neighbor - 最近邻3)Distance Metric - 距离度量4)Euclidean Distance - 欧氏距离5)Manhattan Distance - 曼哈顿距离6)Minkowski Distance - 闵可夫斯基距离7)Cosine Similarity - 余弦相似度8)K Value - K值9)Majority Voting - 多数投票10)Weighted KNN - 加权KNN11)Radius Neighbors - 半径邻居12)Ball Tree - 球树13)KD Tree - KD树14)Locality-Sensitive Hashing (LSH) - 局部敏感哈希15)Curse of Dimensionality - 维度灾难16)Class Label - 类标签17)Training Set - 训练集18)Test Set - 测试集19)Validation Set - 验证集20)Cross-Validation - 交叉验证•Naive Bayes - 朴素贝叶斯1)Naive Bayes - 朴素贝叶斯2)Bayes' Theorem - 贝叶斯定理3)Prior Probability - 先验概率4)Posterior Probability - 后验概率5)Likelihood - 似然6)Class Conditional Probability - 类条件概率7)Feature Independence Assumption - 特征独立假设8)Multinomial Naive Bayes - 多项式朴素贝叶斯9)Gaussian Naive Bayes - 高斯朴素贝叶斯10)Bernoulli Naive Bayes - 伯努利朴素贝叶斯11)Laplace Smoothing - 拉普拉斯平滑12)Add-One Smoothing - 加一平滑13)Maximum A Posteriori (MAP) - 最大后验概率14)Maximum Likelihood Estimation (MLE) - 最大似然估计15)Classification - 分类16)Feature Vectors - 特征向量17)Training Set - 训练集18)Test Set - 测试集19)Class Label - 类标签20)Confusion Matrix - 混淆矩阵•Clustering - 聚类1)Clustering - 聚类2)Centroid - 质心3)Cluster Analysis - 聚类分析4)Partitioning Clustering - 划分式聚类5)Hierarchical Clustering - 层次聚类6)Density-Based Clustering - 基于密度的聚类7)K-Means Clustering - K均值聚类8)K-Medoids Clustering - K中心点聚类9)DBSCAN (Density-Based Spatial Clustering of Applications with Noise) - 基于密度的空间聚类算法10)Agglomerative Clustering - 聚合式聚类11)Dendrogram - 系统树图12)Silhouette Score - 轮廓系数13)Elbow Method - 肘部法则14)Clustering Validation - 聚类验证15)Intra-cluster Distance - 类内距离16)Inter-cluster Distance - 类间距离17)Cluster Cohesion - 类内连贯性18)Cluster Separation - 类间分离度19)Cluster Assignment - 聚类分配20)Cluster Label - 聚类标签•K-Means - K-均值1)K-Means - K-均值2)Centroid - 质心3)Cluster - 聚类4)Cluster Center - 聚类中心5)Cluster Assignment - 聚类分配6)Cluster Analysis - 聚类分析7)K Value - K值8)Elbow Method - 肘部法则9)Inertia - 惯性10)Silhouette Score - 轮廓系数11)Convergence - 收敛12)Initialization - 初始化13)Euclidean Distance - 欧氏距离14)Manhattan Distance - 曼哈顿距离15)Distance Metric - 距离度量16)Cluster Radius - 聚类半径17)Within-Cluster Variation - 类内变异18)Cluster Quality - 聚类质量19)Clustering Algorithm - 聚类算法20)Clustering Validation - 聚类验证•Dimensionality Reduction - 降维1)Dimensionality Reduction - 降维2)Feature Extraction - 特征提取3)Feature Selection - 特征选择4)Principal Component Analysis (PCA) - 主成分分析5)Singular Value Decomposition (SVD) - 奇异值分解6)Linear Discriminant Analysis (LDA) - 线性判别分析7)t-Distributed Stochastic Neighbor Embedding (t-SNE) - t-分布随机邻域嵌入8)Autoencoder - 自编码器9)Manifold Learning - 流形学习10)Locally Linear Embedding (LLE) - 局部线性嵌入11)Isomap - 等度量映射12)Uniform Manifold Approximation and Projection (UMAP) - 均匀流形逼近与投影13)Kernel PCA - 核主成分分析14)Non-negative Matrix Factorization (NMF) - 非负矩阵分解15)Independent Component Analysis (ICA) - 独立成分分析16)Variational Autoencoder (VAE) - 变分自编码器17)Sparse Coding - 稀疏编码18)Random Projection - 随机投影19)Neighborhood Preserving Embedding (NPE) - 保持邻域结构的嵌入20)Curvilinear Component Analysis (CCA) - 曲线成分分析•Principal Component Analysis (PCA) - 主成分分析1)Principal Component Analysis (PCA) - 主成分分析2)Eigenvector - 特征向量3)Eigenvalue - 特征值4)Covariance Matrix - 协方差矩阵。

人工智能原理_北京大学中国大学mooc课后章节答案期末考试题库2023年

人工智能原理_北京大学中国大学mooc课后章节答案期末考试题库2023年1.Turing Test is designed to provide what kind of satisfactory operationaldefinition?图灵测试旨在给予哪一种令人满意的操作定义？答案:machine intelligence 机器智能2.Thinking the differences between agent functions and agent programs, selectcorrect statements from following ones.考虑智能体函数与智能体程序的差异，从下列陈述中选择正确的答案。

答案:An agent program implements an agent function.一个智能体程序实现一个智能体函数。

3.There are two main kinds of formulation for 8-queens problem. Which of thefollowing one is the formulation that starts with all 8 queens on the boardand moves them around?有两种8皇后问题的形式化方式。

“初始时8个皇后都放在棋盘上，然后再进行移动”属于哪一种形式化方式？答案:Complete-state formulation 全态形式化4.What kind of knowledge will be used to describe how a problem is solved?哪种知识可用于描述如何求解问题？答案:Procedural knowledge 过程性知识5.Which of the following is used to discover general facts from trainingexamples?下列中哪个用于训练样本中发现一般的事实？答案:Inductive learning 归纳学习6.Which statement best describes the task of “classification” in machinelearning?哪一个是机器学习中“分类”任务的正确描述？答案:To assign a category to each item. 为每个项目分配一个类别。

人工智能算法基础唐宇迪代码

人工智能算法基础1. 简介人工智能（Artificial Intelligence，简称AI）是一门研究如何使计算机能够像人一样思考、学习和解决问题的学科。

而人工智能算法则是实现人工智能的核心技术之一。

本文将介绍人工智能算法的基础知识和常见的算法模型。

2. 机器学习算法机器学习（Machine Learning，简称ML）是人工智能领域的一个重要分支，其目标是通过数据和经验来改善计算机程序的性能。

机器学习算法可以分为监督学习、无监督学习和强化学习三大类。

2.1 监督学习监督学习是指通过已标记的训练样本来训练模型，并通过模型预测未知数据的标签。

常见的监督学习算法有线性回归、逻辑回归、决策树、支持向量机等。

•线性回归（Linear Regression）：用于建立输入变量与输出变量之间线性关系的回归模型。

•逻辑回归（Logistic Regression）：用于建立输入变量与离散输出变量之间的关系的分类模型。

•决策树（Decision Tree）：通过一系列的决策规则对数据进行分类或预测。

•支持向量机（Support Vector Machine）：通过寻找最优超平面将数据分为两个类别。

2.2 无监督学习无监督学习是指训练模型时没有标记的训练样本，模型需要自行发现数据中的结构和模式。

常见的无监督学习算法有聚类、降维和关联规则等。

•聚类（Clustering）：将相似的数据点分组到同一类别中，常用算法有K-means、层次聚类等。

•降维（Dimensionality Reduction）：将高维数据映射到低维空间，常用算法有主成分分析（PCA）、线性判别分析（LDA）等。

•关联规则（Association Rules）：发现数据中项之间的关联关系，常用算法有Apriori、FP-growth等。

2.3 强化学习强化学习是指智能体通过与环境交互来学习如何做出最优决策的一种学习方式。

智能体根据环境给予的奖励信号来调整自己的行为。

非线性数据降维方法---LLE及其改进算法介绍

⾮线性数据降维⽅法---LLE及其改进算法介绍LLE及其改进算法介绍Locally linear embedding (LLE) (Sam T.Roweis and Lawrence K.Saul, 2000)以及Supervised locally linear embedding (SLLE) (Dick and Robert, 2002) 是最近提出的⾮线性降维⽅法，它能够使降维后的数据保持原有拓扑结构。

LLE算法可以有图1所⽰的⼀个例⼦来描述。

在图1所⽰中，LLE能成功地将三维⾮线性数据映射到⼆维空间中。

如果把图1（B）中红颜⾊和蓝颜⾊的数据分别看成是分布在三维空间中的两类数据，通过LLE算法降维后，则数据在⼆维空间中仍能保持相对独⽴的两类。

在图1（B）中的⿊⾊⼩圈中可以看出，如果将⿊⾊⼩圈中的数据映射到⼆维空间中，如图1（C）中的⿊⾊⼩圈所⽰，映射后的数据任能保持原有的数据流形，这说明LLE算法确实能保持流形的领域不变性。

由此LLE算法可以应⽤于样本的聚类。

⽽线性⽅法，如PCA和MDS，都不能与它⽐拟的。

LLE算法操作简单，且算法中的优化不涉及到局部最⼩化。

该算法能解决⾮线性映射，但是，当处理数据的维数过⼤，数量过多，涉及到的稀疏矩阵过⼤，不易于处理。

在图1中的球形⾯中，当缺少北极⾯时，应⽤LLE算法则能很好的将其映射到⼆维空间中，如图1中的C所⽰。

如果数据分布在整个封闭的球⾯上，LLE则不能将它映射到⼆维空间，且不能保持原有的数据流形。

那么我们在处理数据中，⾸先假设数据不是分布在闭合的球⾯或者椭球⾯上。

图1 ⾮线性降维实例：B是从A中提取的样本点（三维），通过⾮线性降维算法（LLE），将数据映射到⼆维空间中（C）。

从C图中的颜⾊可以看出通过LLE算法处理后的数据，能很好的保持原有数据的邻域特性LLE算法是最近提出的针对⾮线性数据的⼀种新的降维⽅法，处理后的低维数据均能够保持原有的拓扑关系。

蒙特卡洛法的基本原理

蒙特卡洛法的基本原理蒙特卡洛法（Monte Carlo method）是一种基于随机抽样的数值计算方法，用于解决难以通过解析方法或传统数学模型求解的问题。

它在物理学、化学、工程学、计算机科学、金融学、生物学等领域都有广泛应用。

本文将介绍蒙特卡洛法的基本原理，包括随机数生成、统计抽样、蒙特卡洛积分、随机漫步等方面。

一、随机数生成随机数是蒙特卡洛法中的基本元素，其质量直接影响着计算结果的准确性。

随机数的生成必须具有一定的随机性和均匀性。

常见的随机数生成方法有：线性同余法、拉斯维加斯法、梅森旋转算法、反序列化等。

梅森旋转算法是一种广泛使用的准随机数生成方法，其随机数序列的周期性长、随机性好，可以满足大多数应用的需要。

二、统计抽样蒙特卡洛法利用抽样的思想，通过对输入参数进行随机取样，来模拟整个系统的行为，并推断出某个问题的答案。

统计抽样是蒙特卡洛方法中最核心的部分，是通过对概率分布进行样本抽取来模拟随机事件的发生，从而得到数值计算的结果。

常用的统计抽样方法有：均匀分布抽样、正态分布抽样、指数分布抽样、泊松分布抽样等。

通过对这些概率分布进行抽样，可以在大量随机取样后得到一个概率分布近似于输入分布的“抽样分布”，进而求出所需的数值计算结果。

三、蒙特卡洛积分蒙特卡洛积分是蒙特卡洛法的重要应用之一。

它利用统计抽样的思想，通过对输入函数进行随机抽样，计算其随机取样后的平均值，来估算积分的值。

蒙特卡洛积分的计算精度与随机取样的数量、抽样分布的质量等因素有关。

蒙特卡洛积分的计算公式如下：$I=\frac{1}{N}\sum_{i=1}^{N}f(X_{i})\frac{V}{p(X_{i})}$$N$为随机取样的数量，$f(X_{i})$为输入函数在点$X_{i}$的取值，$V$为积分区域的体积，$p(X_{i})$为在点$X_{i}$出现的抽样分布的概率密度函数。

通过大量的样本拟合，可以估算出$I$的值接近于真实积分的值。

单细胞数据高级分析之初步降维和聚类DimensionalityreductionClust。。。

单细胞数据⾼级分析之初步降维和聚类DimensionalityreductionClust。

个⼈的⼀些碎碎念：聚类，直觉就能想到kmeans聚类，另外还有⼀个hierarchical clustering，但是单细胞⾥⾯都⽤得不多，为什么？印象中只有⼀个scoring model是⽤kmean进⾏粗聚类。

（10x就是先做PCA，再⽤kmeans聚类的）鉴于单细胞的教程很多，也有不下于10种针对单细胞的聚类⽅法了。

降维往往是和聚类在⼀起的，所以似乎有点难以区分。

PCA到底是降维、聚类还是可视化的⽅法，t-SNE呢？其实稍微思考⼀下，PCA、t-SNE还有下⾯的diffusionMap，都是⼀种降维⽅法。

区别就在于PCA是完全的线性变换得到PC，t-SNE和diffusionMap 都是⾮线性的。

为什么降维？因为我们特征太多了，基因都是万级的，降维之后才能⽤kmeans啥的。

其次就是，降维了才能可视化！我们可视化的最⾼维度就是三维，⼏万维是⽆法可视化的。

但paper⾥，我们最多选前两维，三维在平⾯上的效果还不如⼆维。

聚类策略：聚类还要什么策略？不就是选好特征之后，再选⼀个k就得到聚类的结果了吗？是的，常规分析确实没有什么⾼深的东西。

但通常我们不是为了聚类⽽聚类，我们的结果是为⽣物学问题⽽服务的，如果从任何⾓度都⽆法解释你的聚类结果，那你还聚什么类，总不可能在paper⾥就写我们聚类了，得到了⼀些marker，然后就没了下⽂把！什么问题？什么叫针对问题的聚类呢？下⾯这篇⽂章就是针对具体问题的聚类。

先知：我们知道我们细胞⾥有些污染的细胞，如何通过聚类将他们识别出来？这种具体的问题就没法通过跑常规流程来解决了，得想办法！Dimensionality reduction.Throughout the manuscript we use diffusion maps, a non-linear dimensionality reduction technique37. We calculate a cell-to-cell distance matrix using 1 - Pearson correlation and use the diffuse function of the diffusionMap R package with default parameters to obtain the first 50 DMCs.To determine the significant DMCs, we look at the reduction of eigenvalues associated with DMCs. We determine all dimensions with an eigenvalue of at least 4% relative to the sum of the first 50 eigenvalues as significant, and scale all dimensions to have mean 0 and standard deviation of 1.有点超前（另类），⽤diffusionMap来降维，计算了细胞-细胞的距离，得到50个DMC，鉴定出显著的DMC，scale⼀下。

面向学术文献的学者兴趣标签识别方法

面向学术文献的学者兴趣标签识别方法首都图书馆　北京　100021Capital Library of China, Beijing 100021, China谢鹏摘要：学术文献是科学进步与发展的载体，各种元数据信息包括作者、论文、期刊以及这些实体之间的关系，具有重要的价值，如何精准构建学者用户画像是一个具有挑战性的问题。

早期的用户画像相对简单，区分度以及可用性都不高。

本文在“2017开放学术精准画像大赛”TASK3的真实数据上，提取学者与期刊的关系和学者与论文的关系，设计关系模型并采用LSI 降维技术以及文本相似度计算，对学者兴趣标签进行识别与评估，并进行数据可视化分析。

实验结果表明，使用本文提出的方法可准确有效的识别学者兴趣标签，准确率为P@1=92%、P@2=94%、P@3=98%。

关键词：用户画像；兴趣标签识别；LSI 中图分类号：G35Abstract: li terature is recognized as the carrier of scientific progress and development. Various metadata information, including the author, thesis, press, and even the relationship between these entities, is of great value. How to construct user profile for academic users exactly is a challenging issue. The early user profile is relatively simple, with little distinction and usability. Based on the real data set of task 3 in “2017 Open Academic Data Challenge”, we extract the relationship between scholars and press, and the relationship between scholars and thesis to design the relationship model. And then, use the LSI dimensionality reduction technology and the similarity calculation of text to recognize the scholar’s interest. The recognized interests are evaluated and do data visualization analysis. The experimental results show that the method proposed based on the information of the press and thesis in this paper can effectively and accurately recognize scholars interest labels. And the accuracy is P@1=92%, P@2=94%, P@3=98%.Keywords: User profile; interest label recognition; LSI作者简介：谢鹏（1980-），学士，馆员，研究方向：图书情报、数字图书馆，E-mail ：xp@ 。

机器学习四大分类

机器学习四⼤分类机器学习分为四⼤块，分别是classification (分类)，regression (回归), clustering (聚类), dimensionality reduction (降维)。

聚类（clustering）⽆监督学习的结果。

聚类的结果将产⽣⼀组集合，集合中的对象与同集合中的对象彼此相似，与其他集合中的对象相异。

没有标准参考的学⽣给书本分的类别，表⽰⾃⼰认为这些书可能是同⼀类别的（具体什么类别不知道，没有标签和⽬标，即不是判断书的好坏（⽬标，标签），只能凭借特征⽽分类）。

分类（classification）有监督学习的两⼤应⽤之⼀，产⽣离散的结果。

例如向模型输⼊⼈的各种数据的训练样本，产⽣“输⼊⼀个⼈的数据，判断是否患有癌症”的结果，结果必定是离散的，只有“是”或“否”。

（即有⽬标和标签，能判断⽬标特征是属于哪⼀个类型）回归（regression）有监督学习的两⼤应⽤之⼀，产⽣连续的结果。

例如向模型输⼊⼈的各种数据的训练样本，产⽣“输⼊⼀个⼈的数据，判断此⼈20年后今后的经济能⼒”的结果，结果是连续的，往往得到⼀条回归曲线。

当输⼊⾃变量不同时，输出的因变量⾮离散分布（不仅仅是⼀条线性直线，多项曲线也是回归曲线）。

1，给定⼀个样本特征 , 我们希望预测其对应的属性值 , 如果是离散的, 那么这就是⼀个分类问题，反之，如果是连续的实数, 这就是⼀个回归问题。

2，如果给定⼀组样本特征 , 我们没有对应的属性值 , ⽽是想发掘这组样本在⼆维空间的分布, ⽐如分析哪些样本靠的更近，哪些样本之间离得很远, 这就是属于聚类问题。

3，如果我们想⽤维数更低的⼦空间来表⽰原来⾼维的特征空间, 那么这就是降维问题。

基于lda的文本情感分析

本科毕业设计（论文）学院(部)计算机科学与技术学院题目基于lda的文本情感分析年级2014专业信息治理与信息系统班级14信管学号1427402014姓名何聪指导老师严建峰职称副教授论文提交日期2019年5月19日目录摘要 (1)前言 (3)第一章概述 (5)1.1情感分析概述 (5)1.1.1主要研究内容 (5)1.1.2文本情感分析的分类 (6)1.1.3主题模型在情感分析中的应用 (7)1.2国内外研究现状 (7)1.3本文内容安排 (8)第二章数据预处理 (10)2.1概述 (10)2.2分词以及简繁体转换 (10)2.3去除停用词 (10)2.4抽取情感信息 (11)2.4.1情感词典的构建 (11)2.4.2抽取情感信息 (11)2.4.3数据 (12)2.5本章小结 (12)第三章 LDA建模 (13)3.1LDA概念 (13)3.1.1概率主题概念的提出 (13)3.1.2LDA模型 (14)3.2试验 (16)3.2.1划分数据集 (16)3.2.2数据词典 (16)3.2.3向量化 (17)3.2.4使用TF-IDF作为特征值 (17)3.2.5LDA模型训练 (19)3.3本章小结 (20)第四章 SVM分类 (21)4.1SVM概念 (21)4.1.1线性分类 (22)4.1.2软间隔最大化 (23)4.1.3非线性支持向量机 (24)4.2本文中的SVC (26)4.2.1算法描述 (26)4.3试验 (28)4.3.1特征选取 (28)4.3.2数据转换 (28)4.3.3将数据随机分为训练集和测试集 (29)4.3.4SVM训练和预测 (29)4.3本章总结 (30)第五章贝叶斯分类 (31)5.1概念 (31)5.2贝叶斯定理 (31)5.2.1简朴贝叶斯 (31)5.2.2伯努利模型 (32)5.3本文中的简朴贝叶斯 (32)5.3.1算法描述 (32)5.3试验 (33)5.3.1特征选取 (33)5.3.2向量化 (33)5.3.3简朴贝叶斯分类训练 (34)5.3.4测试 (34)5.3.5准确率 (35)5.4本章总结 (35)第六章总结与展望 (37)6.1本文主要内容总结 (37)6.2存在的问题以及未来展望 (37)参考文献 (39)致谢 (40)摘要互联网的快速进展让各类社交媒体与日俱增，人们在网络上发表各种各样的评论、博客等信息。

自动驾驶环境下交叉口车辆路径规划与最优控制模型

以上方法均假设在进行aic模型优化时自动驾驶车辆在交叉口内部的行驶路径提前确定且已知即假设自动驾驶车辆在交叉口选择的进口车道和出口车道已知且作为输入在此基础上进行aic模型设计然而在多车道情况下自动驾驶车辆可以通过选择不同的进口车道或出口车道即通过调整车辆在交叉口内部行驶路径的方式规避冲突
第 46 卷第 9 期 2020 年 9 月
Dresner 等[9−10] 于 2004 年和 2008 年提出了早期的 AIC 模型, 所有自动驾驶车辆根据到达交叉口的时间次序, 依次请求交叉口的通行权, 如果车辆在交叉口的通过路径上, 与更早到达的车辆冲突, 则需等待, 否则能顺利通行, 通过与信号控制对比, 证明了其模型的有效性. 其模型基于先到先服务的交叉口控制策略 (First come first served, FCFS), 在大多数情况下, FCFS 被证明可以降低延误[11−13]. 但是, Levin 等[14] 指出在过饱和、列队行驶等情况下, FCFS 策略的延误比信号控制更大.
运输工程学院上海 201804, 中国
1. School of Traffic and Transportation Engineering, Changsha University of Science and Technology, Changsha 410004, China 2. School of Computer Science and Engineering, University of New South Wales, Sydney NSW 2052, Australia 3. School of Traffic and Transportation, Beijing Jiao Tong University, Beijing 110091, China 4. School of Traffic and Transportation Engineering, Tongji University, Shanghai 201804, ChinaICA

L M S A l g o r i t h m 最小均方算法 ( 2 0 2 0 )

常用的机器学习&数据挖掘知识点[转]常用的数【导师实战追-女生资-源】据挖掘机器学习知识(点)Basis(基础)：MSE(【扣扣】MeanSquare Error 均方误差)，LMS(Least MeanSquare 最小均方)【⒈】，LSM(Least Square Methods 最小二乘法)，MLE(Maximum Like 【０】lihoodEstimation最大似然估计)，QP(QuadraticProgramming 二次规划【１】)， CP(ConditionalProbability条件概率)，JP(Joint Pro 【б】bability 联合概率)，MP(Marginal Probability边缘概率)，Bay 【９】esian Formula(贝叶斯公式)，L1 -L2Regularization(L1-L2正则，以及更【５】多的，现在比较火的L2.5正则等)，GD(Gradient Descent 梯度下降【２】)，SGD(Stochastic GradientDescent 随机梯度下降)，Eig 【６】envalue(特征值)，Eigenvector(特征向量)，QR-decomposition(QR 分解)，Quantile (分位数)，Covariance(协方差矩阵)。

Common Distribution(常见分布)：Discrete Distribution(离散型分布)：Bernoulli Distribution-Binomial(贝努利分步-二项分布)，Negative BinomialDistribution(负二项分布)，Multinomial Distribution(多式分布)，Geometric Distribution(几何分布)，Hypergeometric Distribution(超几何分布)，Poisson Distribution (泊松分布) ContinuousDistribution (连续型分布)：Uniform Distribution(均匀分布)，Normal Distribution-GaussianDistribution(正态分布-高斯分布)，Exponential Distribution(指数分布)，Lognormal Distribution(对数正态分布)，Gamma Distribution(Gamma分布)，Beta Distribution(Beta分布)，Dirichlet Distribution(狄利克雷分布)，Rayleigh Distribution(瑞利分布)，Cauchy Distribution(柯西分布)，Weibull Distribution (韦伯分布)Three Sampling Distribution(三大抽样分布)：Chi-square Distribution(卡方分布)，t-distribution(t-distribution)，F-distribution(F-分布)Data Pre-processing(数据预处理)：MissingValue Imputation(缺失值填充)，Discretization(离散化)，Mapping(映射)，Normalization(归一化-标准化)。

美国留学数据科学(DS)专业介绍和文书写作

数据科学在商科应用 – Finance （Cont.）
Part II：Consumer Retail Finance • Default prediction, e.g., anti money laundering, credit card fraud, insider trading （Credit Division） • Forecast expected loss / Risk management of assets （Risk Division） • Customer profiling, credit scoring, acquisition, retention, personalized services (Marketing Division)
目录页
• 数据科学专业概论 • 数据科学在商科的应用领域 • Finance / Marketing / Supply Chain / HR • 数据科学核心技能 • 综述 • 技术技能（Math & Statistics / Programming / Database / Machine Learning / Data Mining / Data Cleaning / Visualization / 数学建模） • Soft Skills （Ethics / Problem Solving / Communication / Critical Thinking / Ask right questions / 学习能力） • 申请材料包装技巧 • Coursework / Course Projects / Research Experience / Internship
数据科学在商科应用 – Marketing （Ch. Companies can conduct qualitative and quantitative market research much more quickly and inexpensively than ever before. Online survey tools mean that focus groups and customer feedback are easy and inexpensive to implement, and data analytics make the results easier to parse and take action on. Reputation management. With big data, companies can easily monitor mentions of their brand across many different websites and social channels to find unfiltered opinions, reviews, and testimonials about their organization and products. The savviest can also use social media to provide customer service and create a trustworthy brand presence. Competitor analysis. New social monitoring tools make it easy to collect and analyze data about competitors and their marketing efforts as well. The companies that can use this information will have a distinct competitive advantage.

华为大数据 HCIE-v2.0版本笔试题库

华为大数据 HCIE-v2.0版本笔试题库1.(判断)数据挖掘是通过对大量的数据进行分析，以发现和提取隐含在其中的具有价值的信息和知识的过程 [单选题] *A.TRUE(正确答案)B.FALSE2.数据挖掘的开发工具除了Python以外，还包含以下哪些工具? *A. Spark MLlib(正确答案)B.MLS(机器学习服务)(正确答案)C.IBM SPSS Modeler(正确答案)D.Oracle Data Mining(正确答案)3.以下哪些属于Python运算符? *A.算数运算符(正确答案)B.推理运算符C.逻辑运算符(正确答案)D.比较运算符(正确答案)4.(单选)假设A，B.C是三个矩阵，A是2X2，B是2X2阶，C是3x 2阶，以下哪一个矩阵的运算是有意义的? [单选题] *A.A+B(正确答案)B.ACC.AB+ACD.B+C5.以下关于Python列表的描述正确的是? *A.Python中的列表可以随时进行元素的添加和删除。

(正确答案)B.Python中的列表是可变的，定的元素可以是任何的数据类型。

(正确答案)C.Python中的列表由个括号包裹住元素，元素用逗号隔开。

(正确答案)D.Python中的列表企形式上类似于数组，是一个有序的序列。

6.(单选)以下哪个选项不是矩阵乘法对向量的变换? [单选题] *A.投影B.伸缩C.曲线化(正确答案)D.旋转7.(单选)若随机变量X服从正态分布N(u,o^2)，则随机变量Y=aX+b服从以下哪个正态分布? [单选题] *A.N(a^2 u+b,a^2 o^2)B.N(a u+b,a^2 o^2)(正确答案)C.N(a u+b,a^2 o^2+b)D.N(a u,a^2 o^2)8.与面向过程相比，以下哪些是面向对象的特点? *A.程序可拓展性没有明显的变化B.提高代码复用性(正确答案)C.增加了开发效率(正确答案)D.使程序的编码更加灵活，提高了代码的可维护性(正确答案)9.(单选)以下哪个措施属于反爬措施? [单选题] *A.字体B.滑块验证码C.数据收费D.以上全部正确(正确答案)10.(判断)数据的ETL，其中E为Extract，T为Transform，L为Load。

lpp算法

Locality Preserving ProjectionsXiaofei He Department of Computer Science The University of ChicagoChicago,IL60637 xiaofei@Partha Niyogi Department of Computer Science The University of ChicagoChicago,IL60637 niyogi@AbstractMany problems in information processing involve some form of dimen-sionality reduction.In this paper,we introduce Locality Preserving Pro-jections(LPP).These are linear projective maps that arise by solving avariational problem that optimally preserves the neighborhood structureof the data set.LPP should be seen as an alternative to Principal Com-ponent Analysis(PCA)–a classical linear technique that projects thedata along the directions of maximal variance.When the high dimen-sional data lies on a low dimensional manifold embedded in the ambientspace,the Locality Preserving Projections are obtained byﬁnding theoptimal linear approximations to the eigenfunctions of the Laplace Bel-trami operator on the manifold.As a result,LPP shares many of thedata representation properties of nonlinear techniques such as LaplacianEigenmaps or Locally Linear Embedding.Yet LPP is linear and morecrucially is deﬁned everywhere in ambient space rather than just on thetraining data points.This is borne out by illustrative examples on somehigh dimensional data sets.1.IntroductionSuppose we have a collection of data points of n-dimensional real vectors drawn from an unknown probability distribution.In increasingly many cases of interest in machine learn-ing and data mining,one is confronted with the situation where n is very large.However, there might be reason to suspect that the“intrinsic dimensionality”of the data is much lower.This leads one to consider methods of dimensionality reduction that allow one to represent the data in a lower dimensional space.In this paper,we propose a new linear dimensionality reduction algorithm,called Locality Preserving Projections(LPP).It builds a graph incorporating neighborhood information of the data ing the notion of the Laplacian of the graph,we then compute a trans-formation matrix which maps the data points to a subspace.This linear transformation optimally preserves local neighborhood information in a certain sense.The representation map generated by the algorithm may be viewed as a linear discrete approximation to a con-tinuous map that naturally arises from the geometry of the manifold[2].The new algorithm is interesting from a number of perspectives.1.The maps are designed to minimize a different objective criterion from the classi-cal linear techniques.2.The locality preserving quality of LPP is likely to be of particular use in informa-tion retrieval applications.If one wishes to retrieve audio,video,text documentsunder a vector space model,then one will ultimately need to do a nearest neighborsearch in the low dimensional space.Since LPP is designed for preserving localstructure,it is likely that a nearest neighbor search in the low dimensional spacewill yield similar results to that in the high dimensional space.This makes for anindexing scheme that would allow quick retrieval.3.LPP is linear.This makes it fast and suitable for practical application.While anumber of non linear techniques have properties(1)and(2)above,we know of noother linear projective technique that has such a property.4.LPP is deﬁned everywhere.Recall that nonlinear dimensionality reduction tech-niques like ISOMAP[6],LLE[5],Laplacian eigenmaps[2]are deﬁned only on thetraining data points and it is unclear how to evaluate the map for new test points.In contrast,the Locality Preserving Projection may be simply applied to any newdata point to locate it in the reduced representation space.5.LPP may be conducted in the original space or in the reproducing kernel Hilbertspace(RKHS)into which data points are mapped.This gives rise to kernel LPP. As a result of all these features,we expect the LPP based techniques to be a natural al-ternative to PCA based techniques in exploratory data analysis,information retrieval,and pattern classiﬁcation applications.2.Locality Preserving Projections2.1.The linear dimensionality reduction problemThe generic problem of linear dimensionality reduction is the following.Given a set x1,x2,···,x m in R n,ﬁnd a transformation matrix A that maps these m points to a set of points y1,y2,···,y m in R l(l n),such that y i”represents”x i,where y i=A T x i. Our method is of particular applicability in the special case where x1,x2,···,x m∈Mand M is a nonlinear manifold embedded in R n.2.2.The algorithmLocality Preserving Projection(LPP)is a linear approximation of the nonlinear Laplacian Eigenmap[2].The algorithmic procedure is formally stated below:1.Constructing the adjacency graph:Let G denote a graph with m nodes.We putan edge between nodes i and j if x i and x j are”close”.There are two variations:(a) -neighborhoods.[parameter ∈R]Nodes i and j are connected by an edgeif x i−x j 2< where the norm is the usual Euclidean norm in R n.(b)k nearest neighbors.[parameter k∈N]Nodes i and j are connected by anedge if i is among k nearest neighbors of j or j is among k nearest neighborsof i.Note:The method of constructing an adjacency graph outlined above is correctif the data actually lie on a low dimensional manifold.In general,however,onemight take a more utilitarian perspective and construct an adjacency graph basedon any principle(for example,perceptual similarity for natural signals,hyperlinkstructures for web documents,etc.).Once such an adjacency graph is obtained,LPP will try to optimally preserve it in choosing projections.2.Choosing the weights:Here,as well,we have two variations for weighting theedges.W is a sparse symmetric m×m matrix with W ij having the weight of the edge joining vertices i and j,and0if there is no such edge.(a)Heat kernel.[parameter t ∈R ].If nodes i and j are connected,putW ij =e− x i−x j 2t The justiﬁcation for this choice of weights can be traced back to [2].(b)Simple-minded.[No parameter].W ij =1if and only if vertices i and j are connected by an edge.3.Eigenmaps :Compute the eigenvectors and eigenvalues for the generalized eigen-vector problem:XLX T a =λXDX T a (1)where D is a diagonal matrix whose entries are column (or row,since W is sym-metric)sums of W ,D ii =Σj W ji .L =D −W is the Laplacian matrix.The i th column of matrix X is x i .Let the column vectors a 0,···,a l −1be the solutions of equation (1),ordered ac-cording to their eigenvalues,λ0<···<λl −1.Thus,the embedding is as follows:x i →y i =A T x i ,A =(a 0,a 1,···,a l −1)where y i is a l -dimensional vector,and A is a n ×l matrix.3.Justiﬁcation3.1.Optimal Linear EmbeddingThe following section is based on standard spectral graph theory.See [4]for a comprehen-sive reference and [2]for applications to data representation.Recall that given a data set we construct a weighted graph G =(V,E )with edges connect-ing nearby points to each other.Consider the problem of mapping the weighted graph G to a line so that connected points stay as close together as possible.Let y =(y 1,y 2,···,y m )T be such a map.A reasonable criterion for choosing a ”good”map is to minimize the fol-lowing objective function [2] ij(y i −y j )2W ijunder appropriate constraints.The objective function with our choice of W ij incurs a heavy penalty if neighboring points x i and x j are mapped far apart.Therefore,minimizing it is an attempt to ensure that if x i and x j are ”close”then y i and y j are close as well.Suppose a is a transformation vector,that is,y T =a T X ,where the i th column vector of X is x i .By simple algebra formulation,the objective function can be reduced to 12 ij (y i −y j )2W ij =12 ij (a T x i −a T x j )2W ij = i a T x i D ii x T i a − ija T x i W ij x T j a =a T X (D −W )X T a =a T XLX T awhere X =[x 1,x 2,···,x m ],and D is a diagonal matrix;its entries are column (or row,since W is symmetric)sum of W,D ii =Σj W ij .L =D −W is the Laplacian matrix[4].Matrix D provides a natural measure on the data points.The bigger the value D ii (corresponding to y i )is,the more ”important”is y i .Therefore,we impose a constraint as follows:y T D y =1⇒a T XDX T a =1Finally,the minimization problem reduces to ﬁnding:arg min aa T XDX T a =1a T XLX T aThe transformation vector a that minimizes the objective function is given by the minimum eigenvalue solution to the generalized eigenvalue problem:XLX T a=λXDX T aIt is easy to show that the matrices XLX T and XDX T are symmetric and positive semi-deﬁnite.The vectors a i(i=0,2,···,l−1)that minimize the objective function are given by the minimum eigenvalue solutions to the generalized eigenvalue problem.3.2.Geometrical JustiﬁcationThe Laplacian matrix L(=D−W)forﬁnite graph,or[4],is analogous to the Laplace Beltrami operator L on compact Riemannian manifolds.While the Laplace Beltrami oper-ator for a manifold is generated by the Riemannian metric,for a graph it comes from the adjacency relation.Let M be a smooth,compact,d-dimensional Riemannian manifold.If the manifold is embedded in R n the Riemannian structure on the manifold is induced by the standard Riemannian structure on R n.We are looking here for a map from the manifold to the real line such that points close together on the manifold get mapped close together on the line. Let f be such a map.Assume that f:M→R is twice differentiable.Belkin and Niyogi[2]showed that the optimal map preserving locality can be found by solving the following optimization problem on the manifold:arg minfL2(M)=1M∇f 2which is equivalent to1arg minfL2(M)=1ML(f)fwhere the integral is taken with respect to the standard measure on a Riemannian mani-fold.L is the Laplace Beltrami operator on the manifold,i.e.L f=−div∇(f).Thus, the optimal f has to be an eigenfunction of L.The integral M L(f)f can be discretely approximated by f(X),Lf(X) =f T(X)Lf(X)on a graph,wheref(X)=[f(x1),f(x2,···,f(x m))]T,f T(X)=[f(x1),f(x2,···,f(x m))]If we restrict the map to be linear,i.e.f(x)=a T x,then we havef(X)=X T a⇒ f(X),Lf(X) =f T(X)Lf(X)=a T XLX T aThe constraint can be computed as follows,f 2L2(M)= M|f(x)|2d x= M(a T x)2d x= M(a T xx T a)d x=a T( M xx T d x)a where d x is the standard measure on a Riemannian manifold.By spectral graph theory[4], the measure d x directly corresponds to the measure for the graph which is the degree ofthe vertex,i.e.D ii.Thus,|f 2L2(M)can be discretely approximated as follows,f 2L2(M)=a T( M xx T d x)a≈a T( i xx T D ii)a=a T XDX T aFinally,we conclude that the optimal linear projective map,i.e.f(x)=a T x,can be obtained by solving the following objective function,arg minaa T XDX T a=1a T XLX T a1If M has a boundary,appropriate boundary conditions for f need to be assumed.These projective maps are the optimal linear approximations to the eigenfunctions of the Laplace Beltrami operator on the manifold.Therefore,they are capable of discovering the nonlinear manifold structure.3.3.Kernel LPPSuppose that the Euclidean space R n is mapped to a Hilbert space H through a nonlinear mapping functionφ:R n→H.Letφ(X)denote the data matrix in the Hilbert space,φ(X)=[φ(x1),φ(x2),···,φ(x m)].Now,the eigenvector problem in the Hilbert space can be written as follows:[φ(X)LφT(X)]ν=λ[φ(X)DφT(X)]ν(2)To generalize LPP to the nonlinear case,we formulate it in a way that uses dot product exclusively.Therefore,we consider an expression of dot product on the Hilbert space H given by the following kernel function:K(x i,x j)=(φ(x i)·φ(x j))=φT(x i)φ(x j)Because the eigenvectors of(2)are linear combinations ofφ(x1),φ(x2),···,φ(x m),there exist coefﬁcientsαi,i=1,2,···,m such thatν=mi=1αiφ(x i)=φ(X)αwhereα=[α1,α2,···,αm]T∈R m.By simple algebra formulation,we canﬁnally obtain the following eigenvector problem:KLKα=λKDKα(3) Let the column vectorsα1,α2,···,αm be the solutions of equation(3).For a test point x, we compute projections onto the eigenvectorsνk according to(νk·φ(x))=mi=1αk i(φ(x)·φ(x i))=mi=1αk i K(x,x i)whereαk i is the i th element of the vectorαk.For the original training points,the maps can be obtained by y=Kα,where the i th element of y is the one-dimensional representation of x i.Furthermore,equation(3)can be reduced toL y=λD y(4) which is identical to the eigenvalue problem of Laplacian Eigenmaps[2].This shows that Kernel LPP yields the same results as Laplacian Eigenmaps on the training points.4.Experimental ResultsIn this section,we will discuss several applications of the LPP algorithm.We begin with two simple synthetic examples to give some intuition about how LPP works.4.1.Simply Synthetic ExampleTwo simple synthetic examples are given in Figure1.Both of the two data sets corre-spond essentially to a one-dimensional manifold.Projection of the data points onto the ﬁrst basis would then correspond to a one-dimensional linear manifold representation.The second basis,shown as a short line segment in theﬁgure,would be discarded in this low-dimensional example.Figure1:Theﬁrst and third plots show the results of PCA.The second and forth plots show the results of LPP.The line segments describe the two bases.Theﬁrst basis is shown as a longer line segment,and the second basis is shown as a shorter line segment.In this example,LPP is insensitive to the outlier and has more discriminating power than PCA.Figure2:The handwritten digits(‘0’-‘9’)are mapped into a2-dimensional space.The left ﬁgure is a representation of the set of all images of digits using the Laplacian eigenmaps. The middleﬁgure shows the results of LPP.The rightﬁgure shows the results of PCA.Each color corresponds to a digit.LPP is derived by preserving local information,hence it is less sensitive to outliers than PCA.This can be clearly seen from Figure1.LPPﬁnds the principal direction along the data points at the left bottom corner,while PCAﬁnds the principal direction on which the data points at the left bottom corner collapse into a single point.Moreover,LPP can has more discriminating power than PCA.As can be seen from Figure1,the two circles are totally overlapped with each other in the principal direction obtained by PCA,while they are well separated in the principal direction obtained by LPP.4.2.2-D Data VisulizationAn experiment was conducted with the Multiple Features Database[3].This dataset con-sists of features of handwritten numbers(‘0’-‘9’)extracted from a collection of Dutch utility maps.200patterns per class(for a total of2,000patterns)have been digitized in binary images.Digits are represented in terms of Fourier coefﬁcients,proﬁle correlations, Karhunen-Love coefﬁcients,pixel average,Zernike moments and morphological features. Each image is represented by a649-dimensional vector.These data points are mapped to a2-dimensional space using different dimensionality reduction algorithms,PCA,LPP,and Laplacian Eigenmaps.The experimental results are shown in Figure2.As can be seen, LPP performs much better than PCA.LPPs are obtained byﬁnding the optimal linear ap-proximations to the eigenfunctions of the Laplace Beltrami operator on the manifold.As a result,LPP shares many of the data representation properties of non linear techniques such as Laplacian Eigenmap.However,LPP is computationally much more tractable.4.3.Manifold of Face ImagesIn this subsection,we applied the LPP to images of faces.The face image data set used here is the same as that used in[5].This dataset contains1965face images taken from sequential frames of a small video.The size of each image is20×28,with256gray levelsFigure3:A two-dimensional repre-sentation of the set ofall images of facesusing the LocalityPreserving Projec-tion.Representativefaces are shown nextto the data pointsin different partsof the space.Ascan be seen,thefacial expressionand the viewingpoint of faces changesmoothly.Table1:Face Recognition Results on Yale DatabaseLPP LDA PCAdims141433error rate(%)16.020.025.3per pixel.Thus,each face image is represented by a point in the560-dimensional ambi-ent space.Figure3shows the mapping results.The images of faces are mapped into the 2-dimensional plane described by theﬁrst two coordinates of the Locality Preserving Pro-jections.It should be emphasized that the mapping from image space to low-dimensional space obtained by our method is linear,rather than nonlinear as in most previous work.The linear algorithm does detect the nonlinear manifold structure of images of faces to some extent.Some representative faces are shown next to the data points in different parts of the space.As can be seen,the images of faces are clearly divided into two parts.The left part are the faces with closed mouth,and the right part are the faces with open mouth.This is because that,by trying to preserve neighborhood structure in the embedding,the LPP algorithm implicitly emphasizes the natural clusters in the data.Speciﬁcally,it makes the neighboring points in the ambient space nearer in the reduced representation space,and faraway points in the ambient space farther in the reduced representation space.The bot-tom images correspond to points along the right path(linked by solid line),illustrating one particular mode of variability in pose.4.4.Face RecognitionPCA and LDA are the two most widely used subspace learning techniques for face recog-nition[1][7].These methods project the training sample faces to a low dimensional rep-resentation space where the recognition is carried out.The main supposition behind this procedure is that the face space(given by the feature vectors)has a lower dimension than the image space(given by the number of pixels in the image),and that the recognition of the faces can be performed in this reduced space.In this subsection,we consider the application of LPP to face recognition.The database used for this experiment is the Yale face database[8].It is constructed at theYale Center for Computational Vision and Control.It contains165grayscale images of 15individuals.The images demonstrate variations in lighting condition(left-light,center-light,right-light),facial expression(normal,happy,sad,sleepy,surprised,and wink),and with/without glasses.Preprocessing to locate the the faces was applied.Original images were normalized(in scale and orientation)such that the two eyes were aligned at the same position.Then,the facial areas were cropped into theﬁnal images for matching.The size of each cropped image is32×32pixels,with256gray levels per pixel.Thus,each image can be represented by a1024-dimensional vector.For each individual,six images were taken with labels to form the training set.The rest of the database was considered to be the testing set.The training samples were used to learn a projection.The testing samples were then projected into the reduced space.Recognition was performed using a nearest neighbor classiﬁer.In general,the performance of PCA, LDA and LPP varies with the number of dimensions.We show the best results obtained by them.The error rates are summarized in Table1.As can be seen,LPP outperforms both PCA and LDA.5.ConclusionsIn this paper,we propose a new linear dimensionality reduction algorithm called Locality Preserving Projections.It is based on the same variational principle that gives rise to the Laplacian Eigenmap[2].As a result it has similar locality preserving properties.Our approach also has several possible advantages over recent nonparametric techniques for global nonlinear dimensionality reduction such as[2][5][6].It yields a map which is simple,linear,and deﬁned everywhere(and therefore on novel test data points).The algorithm can be easily kernelized yielding a natural non-linear extension. Performance improvement of this method over Principal Component Analysis is demon-strated through several experiments.Though our method is a linear algorithm,it is capable of discovering the non-linear structure of the data manifold.References[1]P.N.Belhumeur,J.P.Hepanha,and D.J.Kriegman,“Eigenfaces vs.ﬁsherfaces:recog-nition using class speciﬁc linear projection,”IEEE.Trans.Pattern Analysis and Ma-chine Intelligence,vol.19,no.7,pp.711-720,July1997.[2]M.Belkin and P.Niyogi,“Laplacian Eigenmaps and Spectral Techniques for Em-bedding and Clustering,”Advances in Neural Information Processing Systems14, Vancouver,British Columbia,Canada,2002.[3] C.L.Blake and C.J.Merz,”UCI repository of machine learning databases”,/mlearn/MLRepository.html.Irvine,CA,University of Cali-fornia,Department of Information and Computer Science,1998.[4]Fan R.K.Chung,Spectral Graph Theory,Regional Conference Series in Mathemat-ics,number92,1997.[5]Sam Roweis,and Lawrence K.Saul,“Nonlinear Dimensionality Reduction by Lo-cally Linear Embedding,”Science,vol290,22December2000.[6]Joshua B.Tenenbaum,Vin de Silva,and John ngford,“A Global GeometricFramework for Nonlinear Dimensionality Reduction,”Science,vol290,22December 2000.[7]M.Turk and A.Pentland,“Eigenfaces for recognition,”Journal of Cognitive Neuro-science,3(1):71-86,1991.[8]Yale Univ.Face Database,/projects/yalefaces/yalefaces.html.。

机器学习考试卷 final2006

10-701/15-781,Fall2006,FinalDec15,5:30pm-8:30pm•There are9questions in this exam(15pages including this cover sheet).•If you need more room to work out your answer to a question,use the back of the page and clearly mark on the front of the page if we are to look at what’s on the back.•This exam is open book and open puters,PDAs,cell phones are not allowed.•You have3hours.Best luck!Name:Andrew ID:Q Topic Max.Score Score1Short Questions202Instance-Based Learning73Computational Learning Theory94Gaussian Mixture Models105Bayesian Networks106Hidden Markov Models127Dimensionality Reduction88Graph-Theoretic Clustering89MDPs and Reinforcement Learning16Total10011Short Questions(20pts,2pts each)(a)True or False.The ID3algorithm is guaranteed toﬁnd the optimal decision tree.(b)True or False.Consider a continuous probability distribution with density f()that is nonzeroeverywhere.The probability of a value x is equal to f(x).(c)True or False.In a Bayesian network,the inference results of the junction tree algorithm arethe same as the inference results of variable elimination.(d)True or False.If two random variable X and Y are conditionally independent given anotherrandom variable Z,then in the corresponding Bayesian network,the nodes for X and Y are d-separated given Z.(e)True or False.Besides EM,gradient descent can be used to perform inference or learning ona Gaussian mixture model.(f)In one sentence,characterize the diﬀerences between maximum likelihood and maximum aposteriori approaches.(g)In one sentence,characterize the diﬀerences between classiﬁcation and regression.(h)Give one similarity and one diﬀerence between feature selection and PCA.(i)Give one similarity and one diﬀerence between HMM and MDP.(j)For each of the following datasets,is it appropriate to use HMM?Provide a brief reasoning for your answer.•Gene sequence dataset.•A database of movie reviews(eg.,the IMDB database).•Stock market price dataset.•Daily precipitation data from the Northwest of the US.22Instance-Based Learning (7pts)1.Consider the following training set in the 2-dimensional Euclidean space:x y Class−11−01+02−1−1−10+12+22−23+Figure 1shows a visualization of the data.−2−10123−2−11234x yFigure 1:Dataset for Problem 2(a)(1pt)What is the prediction of the 3-nearest-neighbor classiﬁer at the point (1,1)?(b)(1pt)What is the prediction of the 5-nearest-neighbor classiﬁer at the point (1,1)?(c)(1pt)What is the prediction of the 7-nearest-neighbor classiﬁer at the point (1,1)?32.Consider the two-class classiﬁcation problem.At a data point x,the true conditional proba-bility of a class k,k∈{0,1}is p k(x)=P(C=k|X=x).(a)(2pts)The Bayes error is the probability that an optimal Bayes classiﬁer will misclassifya randomly drawn example.In terms of p k(x),what is the Bayes error E∗at x?(b)(2pts)In terms of p k(x)and p k(x )when x is the nearest neighbor of x,what is the1-nearest-neighbor error E1NN at x?Note that asymptotically as the number of training examples grows,E∗≤E1NN≤2E∗.43Computational Learning Theory(9pts,3pts each)In class we discussed diﬀerent formula to provide a bound on the number of training examples suﬃcient for successful learning under diﬀerent learning models.m≥1(ln(1/δ)+ln|H|)(1)m≥12 2(ln(1/δ)+ln|H|)(2)m≥1(4log2(2/δ)+8V C(H)log2(13/ ))(3)Pick the appropriate one of the above formula to estimate the number of training examples needed for the following machine learning tasks.Brieﬂy explain your choice.1.Consider instances X containing5Boolean variables,{X1,X2,X3,X4,X5},and responses Yare(X1∧X4)∨(X2∧X3).We try to learn the function f:X→Y using a2-layered neural network.2.Consider instances X containing5Boolean variables,{X1,X2,X3,X4,X5},and responses Yare(X1∧X4)∨(X2∧X3).We try to learn the function f:X→Y using a“depth-2decision trees”.A“depth-2decision tree”is a tree with four leaves,all distance2from the root.3.Consider instances X containing5Boolean variables,{X1,X2,X3,X4,X5},and responses Yare(X1∧X4)∨(¬X1∧X3).We try to learn the function f:X→Y using a“depth-2decision trees”.A“depth-2decision tree”is a tree with four leaves,all distance2from the root.54Gaussian Mixture Model (10pts)Consider the labeled training points in Figure 2,where ‘+’and ‘o’denote positive and negative labels,respectively.Tom asks three students (Yifen,Fan and Indra)to ﬁt Gaussian Mixture Models on this dataset.012345012345X1X 2Figure 2:Dataset for Gaussian Mixture Model1.(4pts)Yifen and Fan decide to use one Gaussian distribution for positive examples and one distribution for negative examples.The darker ellipse indicates the positive Gaussian distri-bution contour and the lighter ellipse indicates the negative Gaussian distribution contour.012345012345X1X 2012345012345X1X 2Yifen’s model Fan’s modelWhose model would you prefer for this dataset?What causes the diﬀerence between these two models?62.(6pts)Indra decides to use two Gaussian distributions for positive examples and two Gaussian distributions for negative examples.He uses EM algorithm to iteratively update parameters and also tries diﬀerent initializations.The left column of Figure 3shows 3diﬀerent initial-izations and the right column shows 3possible models after the ﬁrst iteration.For each initialization on the left,draw an arrow to the model on the right that will result after the ﬁrst EM iteration.Your answer should consist of 3arrows,one from each initialization.012345012345X1X 2012345012345X1X 2012345012345X1X 2012345012345X1X 2012345012345X1X 2012345012345X1X 2(a)Initialization (b)After ﬁrst iterationFigure 3:Three diﬀerent initializations and models after the ﬁrst iteration.75Bayesian Networks(10pts)Theﬁgure below shows a Bayesian network with9variables,all of which are binary.1.(3pts)Which of the following statements are always true for this Bayes net?(a)P(A,B|G)=P(A|G)P(B|G);(b)P(A,I)=P(A)P(I);(c)P(B,H|E,G)=P(B|E,G)P(H|E,G);(d)P(C|B,F)=P(C|F).2.(2pts)What is the number of independent parameters in this graphical model?3.(3pts)The computational complexity of a graph elimination algorithm is determined by thesize of the maximal elimination clique produced in the elimination process.What is the minimum size of such maximal elimination clique when we choose a perfect elimination order to compute P(C=1)using the graph elimination algorithm?4.(2pts)We would like to computeµ=P(F=1|A,B,C,D,E,G,H,I) P(F=0|A,B,C,D,E,G,H,I)The value ofµdepends on the values of all the variables other than F.What is the maximum possible number of diﬀerent values ofµ?*Given the value ofµ,as in the setting of Gibbs sampling,we could draw the random variable F from a Bernoulli distribution:F∼Bernoulli[1/(1+µ−1)].86Hidden Markov Models(12pts)Consider an HMM with states Y t∈{S1,S2,S3},observations X t∈{A,B,C},and parameters π1=1a11=1/2a12=1/4a13=1/4b1(A)=1/2b1(B)=1/2b1(C)=0π2=0a21=0a22=1/2a23=1/2b2(A)=1/2b2(B)=0b2(C)=1/2π3=0a31=0a32=0a33=1b3(A)=0b3(B)=1/2b3(C)=1/2(a)(3pts)What is P(Y5=S3)?For6(b)-(d),suppose we observe AABCABC,starting at time point1.(b)(2pts)What is P(Y5=S3|X1:7=AABCABC)?(c)(4pts)Fill in the following table assuming the observation AABCABC.Theα’s are valuesobtained during the forward algorithm:αt(i)=P(X1,...,X t,Y t=i).tαt(1)αt(2)αt(3)1234567(d)(3pts)Write down the sequence of Y1:7with the maximal posterior probability assuming theobservation AABCABC.What is that posterior probability?97Dimensionality Reduction (8pts)In this problem four linear dimensionality reduction methods will be discussed.They are princi-pal component analysis (PCA),linear discriminant analysis (LDA),canonical correlation analysis (CCA),non-negative matrix factorization (NMF).1.(3pts)LDA reduces the dimensionality given labels by maximizing the overall interclass vari-ance relative to intraclass variance .Plot the directions of the ﬁrst PCA and LDA components in the following ﬁgures respectively.−2−10123−3−2−10123−3−2−11231(a)First PCA component 1(b)First LDA component2.(2pts)In practice,each data point may have multiple vector-valued properties,e.g.a gene has its expression levels as well as the position on the genome.The goal of CCA is to reduce the dimensionality of the properties jointly.Suppose we have data points with two properties x and y ,each of which is a 2-dimension vector.This 4-dimensional data is shown in the pair of ﬁgures below;diﬀerent data points are shown in diﬀerent gray A ﬁnds (u ,v )to maximize the correlation corr (u T x )(v T y ).In ﬁgure 2(b)we have given the direction of vector v ,plot the vector u in ﬁgure 2(a).2(a)2(b)103.(3pts)The goal of NMF is to reduce the dimensionality given non-negativity constraints.That is,we would like to ﬁnd principle components u 1,...,u r ,each of which is of dimension d >r ,such that the d -dimensional data x ≈ r i =1z i u i ,and all entries in x ,z ,u 1:r are non-negative.NMF tends to ﬁnd sparse (usually small L1norm)basis vectors u i ’s .Below is an example of applying PCA and NMF on a face image.Please point out the basis vectors in the equations and give them correct labels (NMF or PCA).(∗Figures in 7-2,7-3are originally from /∼asimma/294-fall06/lectures/dimension/talk-maximal-1x2.pdf .)118Graph-Theoretic Clustering(8pts)Part A.Min-Cut and Normalized CutIn this problem,we consider the2-clustering problem,in which we have N data points x1:N to be grouped in two clusters,denoted by A and B.Given the N by N aﬃnity matrix W,•Min-Cut:minimizesi∈Aj∈BW ij;•Normalized Cut:minimizes Pi∈APj∈BW ijPi∈AP Nj=1W ij+Pi∈APj∈BW ijP Ni=1Pj∈BW ij.(A1)(A2)A1.(2pts)The data points are shown in Figure(A1)above.The grid unit is1.Let W ij= e− x i−x j 22,give the clustering results of min-cut and normalized cut respectively(You may show your work in theﬁgure directly).A2.(2pts)The data points are shown in Figure(A2)above.The grid unit is1.Let W ij=e − x i−x j 222σ2,describe the clustering results of min-cut algorithm forσ2=50andσ2=0.5respectively.12Part B.Spectral ClusteringNow back to the setting of the2-clustering problem A1.The grid unit is1.B1.(2pts)If we use Euclidean distance to construct the aﬃnity matrix W as follows:W ij=1if x i−x j 22≤σ20otherwiseWhatσ2value would you choose?Brieﬂy explain.B2.(2pts)The next step is to compute the k=2dominant eigenvectors of the aﬃnity matrix W.For the value ofσ2you chose in the previous question,can you compute analytically eigenvalues corresponding to theﬁrst two eigenvectors?If yes,compute and report the eigenvalues.If not,brieﬂy explain.B3.*(1Extra Credit,please try this question after youﬁnished others!)Suppose the data is of very high dimension so that it is impossible to visualize them and picka good value as we did in Part B1.Suggest a heuristic that couldﬁnd an appropriateσ2.139MDPs and Reinforcement Learning[16pts]Part A.[10pts]Consider the following deterministic Markov Decision Process(MDP),describing a simple robot grid world.Notice the values of the immediate rewards are written next to transitions.Transitions with no value have an immediate reward of0.Assume the discount factorγ=0.8.A1.(2pts)For each state s,write the value for V∗(s)inside the corresponding square in the diagram.A2.(2pts)Mark the state-action transition arrows that correspond to one optimal policy.If there is a tie,always choose the state with the smallest index.A3.(2pts)Give a diﬀerent value forγwhich results in a diﬀerent optimal policy and the number of changed policy actions should be minimal.Give your new value forγ,and describe the resulting policy by indicating whichπ(s)values(i.e.,which policy actions)change.New value forγ:Changed policy actions:For the remainder of this question,assume again thatγ=0.8.A4.(2pts)How many complete loops(iterations)of value iteration are suﬃcient to guarantee ﬁnding the optimal policy for this MDP?Assume that values are initialized to zero,and that states are considered in an arbitrary order on each iteration.A5.(2pts)Is it possible to change the immediate reward function so that V∗changes but the optimal policyπ∗remains unchanged?If yes,give such a change,and describe the resulting change to V∗.Otherwise,explain in at most2sentences why this is impossible.14Part B.(6pts)It is December.Unfortunately for our robot,a patch of ice has appeared in its world,making one of its actions non-deterministic.The resulting MDP is shown below.Note that now the result of the action“go north”from state s6results in one of two outcomes.With probability p the robot succeeds in transitioning to state s3and receives immediate reward100.However,with probability (1−p)it slips on the ice,and remains in state s6with zero immediate reward.Assume the discount factorγ=0.8.B1.(4pts)Assume p=0.7.Write in the values of V∗for each state,and circle the actions in the optimal policy.B2.(2pts)How bad does the ice have to get before the robot will prefer to completely avoid it?Answer this question by giving a value for p below which the optimal policy chooses actions that completely avoid the ice,even choosing the action“go west”over“go north”when the robot is in state s6.15。

lfm降维方法

lfm降维方法Linear Factor Models (LFM) are widely used in various fields such as machine learning, signal processing, and finance. LFM降维方法广泛用于机器学习，信号处理和金融等各个领域。

They are used to reduce the dimensionality of data by projecting it onto a lower-dimensional space using a linear transformation. 它们通过线性变换将数据投影到较低维度的空间中，从而降低数据的维度。

This allows for easier visualization and analysis of the data, as well as potentially reducing the computational complexity of algorithms that operate on the data. 这使得数据更容易可视化和分析，同时可能降低处理数据的算法的计算复杂性。

One of the key advantages of LFM is its ability to uncover underlying patterns or structures in high-dimensional data. LFM的关键优势之一是它能够发现高维数据中的潜在模式或结构。

By representing the datain a lower-dimensional space, LFM can reveal relationships between data points that may not be apparent in the original high-dimensional space. 通过在较低维度的空间中表示数据，LFM可以揭示出数据点之间隐藏的关系，而这些关系在原始的高维空间中可能并不明显。