Session 3-permutaiton combination probability 排列组合和概率

合集下载

泊松融合原理和python代码

泊松融合原理和python代码【原创版】目录1.泊松融合原理概述2.Python 代码实现泊松融合原理3.泊松融合原理的应用正文1.泊松融合原理概述泊松融合原理是一种概率论中的经典理论，由法国数学家泊松提出。

泊松融合原理描述了一个事件在特定时间间隔内发生的概率与另一个事件在相同时间间隔内发生的次数之间的关系。

具体而言，泊松融合原理表明，一个事件在时间间隔Δt 内发生的次数服从泊松分布，即P(X=k)=e^(-λΔt) * (λΔt)^k / k!，其中λ是事件的平均发生率，X 是事件在时间间隔Δt 内发生的次数，k 是事件发生的次数。

2.Python 代码实现泊松融合原理为了验证泊松融合原理，我们可以使用 Python 编写代码模拟事件的发生过程。

以下是一个简单的 Python 代码示例：```pythonimport randomimport mathdef poisson_fusion(lambda_value, dt, num_trials):"""泊松融合原理模拟:param lambda_value: 事件的平均发生率:param dt: 时间间隔:param num_trials: 模拟次数:return: 事件在时间间隔内发生的次数"""count = 0for _ in range(num_trials):# 随机生成一个 0 到 dt 之间的时间间隔t = random.uniform(0, dt)# 计算在时间间隔内事件发生的次数k = int(lambda_value * t)# 计算泊松分布的概率poisson_prob = math.exp(-lambda_value * t) * (lambda_value * t) ** k / k# 根据泊松分布概率随机生成事件发生的次数count += random.choice([0, 1, 2, 3, 4],p=poisson_prob)return count# 示例lambda_value = 1 # 事件的平均发生率为 1dt = 1 # 时间间隔为 1um_trials = 1000 # 模拟次数为 1000# 模拟事件在时间间隔内发生的次数counts = [poisson_fusion(lambda_value, dt, num_trials) for _in range(num_trials)]# 计算平均值和方差mean = sum(counts) / num_trialsvariance = sum((count - mean) ** 2 for count in counts) / num_trialsprint("平均值:", mean)print("方差:", variance)```3.泊松融合原理的应用泊松融合原理在实际应用中有很多场景，例如统计学、保险、生物学等领域。

孟德尔随机化extract clump access代码 -回复

孟德尔随机化extract clump access代码-回复【孟德尔随机化extract clump access代码】是指利用孟德尔随机化方法来提取并分析数据集中的束状访问（clump access）模式的一种编程代码。

在这篇文章中，我将一步一步回答有关这个主题的问题，包括介绍孟德尔随机化方法、解释什么是束状访问模式，以及如何使用代码来提取和分析这种模式。

第一部分：孟德尔随机化方法孟德尔随机化方法是一种在实验设计中用来产生随机对照组的统计学方法。

它的主要原理是通过将研究对象随机分为实验组和对照组，来控制潜在的干扰因素，进而评估因果关系。

在这种方法中，随机分组是非常关键的，因为它可以确保实验组和对照组在其他特征上的相似性，从而降低实验结果的偏见。

第二部分：什么是束状访问模式假设我们有一个数据集，其中包含多个变量。

束状访问模式是指在数据集中出现的一种特殊模式，即某些变量在数据中具有聚集的趋势。

这种模式的存在可能有助于我们理解数据中的内在结构和关系。

第三部分：提取束状访问模式的代码下面是一个用于提取和分析束状访问模式的代码示例：import numpy as npimport pandas as pddef extract_clump_access(data):# 计算每个变量的均值means = data.mean(axis=0)# 根据均值将数据分为两组data_group1 = data[data >= means]data_group2 = data[data < means]# 分别计算每组中变量的方差variances_group1 = data_group1.var(axis=0)variances_group2 = data_group2.var(axis=0)# 根据方差的大小判断是否存在束状访问模式clump_access = []for i in range(len(data.columns)):if variances_group1[i] > variances_group2[i]:clump_access.append(data.columns[i])return clump_access在这段代码中，我们首先导入了numpy和pandas库，以便进行数组和数据框的操作。

残差半循环神经网络[发明专利]

专利名称：残差半循环神经网络专利类型：发明专利
发明人：汤琦,祁褎然
申请号：CN202080036830.3申请日：20200323
公开号：CN114175052A
公开日：
20220311
专利内容由知识产权出版社提供
摘要：残差半循环神经网络(RSNN)可以被配置成接收时不变输入和时变输入数据以生成一个或多个时间序列预测。

所述时不变输入可以由所述RSNN的多层感知器处理。

所述多层感知器的输出可以用作所述RSNN的循环神经网络单元的初始状态。

循环神经网络单元还可以接收时不变输入，并且利用所述时不变输入处理所述时不变输入以生成输出。

所述多层感知器和所述循环神经网络单元的输出可以被组合以生成所述一个或多个时间序列预测。

申请人：赛诺菲
地址：法国巴黎
国籍：FR
代理机构：北京坤瑞律师事务所
代理人：封新琴
更多信息请下载全文后查看。

ABSTRACT Progressive Simplicial Complexes

Progressive Simplicial Complexes Jovan Popovi´c Hugues HoppeCarnegie Mellon University Microsoft ResearchABSTRACTIn this paper,we introduce the progressive simplicial complex(PSC) representation,a new format for storing and transmitting triangu-lated geometric models.Like the earlier progressive mesh(PM) representation,it captures a given model as a coarse base model together with a sequence of reﬁnement transformations that pro-gressively recover detail.The PSC representation makes use of a more general reﬁnement transformation,allowing the given model to be an arbitrary triangulation(e.g.any dimension,non-orientable, non-manifold,non-regular),and the base model to always consist of a single vertex.Indeed,the sequence of reﬁnement transforma-tions encodes both the geometry and the topology of the model in a uniﬁed multiresolution framework.The PSC representation retains the advantages of PM’s.It deﬁnes a continuous sequence of approx-imating models for runtime level-of-detail control,allows smooth transitions between any pair of models in the sequence,supports progressive transmission,and offers a space-efﬁcient representa-tion.Moreover,by allowing changes to topology,the PSC sequence of approximations achieves betterﬁdelity than the corresponding PM sequence.We develop an optimization algorithm for constructing PSC representations for graphics surface models,and demonstrate the framework on models that are both geometrically and topologically complex.CR Categories:I.3.5[Computer Graphics]:Computational Geometry and Object Modeling-surfaces and object representations.Additional Keywords:model simpliﬁcation,level-of-detail representa-tions,multiresolution,progressive transmission,geometry compression.1INTRODUCTIONModeling and3D scanning systems commonly give rise to triangle meshes of high complexity.Such meshes are notoriously difﬁcult to render,store,and transmit.One approach to speed up rendering is to replace a complex mesh by a set of level-of-detail(LOD) approximations;a detailed mesh is used when the object is close to the viewer,and coarser approximations are substituted as the object recedes[6,8].These LOD approximations can be precomputed Work performed while at Microsoft Research.Email:jovan@,hhoppe@Web:/jovan/Web:/hoppe/automatically using mesh simpliﬁcation methods(e.g.[2,10,14,20,21,22,24,27]).For efﬁcient storage and transmission,meshcompression schemes[7,26]have also been developed.The recently introduced progressive mesh(PM)representa-tion[13]provides a uniﬁed solution to these problems.In PM form,an arbitrary mesh M is stored as a coarse base mesh M0together witha sequence of n detail records that indicate how to incrementally re-ﬁne M0into M n=M(see Figure7).Each detail record encodes theinformation associated with a vertex split,an elementary transfor-mation that adds one vertex to the mesh.In addition to deﬁninga continuous sequence of approximations M0M n,the PM rep-resentation supports smooth visual transitions(geomorphs),allowsprogressive transmission,and makes an effective mesh compressionscheme.The PM representation has two restrictions,however.First,it canonly represent meshes:triangulations that correspond to orientable12-dimensional manifolds.Triangulated2models that cannot be rep-resented include1-d manifolds(open and closed curves),higherdimensional polyhedra(e.g.triangulated volumes),non-orientablesurfaces(e.g.M¨o bius strips),non-manifolds(e.g.two cubes joinedalong an edge),and non-regular models(i.e.models of mixed di-mensionality).Second,the expressiveness of the PM vertex splittransformations constrains all meshes M0M n to have the same topological type.Therefore,when M is topologically complex,the simpliﬁed base mesh M0may still have numerous triangles(Fig-ure7).In contrast,a number of existing simpliﬁcation methods allowtopological changes as the model is simpliﬁed(Section6).Ourwork is inspired by vertex uniﬁcation schemes[21,22],whichmerge vertices of the model based on geometric proximity,therebyallowing genus modiﬁcation and component merging.In this paper,we introduce the progressive simplicial complex(PSC)representation,a generalization of the PM representation thatpermits topological changes.The key element of our approach isthe introduction of a more general reﬁnement transformation,thegeneralized vertex split,that encodes changes to both the geometryand topology of the model.The PSC representation expresses anarbitrary triangulated model M(e.g.any dimension,non-orientable,non-manifold,non-regular)as the result of successive reﬁnementsapplied to a base model M1that always consists of a single vertex (Figure8).Thus both geometric and topological complexity are recovered progressively.Moreover,the PSC representation retains the advantages of PM’s,including continuous LOD,geomorphs, progressive transmission,and model compression.In addition,we develop an optimization algorithm for construct-ing a PSC representation from a given model,as described in Sec-tion4.1The particular parametrization of vertex splits in[13]assumes that mesh triangles are consistently oriented.2Throughout this paper,we use the words“triangulated”and“triangula-tion”in the general dimension-independent sense.Figure 1:Illustration of a simplicial complex K and some of its subsets.2BACKGROUND2.1Concepts from algebraic topologyTo precisely deﬁne both triangulated models and their PSC repre-sentations,we ﬁnd it useful to introduce some elegant abstractions from algebraic topology (e.g.[15,25]).The geometry of a triangulated model is denoted as a tuple (K V )where the abstract simplicial complex K is a combinatorial structure specifying the adjacency of vertices,edges,triangles,etc.,and V is a set of vertex positions specifying the shape of the model in 3.More precisely,an abstract simplicial complex K consists of a set of vertices 1m together with a set of non-empty subsets of the vertices,called the simplices of K ,such that any set consisting of exactly one vertex is a simplex in K ,and every non-empty subset of a simplex in K is also a simplex in K .A simplex containing exactly d +1vertices has dimension d and is called a d -simplex.As illustrated pictorially in Figure 1,the faces of a simplex s ,denoted s ,is the set of non-empty subsets of s .The star of s ,denoted star(s ),is the set of simplices of which s is a face.The children of a d -simplex s are the (d 1)-simplices of s ,and its parents are the (d +1)-simplices of star(s ).A simplex with exactly one parent is said to be a boundary simplex ,and one with no parents a principal simplex .The dimension of K is the maximum dimension of its simplices;K is said to be regular if all its principal simplices have the same dimension.To form a triangulation from K ,identify its vertices 1m with the standard basis vectors 1m ofm.For each simplex s ,let the open simplex smdenote the interior of the convex hull of its vertices:s =m:jmj =1j=1jjsThe topological realization K is deﬁned as K =K =s K s .The geometric realization of K is the image V (K )where V :m 3is the linear map that sends the j -th standard basis vector jm to j 3.Only a restricted set of vertex positions V =1m lead to an embedding of V (K )3,that is,prevent self-intersections.The geometric realization V (K )is often called a simplicial complex or polyhedron ;it is formed by an arbitrary union of points,segments,triangles,tetrahedra,etc.Note that there generally exist many triangulations (K V )for a given polyhedron.(Some of the vertices V may lie in the polyhedron’s interior.)Two sets are said to be homeomorphic (denoted =)if there ex-ists a continuous one-to-one mapping between them.Equivalently,they are said to have the same topological type .The topological realization K is a d-dimensional manifold without boundary if for each vertex j ,star(j )=d .It is a d-dimensional manifold if each star(v )is homeomorphic to either d or d +,where d +=d:10.Two simplices s 1and s 2are d-adjacent if they have a common d -dimensional face.Two d -adjacent (d +1)-simplices s 1and s 2are manifold-adjacent if star(s 1s 2)=d +1.Figure 2:Illustration of the edge collapse transformation and its inverse,the vertex split.Transitive closure of 0-adjacency partitions K into connected com-ponents .Similarly,transitive closure of manifold-adjacency parti-tions K into manifold components .2.2Review of progressive meshesIn the PM representation [13],a mesh with appearance attributes is represented as a tuple M =(K V D S ),where the abstract simpli-cial complex K is restricted to deﬁne an orientable 2-dimensional manifold,the vertex positions V =1m determine its ge-ometric realization V (K )in3,D is the set of discrete material attributes d f associated with 2-simplices f K ,and S is the set of scalar attributes s (v f )(e.g.normals,texture coordinates)associated with corners (vertex-face tuples)of K .An initial mesh M =M n is simpliﬁed into a coarser base mesh M 0by applying a sequence of n successive edge collapse transforma-tions:(M =M n )ecol n 1ecol 1M 1ecol 0M 0As shown in Figure 2,each ecol uniﬁes the two vertices of an edgea b ,thereby removing one or two triangles.The position of the resulting uniﬁed vertex can be arbitrary.Because the edge collapse transformation has an inverse,called the vertex split transformation (Figure 2),the process can be reversed,so that an arbitrary mesh M may be represented as a simple mesh M 0together with a sequence of n vsplit records:M 0vsplit 0M 1vsplit 1vsplit n 1(M n =M )The tuple (M 0vsplit 0vsplit n 1)forms a progressive mesh (PM)representation of M .The PM representation thus captures a continuous sequence of approximations M 0M n that can be quickly traversed for interac-tive level-of-detail control.Moreover,there exists a correspondence between the vertices of any two meshes M c and M f (0c f n )within this sequence,allowing for the construction of smooth vi-sual transitions (geomorphs)between them.A sequence of such geomorphs can be precomputed for smooth runtime LOD.In addi-tion,PM’s support progressive transmission,since the base mesh M 0can be quickly transmitted ﬁrst,followed the vsplit sequence.Finally,the vsplit records can be encoded concisely,making the PM representation an effective scheme for mesh compression.Topological constraints Because the deﬁnitions of ecol and vsplit are such that they preserve the topological type of the mesh (i.e.all K i are homeomorphic),there is a constraint on the min-imum complexity that K 0may achieve.For instance,it is known that the minimal number of vertices for a closed genus g mesh (ori-entable 2-manifold)is (7+(48g +1)12)2if g =2(10if g =2)[16].Also,the presence of boundary components may further constrain the complexity of K 0.Most importantly,K may consist of a number of components,and each is required to appear in the base mesh.For example,the meshes in Figure 7each have 117components.As evident from the ﬁgure,the geometry of PM meshes may deteriorate severely as they approach topological lower bound.M 1;100;(1)M 10;511;(7)M 50;4656;(12)M 200;1552277;(28)M 500;3968690;(58)M 2000;14253219;(108)M 5000;029010;(176)M n =34794;0068776;(207)Figure 3:Example of a PSC representation.The image captions indicate the number of principal 012-simplices respectively and the number of connected components (in parenthesis).3PSC REPRESENTATION 3.1Triangulated modelsThe ﬁrst step towards generalizing PM’s is to let the PSC repre-sentation encode more general triangulated models,instead of just meshes.We denote a triangulated model as a tuple M =(K V D A ).The abstract simplicial complex K is not restricted to 2-manifolds,but may in fact be arbitrary.To represent K in memory,we encode the incidence graph of the simplices using the following linked structures (in C++notation):struct Simplex int dim;//0=vertex,1=edge,2=triangle,...int id;Simplex*children[MAXDIM+1];//[0..dim]List<Simplex*>parents;;To render the model,we draw only the principal simplices ofK ,denoted (K )(i.e.vertices not adjacent to edges,edges not adjacent to triangles,etc.).The discrete attributes D associate amaterial identiﬁer d s with each simplex s(K ).For the sake of simplicity,we avoid explicitly storing surface normals at “corners”(using a set S )as done in [13].Instead we let the material identiﬁer d s contain a smoothing group ﬁeld [28],and let a normal discontinuity (crease )form between any pair of adjacent triangles with different smoothing groups.Previous vertex uniﬁcation schemes [21,22]render principal simplices of dimension 0and 1(denoted 01(K ))as points and lines respectively with ﬁxed,device-dependent screen widths.To better approximate the model,we instead deﬁne a set A that associates an area a s A with each simplex s 01(K ).We think of a 0-simplex s 00(K )as approximating a sphere with area a s 0,and a 1-simplex s 1=j k 1(K )as approximating a cylinder (with axis (j k ))of area a s 1.To render a simplex s 01(K ),we determine the radius r model of the corresponding sphere or cylinder in modeling space,and project the length r model to obtain the radius r screen in screen pixels.Depending on r screen ,we render the simplex as a polygonal sphere or cylinder with radius r model ,a 2D point or line with thickness 2r screen ,or do not render it at all.This choice based on r screen can be adjusted to mitigate the overhead of introducing polygonal representations of spheres and cylinders.As an example,Figure 3shows an initial model M of 68,776triangles.One of its approximations M 500is a triangulated model with 3968690principal 012-simplices respectively.3.2Level-of-detail sequenceAs in progressive meshes,from a given triangulated model M =M n ,we deﬁne a sequence of approximations M i :M 1op 1M 2op 2M n1op n 1M nHere each model M i has exactly i vertices.The simpliﬁcation op-erator M ivunify iM i +1is the vertex uniﬁcation transformation,whichmerges two vertices (Section 3.3),and its inverse M igvspl iM i +1is the generalized vertex split transformation (Section 3.4).Thetuple (M 1gvspl 1gvspl n 1)forms a progressive simplicial complex (PSC)representation of M .To construct a PSC representation,we ﬁrst determine a sequence of vunify transformations simplifying M down to a single vertex,as described in Section 4.After reversing these transformations,we renumber the simplices in the order that they are created,so thateach gvspl i (a i)splits the vertex a i K i into two vertices a i i +1K i +1.As vertices may have different positions in the different models,we denote the position of j in M i as i j .To better approximate a surface model M at lower complexity levels,we initially associate with each (principal)2-simplex s an area a s equal to its triangle area in M .Then,as the model is simpliﬁed,wekeep constant the sum of areas a s associated with principal simplices within each manifold component.When2-simplices are eventually reduced to principal1-simplices and0-simplices,their associated areas will provide good estimates of the original component areas.3.3Vertex uniﬁcation transformationThe transformation vunify(a i b i midp i):M i M i+1takes an arbitrary pair of vertices a i b i K i+1(simplex a i b i need not be present in K i+1)and merges them into a single vertex a i K i. Model M i is created from M i+1by updating each member of the tuple(K V D A)as follows:K:References to b i in all simplices of K are replaced by refer-ences to a i.More precisely,each simplex s in star(b i)K i+1is replaced by simplex(s b i)a i,which we call the ancestor simplex of s.If this ancestor simplex already exists,s is deleted.V:Vertex b is deleted.For simplicity,the position of the re-maining(uniﬁed)vertex is set to either the midpoint or is left unchanged.That is,i a=(i+1a+i+1b)2if the boolean parameter midp i is true,or i a=i+1a otherwise.D:Materials are carried through as expected.So,if after the vertex uniﬁcation an ancestor simplex(s b i)a i K i is a new principal simplex,it receives its material from s K i+1if s is a principal simplex,or else from the single parent s a i K i+1 of s.A:To maintain the initial areas of manifold components,the areasa s of deleted principal simplices are redistributed to manifold-adjacent neighbors.More concretely,the area of each princi-pal d-simplex s deleted during the K update is distributed toa manifold-adjacent d-simplex not in star(a ib i).If no suchneighbor exists and the ancestor of s is a principal simplex,the area a s is distributed to that ancestor simplex.Otherwise,the manifold component(star(a i b i))of s is being squashed be-tween two other manifold components,and a s is discarded. 3.4Generalized vertex split transformation Constructing the PSC representation involves recording the infor-mation necessary to perform the inverse of each vunify i.This inverse is the generalized vertex split gvspl i,which splits a0-simplex a i to introduce an additional0-simplex b i.(As mentioned previously, renumbering of simplices implies b i i+1,so index b i need not be stored explicitly.)Each gvspl i record has the formgvspl i(a i C K i midp i()i C D i C A i)and constructs model M i+1from M i by updating the tuple (K V D A)as follows:K:As illustrated in Figure4,any simplex adjacent to a i in K i can be the vunify result of one of four conﬁgurations in K i+1.To construct K i+1,we therefore replace each ancestor simplex s star(a i)in K i by either(1)s,(2)(s a i)i+1,(3)s and(s a i)i+1,or(4)s,(s a i)i+1and s i+1.The choice is determined by a split code associated with s.Thesesplit codes are stored as a code string C Ki ,in which the simplicesstar(a i)are sortedﬁrst in order of increasing dimension,and then in order of increasing simplex id,as shown in Figure5. V:The new vertex is assigned position i+1i+1=i ai+()i.Theother vertex is given position i+1ai =i ai()i if the boolean pa-rameter midp i is true;otherwise its position remains unchanged.D:The string C Di is used to assign materials d s for each newprincipal simplex.Simplices in C Di ,as well as in C Aibelow,are sorted by simplex dimension and simplex id as in C Ki. A:During reconstruction,we are only interested in the areas a s fors01(K).The string C Ai tracks changes in these areas.Figure4:Effects of split codes on simplices of various dimensions.code string:41422312{}Figure5:Example of split code encoding.3.5PropertiesLevels of detail A graphics application can efﬁciently transitionbetween models M1M n at runtime by performing a sequence ofvunify or gvspl transformations.Our current research prototype wasnot designed for efﬁciency;it attains simpliﬁcation rates of about6000vunify/sec and reﬁnement rates of about5000gvspl/sec.Weexpect that a careful redesign using more efﬁcient data structureswould signiﬁcantly improve these rates.Geomorphs As in the PM representation,there exists a corre-spondence between the vertices of the models M1M n.Given acoarser model M c and aﬁner model M f,1c f n,each vertexj K f corresponds to a unique ancestor vertex f c(j)K cfound by recursively traversing the ancestor simplex relations:f c(j)=j j cf c(a j1)j cThis correspondence allows the creation of a smooth visual transi-tion(geomorph)M G()such that M G(1)equals M f and M G(0)looksidentical to M c.The geomorph is deﬁned as the modelM G()=(K f V G()D f A G())in which each vertex position is interpolated between its originalposition in V f and the position of its ancestor in V c:Gj()=()fj+(1)c f c(j)However,we must account for the special rendering of principalsimplices of dimension0and1(Section3.1).For each simplexs01(K f),we interpolate its area usinga G s()=()a f s+(1)a c swhere a c s=0if s01(K c).In addition,we render each simplexs01(K c)01(K f)using area a G s()=(1)a c s.The resultinggeomorph is visually smooth even as principal simplices are intro-duced,removed,or change dimension.The accompanying video demonstrates a sequence of such geomorphs.Progressive transmission As with PM’s,the PSC representa-tion can be progressively transmitted by ﬁrst sending M 1,followed by the gvspl records.Unlike the base mesh of the PM,M 1always consists of a single vertex,and can therefore be sent in a ﬁxed-size record.The rendering of lower-dimensional simplices as spheres and cylinders helps to quickly convey the overall shape of the model in the early stages of transmission.Model compression Although PSC gvspl are more general than PM vsplit transformations,they offer a surprisingly concise representation of M .Table 1lists the average number of bits re-quired to encode each ﬁeld of the gvspl records.Using arithmetic coding [30],the vertex id ﬁeld a i requires log 2i bits,and the boolean parameter midp i requires 0.6–0.9bits for our models.The ()i delta vector is quantized to 16bitsper coordinate (48bits per),and stored as a variable-length ﬁeld [7,13],requiring about 31bits on average.At ﬁrst glance,each split code in the code string C K i seems to have 4possible outcomes (except for the split code for 0-simplex a i which has only 2possible outcomes).However,there exist constraints between these split codes.For example,in Figure 5,the code 1for 1-simplex id 1implies that 2-simplex id 1also has code 1.This in turn implies that 1-simplex id 2cannot have code 2.Similarly,code 2for 1-simplex id 3implies a code 2for 2-simplex id 2,which in turn implies that 1-simplex id 4cannot have code 1.These constraints,illustrated in the “scoreboard”of Figure 6,can be summarized using the following two rules:(1)If a simplex has split code c12,all of its parents havesplit code c .(2)If a simplex has split code 3,none of its parents have splitcode 4.As we encode split codes in C K i left to right,we apply these two rules (and their contrapositives)transitively to constrain the possible outcomes for split codes yet to be ing arithmetic coding with uniform outcome probabilities,these constraints reduce the code string length in Figure 6from 15bits to 102bits.In our models,the constraints reduce the code string from 30bits to 14bits on average.The code string is further reduced using a non-uniform probability model.We create an array T [0dim ][015]of encoding tables,indexed by simplex dimension (0..dim)and by the set of possible (constrained)split codes (a 4-bit mask).For each simplex s ,we encode its split code c using the probability distribution found in T [s dim ][s codes mask ].For 2-dimensional models,only 10of the 48tables are non-trivial,and each table contains at most 4probabilities,so the total size of the probability model is small.These encoding tables reduce the code strings to approximately 8bits as shown in Table 1.By comparison,the PM representation requires approximately 5bits for the same information,but of course it disallows topological changes.To provide more intuition for the efﬁciency of the PSC repre-sentation,we note that capturing the connectivity of an average 2-manifold simplicial complex (n vertices,3n edges,and 2n trian-gles)requires ni =1(log 2i +8)n (log 2n +7)bits with PSC encoding,versus n (12log 2n +95)bits with a traditional one-way incidence graph representation.For improved compression,it would be best to use a hybrid PM +PSC representation,in which the more concise PM vertex split encoding is used when the local neighborhood is an orientableFigure 6:Constraints on the split codes for the simplices in the example of Figure 5.Table 1:Compression results and construction times.Object#verts Space required (bits/n )Trad.Con.n K V D Arepr.time a i C K i midp i (v )i C D i C Ai bits/n hrs.drumset 34,79412.28.20.928.1 4.10.453.9146.1 4.3destroyer 83,79913.38.30.723.1 2.10.347.8154.114.1chandelier 36,62712.47.60.828.6 3.40.853.6143.6 3.6schooner 119,73413.48.60.727.2 2.5 1.353.7148.722.2sandal 4,6289.28.00.733.4 1.50.052.8123.20.4castle 15,08211.0 1.20.630.70.0-43.5-0.5cessna 6,7959.67.60.632.2 2.50.152.6132.10.5harley 28,84711.97.90.930.5 1.40.453.0135.7 3.52-dimensional manifold (this occurs on average 93%of the time in our examples).To compress C D i ,we predict the material for each new principalsimplex sstar(a i )star(b i )K i +1by constructing an ordered set D s of materials found in star(a i )K i .To improve the coding model,the ﬁrst materials in D s are those of principal simplices in star(s )K i where s is the ancestor of s ;the remainingmaterials in star(a i )K i are appended to D s .The entry in C D i associated with s is the index of its material in D s ,encoded arithmetically.If the material of s is not present in D s ,it is speciﬁed explicitly as a global index in D .We encode C A i by specifying the area a s for each new principalsimplex s 01(star(a i )star(b i ))K i +1.To account for this redistribution of area,we identify the principal simplex from which s receives its area by specifying its index in 01(star(a i ))K i .The column labeled in Table 1sums the bits of each ﬁeld of the gvspl records.Multiplying by the number n of vertices in M gives the total number of bits for the PSC representation of the model (e.g.500KB for the destroyer).By way of compari-son,the next column shows the number of bits per vertex required in a traditional “IndexedFaceSet”representation,with quantization of 16bits per coordinate and arithmetic coding of face materials (3n 16+2n 3log 2n +materials).4PSC CONSTRUCTIONIn this section,we describe a scheme for iteratively choosing pairs of vertices to unify,in order to construct a PSC representation.Our algorithm,a generalization of [13],is time-intensive,seeking high quality approximations.It should be emphasized that many quality metrics are possible.For instance,the quadric error metric recently introduced by Garland and Heckbert [9]provides a different trade-off of execution speed and visual quality.As in [13,20],we ﬁrst compute a cost E for each candidate vunify transformation,and enter the candidates into a priority queueordered by ascending cost.Then,in each iteration i =n 11,we perform the vunify at the front of the queue and update the costs of affected candidates.4.1Forming set of candidate vertex pairs In principle,we could enter all possible pairs of vertices from M into the priority queue,but this would be prohibitively expensive since simpliﬁcation would then require at least O(n2log n)time.Instead, we would like to consider only a smaller set of candidate vertex pairs.Naturally,should include the1-simplices of K.Additional pairs should also be included in to allow distinct connected com-ponents of M to merge and to facilitate topological changes.We considered several schemes for forming these additional pairs,in-cluding binning,octrees,and k-closest neighbor graphs,but opted for the Delaunay triangulation because of its adaptability on models containing components at different scales.We compute the Delaunay triangulation of the vertices of M, represented as a3-dimensional simplicial complex K DT.We deﬁne the initial set to contain both the1-simplices of K and the subset of1-simplices of K DT that connect vertices in different connected components of K.During the simpliﬁcation process,we apply each vertex uniﬁcation performed on M to as well in order to keep consistent the set of candidate pairs.For models in3,star(a i)has constant size in the average case,and the overall simpliﬁcation algorithm requires O(n log n) time.(In the worst case,it could require O(n2log n)time.)4.2Selecting vertex uniﬁcations fromFor each candidate vertex pair(a b),the associated vunify(a b):M i M i+1is assigned the costE=E dist+E disc+E area+E foldAs in[13],theﬁrst term is E dist=E dist(M i)E dist(M i+1),where E dist(M)measures the geometric accuracy of the approximate model M.Conceptually,E dist(M)approximates the continuous integralMd2(M)where d(M)is the Euclidean distance of the point to the closest point on M.We discretize this integral by deﬁning E dist(M)as the sum of squared distances to M from a dense set of points X sampled from the original model M.We sample X from the set of principal simplices in K—a strategy that generalizes to arbitrary triangulated models.In[13],E disc(M)measures the geometric accuracy of disconti-nuity curves formed by a set of sharp edges in the mesh.For the PSC representation,we generalize the concept of sharp edges to that of sharp simplices in K—a simplex is sharp either if it is a boundary simplex or if two of its parents are principal simplices with different material identiﬁers.The energy E disc is deﬁned as the sum of squared distances from a set X disc of points sampled from sharp simplices to the discontinuity components from which they were sampled.Minimization of E disc therefore preserves the geom-etry of material boundaries,normal discontinuities(creases),and triangulation boundaries(including boundary curves of a surface and endpoints of a curve).We have found it useful to introduce a term E area that penalizes surface stretching(a more sophisticated version of the regularizing E spring term of[13]).Let A i+1N be the sum of triangle areas in the neighborhood star(a i)star(b i)K i+1,and A i N the sum of triangle areas in star(a i)K i.The mean squared displacement over the neighborhood N due to the change in area can be approx-imated as disp2=12(A i+1NA iN)2.We let E area=X N disp2,where X N is the number of points X projecting in the neighborhood. To prevent model self-intersections,the last term E fold penalizes surface folding.We compute the rotation of each oriented triangle in the neighborhood due to the vertex uniﬁcation(as in[10,20]).If any rotation exceeds a threshold angle value,we set E fold to a large constant.Unlike[13],we do not optimize over the vertex position i a, but simply evaluate E for i a i+1a i+1b(i+1a+i+1b)2and choose the best one.This speeds up the optimization,improves model compression,and allows us to introduce non-quadratic energy terms like E area.5RESULTSTable1gives quantitative results for the examples in theﬁgures and in the video.Simpliﬁcation times for our prototype are measured on an SGI Indigo2Extreme(150MHz R4400).Although these times may appear prohibitive,PSC construction is an off-line task that only needs to be performed once per model.Figure9highlights some of the beneﬁts of the PSC representa-tion.The pearls in the chandelier model are initially disconnected tetrahedra;these tetrahedra merge and collapse into1-d curves in lower-complexity approximations.Similarly,the numerous polyg-onal ropes in the schooner model are simpliﬁed into curves which can be rendered as line segments.The straps of the sandal model initially have some thickness;the top and bottom sides of these straps merge in the simpliﬁcation.Also note the disappearance of the holes on the sandal straps.The castle example demonstrates that the original model need not be a mesh;here M is a1-dimensional non-manifold obtained by extracting edges from an image.6RELATED WORKThere are numerous schemes for representing and simplifying tri-angulations in computer graphics.A common special case is that of subdivided2-manifolds(meshes).Garland and Heckbert[12] provide a recent survey of mesh simpliﬁcation techniques.Several methods simplify a given model through a sequence of edge col-lapse transformations[10,13,14,20].With the exception of[20], these methods constrain edge collapses to preserve the topological type of the model(e.g.disallow the collapse of a tetrahedron into a triangle).Our work is closely related to several schemes that generalize the notion of edge collapse to that of vertex uniﬁcation,whereby separate connected components of the model are allowed to merge and triangles may be collapsed into lower dimensional simplices. Rossignac and Borrel[21]overlay a uniform cubical lattice on the object,and merge together vertices that lie in the same cubes. Schauﬂer and St¨u rzlinger[22]develop a similar scheme in which vertices are merged using a hierarchical clustering algorithm.Lue-bke[18]introduces a scheme for locally adapting the complexity of a scene at runtime using a clustering octree.In these schemes, the approximating models correspond to simplicial complexes that would result from a set of vunify transformations(Section3.3).Our approach differs in that we order the vunify in a carefully optimized sequence.More importantly,we deﬁne not only a simpliﬁcation process,but also a new representation for the model using an en-coding of gvspl=vunify1transformations.Recent,independent work by Schmalstieg and Schauﬂer[23]de-velops a similar strategy of encoding a model using a sequence of vertex split transformations.Their scheme differs in that it tracks only triangles,and therefore requires regular,2-dimensional trian-gulations.Hence,it does not allow lower-dimensional simplices in the model approximations,and does not generalize to higher dimensions.Some simpliﬁcation schemes make use of an intermediate vol-umetric representation to allow topological changes to the model. He et al.[11]convert a mesh into a binary inside/outside function discretized on a three-dimensional grid,low-passﬁlter this function,。

Efficient RANSAC for Point-Cloud Shape Detection

Volume0(1981),Number0pp.1–12Efﬁcient RANSAC for Point-Cloud Shape DetectionRuwen Schnabel Roland Wahl Reinhard Klein†Universität Bonn,Computer Graphics GroupAbstractIn this work we present an automatic algorithm to detect basic shapes in unorganized point clouds.The algorithm decomposes the point cloud into a concise,hybrid structure of inherent shapes and a set of remaining points.Each detected shape serves as a proxy for a set of corresponding points.Our method is based on random sampling and detects planes,spheres,cylinders,cones and tori.For models with surfaces composed of these basic shapes only,e.g.CAD models,we automatically obtain a representation solely consisting of shape proxies.We demonstratethat the algorithm is robust even in the presence of many outliers and a high degree of noise.The proposed method scales well with respect to the size of the input point cloud and the number and size of the shapes within the data.Even point sets with several millions of samples are robustly decomposed within less than a minute.Moreover the algorithm is conceptually simple and easy to implement.Application areas include measurement of physical parameters,scan registration,surface compression,hybrid rendering,shape classiﬁcation,meshing, simpliﬁcation,approximation and reverse engineering.Categories and Subject Descriptors(according to ACM CCS):I.4.8[Image Processing and Computer Vision]:Scene AnalysisShape;Surface Fitting;I.3.5[Computer Graphics]:Computational Geometry and Object ModelingCurve, surface,solid,and object representations1.IntroductionDue to the increasing size and complexity of geometric data sets there is an ever-growing demand for concise and mean-ingful abstractions of this data.Especially when dealing with digitized geometry,e.g.acquired with a laser scanner,no handles for modiﬁcation of the data are available to the user other than the digitized points themselves.However,in or-der to be able to make use of the data effectively,the raw digitized data has to be enriched with abstractions and pos-sibly semantic information,providing the user with higher-level interaction possibilities.Only such handles can pro-vide the interaction required for involved editing processes, such as deleting,moving or resizing certain parts and hence can make the data more readily usable for modeling pur-poses.Of course,traditional reverse engineering approaches can provide some of the abstractions that we seek,but usu-ally reverse engineering focuses onﬁnding a reconstruction of the underlying geometry and typically involves quite te-dious user interaction.This is not justiﬁed in a setting where †e-mail:{schnabel,wahl,rk}@cs.uni-bonn.de a complete and detailed reconstruction is not required at all, or shall take place only after some basic editing operations have been applied to the data.On the other hand,detecting instances of a set of primitive geometric shapes in the point sampled data is a means to quickly derive higher levels of ab-straction.For example in Fig.1patches of primitive shapes provide a coarse approximation of the geometry that could be used to compress the point-cloud very effectively. Another problem arising when dealing with digitized geom-etry is the often huge size of the datasets.Therefore the efﬁciency of algorithms inferring abstractions of the data is of utmost importance,especially in interactive settings. Thus,in this paper we focus especially onﬁnding an efﬁ-cient algorithm for point-cloud shape detection,in order to be able to deal even with large point-clouds.Our work is a high performance RANSAC[FB81]algorithm that is capa-ble to extract a variety of different types of primitive shapes, while retaining such favorable properties of the RANSAC paradigm as robustness,generality and simplicity.At the heart of our algorithm are a novel,hierarchically structured sampling strategy for candidate shape generation as well as a novel,lazy cost function evaluation scheme,which signif-c The Eurographics Association and Blackwell Publishing2007.Published by Blackwell Publishing,9600Garsington Road,Oxford OX42DQ,UK and350Main Street,Malden, MA02148,USA.(a)Original(b)ApproximationFigure1:The372detected shapes in the choir screen deﬁne a coarse approximation of the surface.icantly reduces overall computational cost.Our method de-tects planes,spheres,cylinders,cones and tori,but additional primitives are possible.The goal of our algorithm is to reli-ably extract these shapes from the data,even under adverse conditions such as heavy noise.As has been indicated above,our method is especially well suited in situations where geometric data is automatically acquired and users refrain from applying surface reconstruc-tion methods,either due to the data’s low quality or due to processing time constraints.Such constraints are typical for areas where high level model interaction is required,as is the case when measuring physical parameters or in interactive, semi-automatic segmentation and postprocessing.Further applications are,for instance,registering many scans of an object,where detecting corresponding primitive shapes in multiple scans can provide good initial matches.High compression rates for point clouds can be achieved if prim-itive shapes are used to represent a large number of points with a small set of parameters.Other areas that can beneﬁt from primitive shape information include hybrid rendering and shape classiﬁcation.Additionally,a fast shape extraction method as ours can serve as building block in applications such as meshing,simpliﬁcation,approximation and reverse engineering and bears the potential of signiﬁcant speed up.2.Previous workThe detection of primitive shapes is a common problem en-countered in many areas of geometry related computer sci-ence.Over the years a vast number of methods have been proposed which cannot all be discussed here in depth.In-stead,here we give a short overview of some of the most important algorithms developed in the differentﬁelds.We treat the previous work on RANSAC algorithms separately in section2.1as it is of special relevance to our work. Vision In computer vision,the two most widely known methodologies for shape extraction are the RANSAC paradigm[FB81]and the Hough transform[Hou62].Both have been proven to successfully detect shapes in2D as well as3D.RANSAC and the Hough transform are reliable even in the presence of a high proportion of outliers,but lack of efﬁciency or high memory consumption remains their ma-jor drawback[IK88].For both schemes,many acceleration techniques have been proposed,but no one on its own,or combinations thereof,have been shown to be able to provide an algorithm as efﬁcient as ours for the3D primitive shape extraction problem.The Hough transform maps,for a given type of parameter-ized primitive,every point in the data to a manifold in the pa-rameter space.The manifold describes all possible variants of the primitive that contain the original point,i.e.in practice each point casts votes for many cells in a discretized param-eter space.Shapes are extracted by selecting those parame-ter vectors that have received a signiﬁcant amount of votes. If the parameter space is discretized naively using a simple grid,the memory requirements quickly become prohibitive even for primitives with a moderate number of parameters, such as,for instance,cones.Although several methods have been suggested to alleviate this problem[IK87][XO93]its major application area remains the2D domain where the number of parameters typically is quite small.A notable ex-ception is[VGSR04]where the Hough transform is used to detect planes in3D datasets,as3D planes still have only a small number of parameters.They also propose a two-step procedure for the Hough based detection of cylinders that uses estimated normals in the data points.In the vision community many approaches have been pro-posed for segmentation of range images with primitive shapes.When working on range images these algorithms usually efﬁciently exploit the implicitly given connectiv-ity information of the image grid in some kind of region growing or region merging step[FEF97][GBS03].This is a fundamental difference to our case,where we are given only an unstructured cloud of points that lacks any explicit connectivity information.In[LGB95]and[LJS97]shapes are found by concurrently growing different seed primi-tives from which a suitable subset is selected according to an MDL criterion(coined the recover-and-select paradigm). [GBS03]detect shapes using a genetic algorithm to optimize a robust MSACﬁtness function(see also sec.2.1).[MLM01]c The Eurographics Association and Blackwell Publishing2007.introduce involved non-linearﬁtting functions for primitive shapes that are able to handle geometric degeneracy in the context of recover-and-select segmentation.Another robust method frequently employed in the vision community is the tensor voting framework[MLT00]which has been applied to successfully reconstruct surface geome-try from extremely cluttered scenes.While tensor voting can compete with RANSAC in terms of robustness,it is,how-ever,inherently model-free and therefore cannot be applied to the detection of predeﬁned types of primitive shapes. Reverse engineering In reverse engineering,surface re-covery techniques are usually based on either a separate seg-mentation step or on a variety of region growing algorithms [VMC97][SB95][BGV∗02].Most methods call for some kind of connectivity information and are not well equipped to deal with a large amount of outliers[VMC97].Also these approaches try toﬁnd a shape proxy for every part of the pro-cessed surface with the intent of loading the reconstructed geometry information into a CAD application.[BMV01]de-scribe a system which reconstructs a boundary representa-tion that can be imported into a CAD application from an unorganized point-cloud.However,their method is based on ﬁnding a triangulation for the point-set,whereas the method presented in this work is able to operate directly on the input points.This is advantageous as computing a suitable tessela-tion may be extremely costly and becomes very intricate or even ill-deﬁned when there is heavy noise in the data.We do not,however,intend to present a method implementing all stages of a typical reverse engineering process.Graphics In computer graphics,[CSAD04]have recently proposed a general variational framework for approximation of surfaces by planes,which was extended to a set of more elaborate shape proxies by[WK05].Their aim is not only to extract certain shapes in the data,but toﬁnd a globally optimal representation of the object by a given number of primitives.However,these methods require connectivity in-formation and are,due to their exclusive use of least squares ﬁtting,susceptible to errors induced by outliers.Also,the optimization procedure is computationally expensive,which makes the method less suitable for large data sets.The out-put of our algorithm,however,could be used to initialize the set of shape proxies used by these methods,potentially accelerating the convergence of the optimization procedure. While the Hough transform and the RANSAC paradigm have been mainly used in computer vision some applica-tions have also been proposed in the computer graphics com-munity.[DDSD03]employ the Hough transform to identify planes for billboard clouds for triangle data.They propose an extension of the standard Hough transform to include a compactness criterion,but due to the high computational de-mand of the Hough transform,the method exhibits poor run-time performance on large or complex geometry.[WGK05] proposed a RANSAC-based plane detection method for hy-brid rendering of point clouds.To facilitate an efﬁcient plane detection,planes are detected only in the cells of a hier-archical space decomposition and therefore what is essen-tially one plane on the surface is approximated by several planar patches.While this is acceptable for their hybrid ren-dering technique,our methodﬁnds maximal surface patches in order to yield a more concise representation of the ob-ject.Moreover,higher order primitives are not considered in their approach.[GG04]detect so-called slippable shapes which is a superset of the shapes recognized by our method. They use the eigenvalues of a symmetric matrix derived from the points and their normals to determine the slippability of a point-set.Their detection is a bottom-up approach that merges small initial slippable surfaces to obtain a global de-composition of the model.However the computation of the eigenvalues is costly for large models,the method is sen-sitive to noise and it is hard to determine the correct size of the initial surface patches.A related approach is taken by [HOP∗05].They also use the eigenvalues of a matrix derived from line element geometry to classify surfaces.A RANSAC based segmentation algorithm is employed to detect several shapes in a point-cloud.The method is aimed mainly at mod-els containing small numbers of points and shapes as no opti-mizations or extensions to the general RANSAC framework are adopted.2.1.RANSACThe RANSAC paradigm extracts shapes by randomly draw-ing minimal sets from the point data and constructing cor-responding shape primitives.A minimal set is the smallest number of points required to uniquely deﬁne a given type of geometric primitive.The resulting candidate shapes are tested against all points in the data to determine how many of the points are well approximated by the primitive(called the score of the shape).After a given number of trials,the shape which approximates the most points is extracted and the algorithm continues on the remaining data.RANSAC ex-hibits the following,desirable properties:•It is conceptually simple,which makes it easily extensible and straightforward to implement•It is very general,allowing its application in a wide range of settings•It can robustly deal with data containing more than50% of outliers[RL93]Its major deﬁciency is the considerable computational de-mand if no further optimizations are applied.[BF81]apply RANSAC to extract cylinders from range data,[CG01]use RANSAC and the gaussian image toﬁnd cylinders in3D point clouds.Both methods,though,do not consider a larger number of different classes of shape prim-itives.[RL93]describe an algorithm that uses RANSAC to detect a set of different types of simple shapes.However, their method was adjusted to work in the image domain orc The Eurographics Association and Blackwell Publishing2007.on range images,and they did not provide the optimization necessary for processing large unstructured3D data sets.A vast number of extensions to the general RANSAC scheme have been proposed.Among the more recent ad-vances,methods such as MLESAC[TZ00]or MSAC[TZ98] improve the robustness of RANSAC with a modiﬁed score function,but do not provide any enhancement in the perfor-mance of the algorithm,which is the main focus of our work. Nonetheless the integration of a MLESAC scoring function is among the directions of our future work.[Nis05]pro-poses an acceleration technique for the case that the num-ber of candidates isﬁxed in advance.As it is a fundamen-tal property of our setup that an unknown large number of possibly very small shapes has to be detected in huge point-clouds,the amount of necessary candidates cannot,however, be speciﬁed in advance.3.OverviewGiven a point-cloud P={p1,...,p N}with associated nor-mals{n1,...,n N}the output of our algorithm is a set of primitive shapesΨ={ψ1,...,ψn}with corresponding dis-joint sets of points Pψ1⊂P,...,Pψn⊂P and a set of re-maining points R=P\{Pψ1,...,Pψn}.Similar to[RL93]and[DDSD03],we frame the shape extraction problem as an optimization problem deﬁned by a score function.The overall structure of our method is outlined in pseudo-code in algorithm1.In each iteration of the algorithm,the prim-itive with maximal score is searched using the RANSAC paradigm.New shape candidates are generated by randomly sampling minimal subsets of P using our novel sampling strategy(see sec.4.3).Candidates of all considered shape types are generated for every minimal set and all candidates are collected in the set C.Thus no special ordering has to be imposed on the detection of different types of shapes.After new candidates have been generated the one with the high-est score m is computed employing the efﬁcient lazy score evaluation scheme presented in sec.4.5.The best candidate is only accepted if,given the size|m|(in number of points) of the candidate and the number of drawn candidates|C|, the probability P(|m|,|C|)that no better candidate was over-looked during sampling is high enough(see sec.4.2.1).We provide an analysis of our sampling strategy to derive a suit-able probability computation.If a candidate is accepted,the corresponding points P m are removed from P and the can-didates C m generated with points in P m are deleted from C. The algorithm terminates as soon as P(τ,|C|)for a user de-ﬁned minimal shape sizeτis large enough.In our implementation we use a standard score function that counts the number of compatible points for a shape candi-date[RL93][GBS03].The function has two free parame-ters:εspeciﬁes the maximum distance of a compatible point whileαrestricts the deviation of a points’normal from that of the shape.We also ensure that only points forming a con-nected component on the surface are considered(see sec.4.4).Algorithm1Extract shapes in the point cloud PΨ←/0{extracted shapes}C←/0{shape candidates}repeatC←C∪newCandidates(){see sec.4.1and4.3}m←bestCandidate(C){see sec.4.4}if P(|m|,|C|)>p t thenP←P\P m{remove points}Ψ←Ψ∪mC←C\C m{remove invalid candidates}end ifuntil P(τ,|C|)>p treturnΨ4.Our method4.1.Shape estimationAs mentioned above,the shapes we consider in this work are planes,spheres,cylinders,cones and tori which have be-tween three and seven parameters.Every3D-point p i sam-pleﬁxes only one parameter of the shape.In order to reduce the number of required points we compute an approximate surface normal n i for each point[HDD∗92],so that the ori-entation gives us two more parameters per sample.That way it is possible to estimate each of the considered basic shapes from only one or two point samples.However,always using one additional sample is advantageous,because the surplus parameters can be used to immediately verify a candidate and thus eliminate the need of evaluating many relatively low scored shapes[MC02].Plane For a plane,{p1,p2,p3}constitutes a minimal set when not taking into account the normals in the points.To conﬁrm the plausibility of the generated plane,the deviation of the plane’s normal from n1,n2,n3is determined and the candidate plane is accepted only if all deviations are less than the predeﬁned angleα.Sphere A sphere is fully deﬁned by two points with corre-sponding normal vectors.We use the midpoint of the short-est line segment between the two lines given by the points p1and p2and their normals n1and n2to deﬁne the center of the sphere c.We take r= p1−c + p2−c2as the sphere ra-dius.The sphere is accepted as a shape candidate only if all three points are within a distance ofεof the sphere and their normals do not deviate by more thanαdegrees.Cylinder To generate a cylinder from two points with nor-mals weﬁrst establish the direction of the axis with a= n1×n2.Then we project the two parametric lines p1+tn1 and p2+tn2along the axis onto the a·x=0plane and take their intersection as the center c.We set the radius to the dis-tance between c and p1in that plane.Again the cylinder isc The Eurographics Association and Blackwell Publishing2007.veriﬁed by applying the thresholds εand αto distance and normal deviation of the samples.Cone Although the cone,too,is fully deﬁned by two points with corresponding normals,for simplicity we use all three points and normals in its generation.To derive the po-sition of the apex c ,we intersect the three planes deﬁned by the point and normal pairs.Then the normal of the plane de-ﬁned by the three points {c +p 1−c p 1−c ,...,c +p 3−c p 3−c }givesthe direction of the axis a .Now the opening angle ωis givenas ω=∑i arccos ((p i −c )·a )3.Afterwards,similar to above,the cone is veriﬁed before becoming a candidate shape.Torus Just as in the case of the cone we use one more point than theoretically necessary to ease the computations required for estimation,i.e.four point and normal pairs.The rotational axis of the torus is found as one of the up to two lines intersecting the four point-normal lines p i +λn i [MLM01].To choose between the two possible axes,a full torus is estimated for both choices and the one which causes the smaller error in respect to the four points is selected.To ﬁnd the minor radius,the points are collected in a plane that is rotated around the axis.Then a circle is computed using three points in this plane.The major radius is given as the distance of the circle center to the plexityThe complexity of RANSAC is dominated by two major fac-tors:The number of minimal sets that are drawn and the cost of evaluating the score for every candidate shape.As we de-sire to extract the shape that achieves the highest possible score,the number of candidates that have to be considered is governed by the probability that the best possible shape is indeed detected,i.e.that a minimal set is drawn that deﬁnes this shape.4.2.1.ProbabilitiesConsider a point cloud P of size N and a shape ψtherein consisting of n points.Let k denote the size of a minimal set required to deﬁne a shape candidate.If we assume that any k points of the shape will lead to an appropriate candidate shape then the probability of detecting ψin a single pass is:P (n )= n k N k ≈ n N k(1)The probability of a successful detection P (n ,s )after s can-didates have been drawn equals the complementary of s con-secutive failures:P (n ,s )=1−(1−P (n ))s(2)Solving for s tells us the number of candidates T required to detect shapes of size n with a probability P (n ,T )≥p t :T ≥ln (1−p t )ln (1−P (n ))(3)Figure 2:A small cylinder that has been detected by ourmethod.The shape consists of 1066points and was detected among 341,587points.That corresponds to a relative size of 1/3000.For small P (n )the logarithm in the denominator can be approximated by its Taylor series ln (1−P (n ))=−P (n )+O (P (n )2)so that:T ≈−ln (1−p t )P (n )(4)Given the cost C of evaluating the cost function,the asymp-totic complexity of the RANSAC approach is O (TC )=O (1P (n )C ).4.3.Sampling strategyAs can be seen from the last formula,the runtime complexity is directly linked to the success rate of ﬁnding good sample sets.Therefore we will now discuss in detail how sampling is performed.4.3.1.Localized samplingSince shapes are local phenomena,the a priori probability that two points belong to the same shape is higher the smaller the distance between the points.In our sampling strategy we want to exploit this fact to increase the probability of draw-ing minimal sets that belong to the same shape.[MTN ∗02]have shown that non-uniform sampling based on locality leads to a signiﬁcantly increased probability of selecting a set of inliers.From a ball of given radius around an ini-tially unrestrainedly drawn sample the remaining samples are picked to obtain a complete minimal set.This requires to ﬁx a radius in advance,which they derive from a known (or assumed)outlier density and distribution.In our setup however,outlier density and distribution vary strongly for different models and even within in a single model,which renders a ﬁxed radius inadequate.Also,in our case,using minimal sets with small diameter introduces unnecessary stability issues in the shape estimation procedure for shapes that could have been estimated from samples spread farther apart.Therefore we propose a novel sampling strategy that is able to adapt the diameter of the minimal sets to both,outlier density and shape size.cThe Eurographics Association and Blackwell Publishing 2007.We use an octree to establish spatial proximity between sam-ples very efﬁciently.When choosing points for a new candi-date,we draw theﬁrst sample p1without restrictions among all points.Then a cell C is randomly chosen from any level of the octree such that p1is contained in C.The k−1other samples are then drawn only from within cell C.The effect of this sampling strategy can be expressed in a new probability P local(n)forﬁnding a shapeψof size n:P local(n)=P(p1∈ψ)P(p2...p k∈ψ|p2...p k∈C)(5) Theﬁrst factor evaluates to n/N.The second factor obvi-ously depends on the choice of C.C is well chosen if it con-tains mostly points belonging toψ.The existence of such a cell is backed by the observation that for most points on a shape,except on edges and corners,there exists a neighbor-hood such that all of the points therein belong to that shape. Although in general it is not guaranteed that this neighbor-hood is captured in the cells of the octree,in the case of real-life data,shapes have to be sampled with an adequate density for reliable representation and,as a consequence,for all but very few points such a neighborhood will be at least as large as the smallest cells of the octree.For the sake of analysis,we assume that there exists a C for every p i∈ψsuch thatψwill be supported by half of the points in C, which accounts for up to50%local noise and outliers.We conservatively estimate the probability ofﬁnding a good C by1d where d is the depth of the octree(in practice a path of cells starting at the highest good cell to a good leaf will be good as well).The conditional probability for p2,p3∈ψinthe case of a good cell is then described by (|C|/2k−1)(|C|k−1)≈(12)k−1.And substituting yields:P local(n)=nNd2k−1(6)As large shapes can be estimated from large cells(and with high probability this will happen),the stability of the shape estimation is not affected by the sampling strategy.The impact of this sampling strategy is best illustrated with an example.The cylinder depicted in Figure2consists of 1066points.At the time that it belongs to one of the largest shapes in the point-cloud,341,547points of the original2 million still remain.Thus,it then comprises only three thou-sandth of the point-cloud.If an ordinary uniform sampling strategy were to be applied,151,522,829candidates would have to be drawn to achieve a detection probability of99%. With our strategy only64,929candidates have to be gen-erated for the same probability.That is an improvement by three orders of magnitude,i.e.in this case that is the differ-ence between hours and seconds.4.3.1.1.Level weighting Choosing C from a proper level is an important aspect of our sampling scheme.Therefore we can further improve the sampling efﬁciency by choosing C from a level according to a non-uniform distribution that re-ﬂects the likelihood of the respective level to contain a good cell.To this end,the probability P l of choosing C from level l isﬁrst initialized with1d.Then for every level l,we keep track of the sumσl of the scores achieved by the candidates generated from a cell on level l.After a given number of candidates has been tested,a new distribution for the levels is computed.The new probabilityˆP l of the level l is given asˆPl=xσlwP l+(1−x)1d,(7)where w=∑d i=1σPi.We set x=.9to ensure that at all times at least10%of the samples are spread uniformly over the levels to be able to detect when new levels start to become of greater importance as more and more points are removed from P.4.3.2.Number of candidatesIn section4.2we gave a formula for the number of candi-dates necessary to detect a shape of size n with a given prob-ability.However,in our case,the size n of the largest shape is not known in advance.Moreover,if the largest candidate has been generated early in the process we should be able to de-tect this lucky case and extract the shape well before achiev-ing a precomputed number of candidates while on the other hand we should use additional candidates if it is still unsure that indeed the best candidate has been detected.Therefore, instead ofﬁxing the number of candidates,we repeatedly an-alyze small numbers t of additional candidates and consider the best oneψm generated so far each time.As we want to achieve a low probability that a shape is extracted which is not the real maximum,we observe the probability P(|ψm|,s) with which we would have found another shape of the same size asψm.Once this probability is higher than a threshold p t(we use99%)we conclude that there is a low chance that we have overlooked a better candidate and extractψm.The algorithm terminates as soon as P(τ,s)>p t.4.4.ScoreThe score functionσP is responsible for measuring the qual-ity of a given shape candidate.We use the following aspects in our scoring function:•To measure the support of a candidate,we use the number of points that fall within anε-band around the shape.•To ensure that the points inside the band roughly follow the curvature pattern of the given primitive,we only count those points inside the band whose normals do not deviate from the normal of the shape more than a given angleα.•Additionally we incorporate a connectivity measure: Among the points that fulﬁll the previous two conditions, only those are considered that constitute the largest con-nected component on the shape.c The Eurographics Association and Blackwell Publishing2007.。

spatio-temporall...

Spatio-Temporal LSTM with Trust Gates for3D Human Action Recognition817 respectively,and utilized a SVM classiﬁer to classify the actions.A skeleton-based dictionary learning utilizing group sparsity and geometry constraint was also proposed by[8].An angular skeletal representation over the tree-structured set of joints was introduced in[9],which calculated the similarity of these fea-tures over temporal dimension to build the global representation of the action samples and fed them to SVM forﬁnal classiﬁcation.Recurrent neural networks(RNNs)which are a variant of neural nets for handling sequential data with variable length,have been successfully applied to language modeling[10–12],image captioning[13,14],video analysis[15–24], human re-identiﬁcation[25,26],and RGB-based action recognition[27–29].They also have achieved promising performance in3D action recognition[30–32].Existing RNN-based3D action recognition methods mainly model the long-term contextual information in the temporal domain to represent motion-based dynamics.However,there is also strong dependency between joints in the spatial domain.And the spatial conﬁguration of joints in video frames can be highly discriminative for3D action recognition task.In this paper,we propose a spatio-temporal long short-term memory(ST-LSTM)network which extends the traditional LSTM-based learning to two con-current domains(temporal and spatial domains).Each joint receives contextual information from neighboring joints and also from previous frames to encode the spatio-temporal context.Human body joints are not naturally arranged in a chain,therefore feeding a simple chain of joints to a sequence learner can-not perform well.Instead,a tree-like graph can better represent the adjacency properties between the joints in the skeletal data.Hence,we also propose a tree structure based skeleton traversal method to explore the kinematic relationship between the joints for better spatial dependency modeling.In addition,since the acquisition of depth sensors is not always accurate,we further improve the design of the ST-LSTM by adding a new gating function, so called“trust gate”,to analyze the reliability of the input data at each spatio-temporal step and give better insight to the network about when to update, forget,or remember the contents of the internal memory cell as the representa-tion of long-term context information.The contributions of this paper are:(1)spatio-temporal design of LSTM networks for3D action recognition,(2)a skeleton-based tree traversal technique to feed the structure of the skeleton data into a sequential LSTM,(3)improving the design of the ST-LSTM by adding the trust gate,and(4)achieving state-of-the-art performance on all the evaluated datasets.2Related WorkHuman action recognition using3D skeleton information is explored in diﬀerent aspects during recent years[33–50].In this section,we limit our review to more recent RNN-based and LSTM-based approaches.HBRNN[30]applied bidirectional RNNs in a novel hierarchical fashion.They divided the entire skeleton toﬁve major groups of joints and each group was fedSpatio-Temporal LSTM with Trust Gates for3D Human Action RecognitionJun Liu1,Amir Shahroudy1,Dong Xu2,and Gang Wang1(B)1School of Electrical and Electronic Engineering,Nanyang Technological University,Singapore,Singapore{jliu029,amir3,wanggang}@.sg2School of Electrical and Information Engineering,University of Sydney,Sydney,Australia******************.auAbstract.3D action recognition–analysis of human actions based on3D skeleton data–becomes popular recently due to its succinctness,robustness,and view-invariant representation.Recent attempts on thisproblem suggested to develop RNN-based learning methods to model thecontextual dependency in the temporal domain.In this paper,we extendthis idea to spatio-temporal domains to analyze the hidden sources ofaction-related information within the input data over both domains con-currently.Inspired by the graphical structure of the human skeleton,wefurther propose a more powerful tree-structure based traversal method.To handle the noise and occlusion in3D skeleton data,we introduce newgating mechanism within LSTM to learn the reliability of the sequentialinput data and accordingly adjust its eﬀect on updating the long-termcontext information stored in the memory cell.Our method achievesstate-of-the-art performance on4challenging benchmark datasets for3D human action analysis.Keywords:3D action recognition·Recurrent neural networks·Longshort-term memory·Trust gate·Spatio-temporal analysis1IntroductionIn recent years,action recognition based on the locations of major joints of the body in3D space has attracted a lot of attention.Diﬀerent feature extraction and classiﬁer learning approaches are studied for3D action recognition[1–3].For example,Yang and Tian[4]represented the static postures and the dynamics of the motion patterns via eigenjoints and utilized a Na¨ıve-Bayes-Nearest-Neighbor classiﬁer learning.A HMM was applied by[5]for modeling the temporal dynam-ics of the actions over a histogram-based representation of3D joint locations. Evangelidis et al.[6]learned a GMM over the Fisher kernel representation of a succinct skeletal feature,called skeletal quads.Vemulapalli et al.[7]represented the skeleton conﬁgurations and actions as points and curves in a Lie group c Springer International Publishing AG2016B.Leibe et al.(Eds.):ECCV2016,Part III,LNCS9907,pp.816–833,2016.DOI:10.1007/978-3-319-46487-950。

Stata MI命令详解说明书

Multiple imputationEstimate model parameters from each imputation, and combine the results in one easy step using mi estimate .Choose from many supported estimation commands, and simply preﬁx them with mi estimate . Select how many imputations to use during estimation, request a detailed MI summary, and more.Missing data occur frequently in practice. MI is one of the most flexible ways of handling missing data. Its three stages are multiply imputing missing values, estimating model parameters from each imputed dataset, and combining multiple estimation results in one ﬁnal inference. In Stata, you can use the mi command to perform these three stages in two simple steps.Use predictive mean matching, linear, logistic, Poisson, and other regressions to impute variables of di erent types. Use multiple imputation using ch ained equations (MICE), multivariate normal imputation (MVN), and monotoneImpute missing values using mi impute .• Support for all three stages of MI: impute missing values,estimate model parameters, and combine estimation results • Imputation– Nine univariate methods– Multivariate methods: MICE (FCS) and MVN – Monotone and arbitrary missing-value pa erns – Add your own methods • Estimation : estimate and combine in one easy step • Inference : linear and nonlinear combinations, hypothesis testing, predictions• MI data : e cient storage, veriﬁcation, import, full data management• Control Panel to guide you through your MI analysis imputation to impute multiple variables. Add your own imputation methods. With MICE, build flexible imputation models—use any of the nine univariate methods, customize prediction equations, include functions of imputed variables, perform conditional imputation, and more.Already have imputed data? Simply import them to Stata for further MI analysis. For example, to import imputed datasets imp1, imp2, ..., imp5 from NHANES, use. mi import nhanes1 mymidata, using(imp{1-5}) id(obs)Impute missing dataEstimate and combine: One easy step© 2023 StataCorp LLC | Stata is a registered trademark of StataCorp LLC, 4905 Lakeway Drive, College Station, TX 77845, USA./multiple-imputationA er estimation, for example, perform hypothesis testing.At any stage of your analysis, perform data management as if you are working with one dataset, and mi will replicate the changes correctly across the imputed datasets. Stata o ers full data management of MI data: create or drop variables and observations, change values, merge or append ﬁles, add imputations, and more.Accidentally dropped an observation from one of the imputed datasets, or changed a value of a variable, or dropped a variable, or ...? Stata veriﬁes the integrity of your MI data each time the mi command is run. (You can also do this manually by using mi update .) For example, Stata checks that complete variables contain the same values in the imputed data as in the original data, that incomplete variables contain the same nonmissing values in the imputed data as in the original,and more.If an inconsistency is detected, Stata tries to ﬁx the problem and notiﬁes you about the result.Estimate transformations of coe cients, compute predictions, and more.Stata’s mi command uniquely provides full data management support, veriﬁcation of integrity of MI data at any step of the analysis, and multiple formats for storing MI data e ciently. And you can even add your own imputation methods!Use an intuitive MI Control Panel to guide you through all the stages of your MI analysis—from examining missing values and th eir pa erns to performing MI inference. Th e corresponding Stata commands are produced with every step for reproducibility and, if desired, later interactive use.Stata o ers several styles for storing your MI data: you can store imputations in one ﬁle or separate ﬁles or in one variable or multiple variables. Some styles are more memory e cient, and others are more computationally e cient. Also, some tasks are easier in speciﬁc styles.You can start with one style at the beginning of your MI analysis, for example, “full long”, in which imputations are saved as extra observations:. mi set flongIf needed, switch to another style during your mi session, for example, to the wide style, in which imputations are saved as extra variables:. mi convert wideCan’t ﬁnd an imputation method you need? With li le e ort, you can program your own. Write a program to impute your variables once, and then simply use it with mi impute to obtain multiple imputations.program mi_impute_cmd_mymethod... program imputing missing values once ...end. mi impute mymethod ..., add(5) ...Manage imputed dataMultiple storage formatsAdd your own imputation methodsVerify imputed dataInferenceIn addition...Control Panel。

《神经网络与深度学习综述DeepLearning15May2014

Draft:Deep Learning in Neural Networks:An OverviewTechnical Report IDSIA-03-14/arXiv:1404.7828(v1.5)[cs.NE]J¨u rgen SchmidhuberThe Swiss AI Lab IDSIAIstituto Dalle Molle di Studi sull’Intelligenza ArtiﬁcialeUniversity of Lugano&SUPSIGalleria2,6928Manno-LuganoSwitzerland15May2014AbstractIn recent years,deep artiﬁcial neural networks(including recurrent ones)have won numerous con-tests in pattern recognition and machine learning.This historical survey compactly summarises relevantwork,much of it from the previous millennium.Shallow and deep learners are distinguished by thedepth of their credit assignment paths,which are chains of possibly learnable,causal links between ac-tions and effects.I review deep supervised learning(also recapitulating the history of backpropagation),unsupervised learning,reinforcement learning&evolutionary computation,and indirect search for shortprograms encoding deep and large networks.PDF of earlier draft(v1):http://www.idsia.ch/∼juergen/DeepLearning30April2014.pdfLATEX source:http://www.idsia.ch/∼juergen/DeepLearning30April2014.texComplete BIBTEXﬁle:http://www.idsia.ch/∼juergen/bib.bibPrefaceThis is the draft of an invited Deep Learning(DL)overview.One of its goals is to assign credit to those who contributed to the present state of the art.I acknowledge the limitations of attempting to achieve this goal.The DL research community itself may be viewed as a continually evolving,deep network of scientists who have inﬂuenced each other in complex ways.Starting from recent DL results,I tried to trace back the origins of relevant ideas through the past half century and beyond,sometimes using“local search”to follow citations of citations backwards in time.Since not all DL publications properly acknowledge earlier relevant work,additional global search strategies were employed,aided by consulting numerous neural network experts.As a result,the present draft mostly consists of references(about800entries so far).Nevertheless,through an expert selection bias I may have missed important work.A related bias was surely introduced by my special familiarity with the work of my own DL research group in the past quarter-century.For these reasons,the present draft should be viewed as merely a snapshot of an ongoing credit assignment process.To help improve it,please do not hesitate to send corrections and suggestions to juergen@idsia.ch.Contents1Introduction to Deep Learning(DL)in Neural Networks(NNs)3 2Event-Oriented Notation for Activation Spreading in FNNs/RNNs3 3Depth of Credit Assignment Paths(CAPs)and of Problems4 4Recurring Themes of Deep Learning54.1Dynamic Programming(DP)for DL (5)4.2Unsupervised Learning(UL)Facilitating Supervised Learning(SL)and RL (6)4.3Occam’s Razor:Compression and Minimum Description Length(MDL) (6)4.4Learning Hierarchical Representations Through Deep SL,UL,RL (6)4.5Fast Graphics Processing Units(GPUs)for DL in NNs (6)5Supervised NNs,Some Helped by Unsupervised NNs75.11940s and Earlier (7)5.2Around1960:More Neurobiological Inspiration for DL (7)5.31965:Deep Networks Based on the Group Method of Data Handling(GMDH) (8)5.41979:Convolution+Weight Replication+Winner-Take-All(WTA) (8)5.51960-1981and Beyond:Development of Backpropagation(BP)for NNs (8)5.5.1BP for Weight-Sharing Feedforward NNs(FNNs)and Recurrent NNs(RNNs)..95.6Late1980s-2000:Numerous Improvements of NNs (9)5.6.1Ideas for Dealing with Long Time Lags and Deep CAPs (10)5.6.2Better BP Through Advanced Gradient Descent (10)5.6.3Discovering Low-Complexity,Problem-Solving NNs (11)5.6.4Potential Beneﬁts of UL for SL (11)5.71987:UL Through Autoencoder(AE)Hierarchies (12)5.81989:BP for Convolutional NNs(CNNs) (13)5.91991:Fundamental Deep Learning Problem of Gradient Descent (13)5.101991:UL-Based History Compression Through a Deep Hierarchy of RNNs (14)5.111992:Max-Pooling(MP):Towards MPCNNs (14)5.121994:Contest-Winning Not So Deep NNs (15)5.131995:Supervised Recurrent Very Deep Learner(LSTM RNN) (15)5.142003:More Contest-Winning/Record-Setting,Often Not So Deep NNs (16)5.152006/7:Deep Belief Networks(DBNs)&AE Stacks Fine-Tuned by BP (17)5.162006/7:Improved CNNs/GPU-CNNs/BP-Trained MPCNNs (17)5.172009:First Ofﬁcial Competitions Won by RNNs,and with MPCNNs (18)5.182010:Plain Backprop(+Distortions)on GPU Yields Excellent Results (18)5.192011:MPCNNs on GPU Achieve Superhuman Vision Performance (18)5.202011:Hessian-Free Optimization for RNNs (19)5.212012:First Contests Won on ImageNet&Object Detection&Segmentation (19)5.222013-:More Contests and Benchmark Records (20)5.22.1Currently Successful Supervised Techniques:LSTM RNNs/GPU-MPCNNs (21)5.23Recent Tricks for Improving SL Deep NNs(Compare Sec.5.6.2,5.6.3) (21)5.24Consequences for Neuroscience (22)5.25DL with Spiking Neurons? (22)6DL in FNNs and RNNs for Reinforcement Learning(RL)236.1RL Through NN World Models Yields RNNs With Deep CAPs (23)6.2Deep FNNs for Traditional RL and Markov Decision Processes(MDPs) (24)6.3Deep RL RNNs for Partially Observable MDPs(POMDPs) (24)6.4RL Facilitated by Deep UL in FNNs and RNNs (25)6.5Deep Hierarchical RL(HRL)and Subgoal Learning with FNNs and RNNs (25)6.6Deep RL by Direct NN Search/Policy Gradients/Evolution (25)6.7Deep RL by Indirect Policy Search/Compressed NN Search (26)6.8Universal RL (27)7Conclusion271Introduction to Deep Learning(DL)in Neural Networks(NNs) Which modiﬁable components of a learning system are responsible for its success or failure?What changes to them improve performance?This has been called the fundamental credit assignment problem(Minsky, 1963).There are general credit assignment methods for universal problem solvers that are time-optimal in various theoretical senses(Sec.6.8).The present survey,however,will focus on the narrower,but now commercially important,subﬁeld of Deep Learning(DL)in Artiﬁcial Neural Networks(NNs).We are interested in accurate credit assignment across possibly many,often nonlinear,computational stages of NNs.Shallow NN-like models have been around for many decades if not centuries(Sec.5.1).Models with several successive nonlinear layers of neurons date back at least to the1960s(Sec.5.3)and1970s(Sec.5.5). An efﬁcient gradient descent method for teacher-based Supervised Learning(SL)in discrete,differentiable networks of arbitrary depth called backpropagation(BP)was developed in the1960s and1970s,and ap-plied to NNs in1981(Sec.5.5).BP-based training of deep NNs with many layers,however,had been found to be difﬁcult in practice by the late1980s(Sec.5.6),and had become an explicit research subject by the early1990s(Sec.5.9).DL became practically feasible to some extent through the help of Unsupervised Learning(UL)(e.g.,Sec.5.10,5.15).The1990s and2000s also saw many improvements of purely super-vised DL(Sec.5).In the new millennium,deep NNs haveﬁnally attracted wide-spread attention,mainly by outperforming alternative machine learning methods such as kernel machines(Vapnik,1995;Sch¨o lkopf et al.,1998)in numerous important applications.In fact,supervised deep NNs have won numerous of-ﬁcial international pattern recognition competitions(e.g.,Sec.5.17,5.19,5.21,5.22),achieving theﬁrst superhuman visual pattern recognition results in limited domains(Sec.5.19).Deep NNs also have become relevant for the more generalﬁeld of Reinforcement Learning(RL)where there is no supervising teacher (Sec.6).Both feedforward(acyclic)NNs(FNNs)and recurrent(cyclic)NNs(RNNs)have won contests(Sec.5.12,5.14,5.17,5.19,5.21,5.22).In a sense,RNNs are the deepest of all NNs(Sec.3)—they are general computers more powerful than FNNs,and can in principle create and process memories of ar-bitrary sequences of input patterns(e.g.,Siegelmann and Sontag,1991;Schmidhuber,1990a).Unlike traditional methods for automatic sequential program synthesis(e.g.,Waldinger and Lee,1969;Balzer, 1985;Soloway,1986;Deville and Lau,1994),RNNs can learn programs that mix sequential and parallel information processing in a natural and efﬁcient way,exploiting the massive parallelism viewed as crucial for sustaining the rapid decline of computation cost observed over the past75years.The rest of this paper is structured as follows.Sec.2introduces a compact,event-oriented notation that is simple yet general enough to accommodate both FNNs and RNNs.Sec.3introduces the concept of Credit Assignment Paths(CAPs)to measure whether learning in a given NN application is of the deep or shallow type.Sec.4lists recurring themes of DL in SL,UL,and RL.Sec.5focuses on SL and UL,and on how UL can facilitate SL,although pure SL has become dominant in recent competitions(Sec.5.17-5.22). Sec.5is arranged in a historical timeline format with subsections on important inspirations and technical contributions.Sec.6on deep RL discusses traditional Dynamic Programming(DP)-based RL combined with gradient-based search techniques for SL or UL in deep NNs,as well as general methods for direct and indirect search in the weight space of deep FNNs and RNNs,including successful policy gradient and evolutionary methods.2Event-Oriented Notation for Activation Spreading in FNNs/RNNs Throughout this paper,let i,j,k,t,p,q,r denote positive integer variables assuming ranges implicit in the given contexts.Let n,m,T denote positive integer constants.An NN’s topology may change over time(e.g.,Fahlman,1991;Ring,1991;Weng et al.,1992;Fritzke, 1994).At any given moment,it can be described as aﬁnite subset of units(or nodes or neurons)N= {u1,u2,...,}and aﬁnite set H⊆N×N of directed edges or connections between nodes.FNNs are acyclic graphs,RNNs cyclic.Theﬁrst(input)layer is the set of input units,a subset of N.In FNNs,the k-th layer(k>1)is the set of all nodes u∈N such that there is an edge path of length k−1(but no longer path)between some input unit and u.There may be shortcut connections between distant layers.The NN’s behavior or program is determined by a set of real-valued,possibly modiﬁable,parameters or weights w i(i=1,...,n).We now focus on a singleﬁnite episode or epoch of information processing and activation spreading,without learning through weight changes.The following slightly unconventional notation is designed to compactly describe what is happening during the runtime of the system.During an episode,there is a partially causal sequence x t(t=1,...,T)of real values that I call events.Each x t is either an input set by the environment,or the activation of a unit that may directly depend on other x k(k<t)through a current NN topology-dependent set in t of indices k representing incoming causal connections or links.Let the function v encode topology information and map such event index pairs(k,t)to weight indices.For example,in the non-input case we may have x t=f t(net t)with real-valued net t= k∈in t x k w v(k,t)(additive case)or net t= k∈in t x k w v(k,t)(multiplicative case), where f t is a typically nonlinear real-valued activation function such as tanh.In many recent competition-winning NNs(Sec.5.19,5.21,5.22)there also are events of the type x t=max k∈int (x k);some networktypes may also use complex polynomial activation functions(Sec.5.3).x t may directly affect certain x k(k>t)through outgoing connections or links represented through a current set out t of indices k with t∈in k.Some non-input events are called output events.Note that many of the x t may refer to different,time-varying activations of the same unit in sequence-processing RNNs(e.g.,Williams,1989,“unfolding in time”),or also in FNNs sequentially exposed to time-varying input patterns of a large training set encoded as input events.During an episode,the same weight may get reused over and over again in topology-dependent ways,e.g.,in RNNs,or in convolutional NNs(Sec.5.4,5.8).I call this weight sharing across space and/or time.Weight sharing may greatly reduce the NN’s descriptive complexity,which is the number of bits of information required to describe the NN (Sec.4.3).In Supervised Learning(SL),certain NN output events x t may be associated with teacher-given,real-valued labels or targets d t yielding errors e t,e.g.,e t=1/2(x t−d t)2.A typical goal of supervised NN training is toﬁnd weights that yield episodes with small total error E,the sum of all such e t.The hope is that the NN will generalize well in later episodes,causing only small errors on previously unseen sequences of input events.Many alternative error functions for SL and UL are possible.SL assumes that input events are independent of earlier output events(which may affect the environ-ment through actions causing subsequent perceptions).This assumption does not hold in the broaderﬁelds of Sequential Decision Making and Reinforcement Learning(RL)(Kaelbling et al.,1996;Sutton and Barto, 1998;Hutter,2005)(Sec.6).In RL,some of the input events may encode real-valued reward signals given by the environment,and a typical goal is toﬁnd weights that yield episodes with a high sum of reward signals,through sequences of appropriate output actions.Sec.5.5will use the notation above to compactly describe a central algorithm of DL,namely,back-propagation(BP)for supervised weight-sharing FNNs and RNNs.(FNNs may be viewed as RNNs with certainﬁxed zero weights.)Sec.6will address the more general RL case.3Depth of Credit Assignment Paths(CAPs)and of ProblemsTo measure whether credit assignment in a given NN application is of the deep or shallow type,I introduce the concept of Credit Assignment Paths or CAPs,which are chains of possibly causal links between events.Let usﬁrst focus on SL.Consider two events x p and x q(1≤p<q≤T).Depending on the appli-cation,they may have a Potential Direct Causal Connection(PDCC)expressed by the Boolean predicate pdcc(p,q),which is true if and only if p∈in q.Then the2-element list(p,q)is deﬁned to be a CAP from p to q(a minimal one).A learning algorithm may be allowed to change w v(p,q)to improve performance in future episodes.More general,possibly indirect,Potential Causal Connections(PCC)are expressed by the recursively deﬁned Boolean predicate pcc(p,q),which in the SL case is true only if pdcc(p,q),or if pcc(p,k)for some k and pdcc(k,q).In the latter case,appending q to any CAP from p to k yields a CAP from p to q(this is a recursive deﬁnition,too).The set of such CAPs may be large but isﬁnite.Note that the same weight may affect many different PDCCs between successive events listed by a given CAP,e.g.,in the case of RNNs, or weight-sharing FNNs.Suppose a CAP has the form(...,k,t,...,q),where k and t(possibly t=q)are theﬁrst successive elements with modiﬁable w v(k,t).Then the length of the sufﬁx list(t,...,q)is called the CAP’s depth (which is0if there are no modiﬁable links at all).This depth limits how far backwards credit assignment can move down the causal chain toﬁnd a modiﬁable weight.1Suppose an episode and its event sequence x1,...,x T satisfy a computable criterion used to decide whether a given problem has been solved(e.g.,total error E below some threshold).Then the set of used weights is called a solution to the problem,and the depth of the deepest CAP within the sequence is called the solution’s depth.There may be other solutions(yielding different event sequences)with different depths.Given someﬁxed NN topology,the smallest depth of any solution is called the problem’s depth.Sometimes we also speak of the depth of an architecture:SL FNNs withﬁxed topology imply a problem-independent maximal problem depth bounded by the number of non-input layers.Certain SL RNNs withﬁxed weights for all connections except those to output units(Jaeger,2001;Maass et al.,2002; Jaeger,2004;Schrauwen et al.,2007)have a maximal problem depth of1,because only theﬁnal links in the corresponding CAPs are modiﬁable.In general,however,RNNs may learn to solve problems of potentially unlimited depth.Note that the deﬁnitions above are solely based on the depths of causal chains,and agnostic of the temporal distance between events.For example,shallow FNNs perceiving large“time windows”of in-put events may correctly classify long input sequences through appropriate output events,and thus solve shallow problems involving long time lags between relevant events.At which problem depth does Shallow Learning end,and Deep Learning begin?Discussions with DL experts have not yet yielded a conclusive response to this question.Instead of committing myself to a precise answer,let me just deﬁne for the purposes of this overview:problems of depth>10require Very Deep Learning.The difﬁculty of a problem may have little to do with its depth.Some NNs can quickly learn to solve certain deep problems,e.g.,through random weight guessing(Sec.5.9)or other types of direct search (Sec.6.6)or indirect search(Sec.6.7)in weight space,or through training an NNﬁrst on shallow problems whose solutions may then generalize to deep problems,or through collapsing sequences of(non)linear operations into a single(non)linear operation—but see an analysis of non-trivial aspects of deep linear networks(Baldi and Hornik,1994,Section B).In general,however,ﬁnding an NN that precisely models a given training set is an NP-complete problem(Judd,1990;Blum and Rivest,1992),also in the case of deep NNs(S´ıma,1994;de Souto et al.,1999;Windisch,2005);compare a survey of negative results(S´ıma, 2002,Section1).Above we have focused on SL.In the more general case of RL in unknown environments,pcc(p,q) is also true if x p is an output event and x q any later input event—any action may affect the environment and thus any later perception.(In the real world,the environment may even inﬂuence non-input events computed on a physical hardware entangled with the entire universe,but this is ignored here.)It is possible to model and replace such unmodiﬁable environmental PCCs through a part of the NN that has already learned to predict(through some of its units)input events(including reward signals)from former input events and actions(Sec.6.1).Its weights are frozen,but can help to assign credit to other,still modiﬁable weights used to compute actions(Sec.6.1).This approach may lead to very deep CAPs though.Some DL research is about automatically rephrasing problems such that their depth is reduced(Sec.4). In particular,sometimes UL is used to make SL problems less deep,e.g.,Sec.5.10.Often Dynamic Programming(Sec.4.1)is used to facilitate certain traditional RL problems,e.g.,Sec.6.2.Sec.5focuses on CAPs for SL,Sec.6on the more complex case of RL.4Recurring Themes of Deep Learning4.1Dynamic Programming(DP)for DLOne recurring theme of DL is Dynamic Programming(DP)(Bellman,1957),which can help to facili-tate credit assignment under certain assumptions.For example,in SL NNs,backpropagation itself can 1An alternative would be to count only modiﬁable links when measuring depth.In many typical NN applications this would not make a difference,but in some it would,e.g.,Sec.6.1.be viewed as a DP-derived method(Sec.5.5).In traditional RL based on strong Markovian assumptions, DP-derived methods can help to greatly reduce problem depth(Sec.6.2).DP algorithms are also essen-tial for systems that combine concepts of NNs and graphical models,such as Hidden Markov Models (HMMs)(Stratonovich,1960;Baum and Petrie,1966)and Expectation Maximization(EM)(Dempster et al.,1977),e.g.,(Bottou,1991;Bengio,1991;Bourlard and Morgan,1994;Baldi and Chauvin,1996; Jordan and Sejnowski,2001;Bishop,2006;Poon and Domingos,2011;Dahl et al.,2012;Hinton et al., 2012a).4.2Unsupervised Learning(UL)Facilitating Supervised Learning(SL)and RL Another recurring theme is how UL can facilitate both SL(Sec.5)and RL(Sec.6).UL(Sec.5.6.4) is normally used to encode raw incoming data such as video or speech streams in a form that is more convenient for subsequent goal-directed learning.In particular,codes that describe the original data in a less redundant or more compact way can be fed into SL(Sec.5.10,5.15)or RL machines(Sec.6.4),whose search spaces may thus become smaller(and whose CAPs shallower)than those necessary for dealing with the raw data.UL is closely connected to the topics of regularization and compression(Sec.4.3,5.6.3). 4.3Occam’s Razor:Compression and Minimum Description Length(MDL) Occam’s razor favors simple solutions over complex ones.Given some programming language,the prin-ciple of Minimum Description Length(MDL)can be used to measure the complexity of a solution candi-date by the length of the shortest program that computes it(e.g.,Solomonoff,1964;Kolmogorov,1965b; Chaitin,1966;Wallace and Boulton,1968;Levin,1973a;Rissanen,1986;Blumer et al.,1987;Li and Vit´a nyi,1997;Gr¨u nwald et al.,2005).Some methods explicitly take into account program runtime(Al-lender,1992;Watanabe,1992;Schmidhuber,2002,1995);many consider only programs with constant runtime,written in non-universal programming languages(e.g.,Rissanen,1986;Hinton and van Camp, 1993).In the NN case,the MDL principle suggests that low NN weight complexity corresponds to high NN probability in the Bayesian view(e.g.,MacKay,1992;Buntine and Weigend,1991;De Freitas,2003), and to high generalization performance(e.g.,Baum and Haussler,1989),without overﬁtting the training data.Many methods have been proposed for regularizing NNs,that is,searching for solution-computing, low-complexity SL NNs(Sec.5.6.3)and RL NNs(Sec.6.7).This is closely related to certain UL methods (Sec.4.2,5.6.4).4.4Learning Hierarchical Representations Through Deep SL,UL,RLMany methods of Good Old-Fashioned Artiﬁcial Intelligence(GOFAI)(Nilsson,1980)as well as more recent approaches to AI(Russell et al.,1995)and Machine Learning(Mitchell,1997)learn hierarchies of more and more abstract data representations.For example,certain methods of syntactic pattern recog-nition(Fu,1977)such as grammar induction discover hierarchies of formal rules to model observations. The partially(un)supervised Automated Mathematician/EURISKO(Lenat,1983;Lenat and Brown,1984) continually learns concepts by combining previously learnt concepts.Such hierarchical representation learning(Ring,1994;Bengio et al.,2013;Deng and Yu,2014)is also a recurring theme of DL NNs for SL (Sec.5),UL-aided SL(Sec.5.7,5.10,5.15),and hierarchical RL(Sec.6.5).Often,abstract hierarchical representations are natural by-products of data compression(Sec.4.3),e.g.,Sec.5.10.4.5Fast Graphics Processing Units(GPUs)for DL in NNsWhile the previous millennium saw several attempts at creating fast NN-speciﬁc hardware(e.g.,Jackel et al.,1990;Faggin,1992;Ramacher et al.,1993;Widrow et al.,1994;Heemskerk,1995;Korkin et al., 1997;Urlbe,1999),and at exploiting standard hardware(e.g.,Anguita et al.,1994;Muller et al.,1995; Anguita and Gomes,1996),the new millennium brought a DL breakthrough in form of cheap,multi-processor graphics cards or GPUs.GPUs are widely used for video games,a huge and competitive market that has driven down hardware prices.GPUs excel at fast matrix and vector multiplications required not only for convincing virtual realities but also for NN training,where they can speed up learning by a factorof50and more.Some of the GPU-based FNN implementations(Sec.5.16-5.19)have greatly contributed to recent successes in contests for pattern recognition(Sec.5.19-5.22),image segmentation(Sec.5.21), and object detection(Sec.5.21-5.22).5Supervised NNs,Some Helped by Unsupervised NNsThe main focus of current practical applications is on Supervised Learning(SL),which has dominated re-cent pattern recognition contests(Sec.5.17-5.22).Several methods,however,use additional Unsupervised Learning(UL)to facilitate SL(Sec.5.7,5.10,5.15).It does make sense to treat SL and UL in the same section:often gradient-based methods,such as BP(Sec.5.5.1),are used to optimize objective functions of both UL and SL,and the boundary between SL and UL may blur,for example,when it comes to time series prediction and sequence classiﬁcation,e.g.,Sec.5.10,5.12.A historical timeline format will help to arrange subsections on important inspirations and techni-cal contributions(although such a subsection may span a time interval of many years).Sec.5.1brieﬂy mentions early,shallow NN models since the1940s,Sec.5.2additional early neurobiological inspiration relevant for modern Deep Learning(DL).Sec.5.3is about GMDH networks(since1965),perhaps theﬁrst (feedforward)DL systems.Sec.5.4is about the relatively deep Neocognitron NN(1979)which is similar to certain modern deep FNN architectures,as it combines convolutional NNs(CNNs),weight pattern repli-cation,and winner-take-all(WTA)mechanisms.Sec.5.5uses the notation of Sec.2to compactly describe a central algorithm of DL,namely,backpropagation(BP)for supervised weight-sharing FNNs and RNNs. It also summarizes the history of BP1960-1981and beyond.Sec.5.6describes problems encountered in the late1980s with BP for deep NNs,and mentions several ideas from the previous millennium to overcome them.Sec.5.7discusses aﬁrst hierarchical stack of coupled UL-based Autoencoders(AEs)—this concept resurfaced in the new millennium(Sec.5.15).Sec.5.8is about applying BP to CNNs,which is important for today’s DL applications.Sec.5.9explains BP’s Fundamental DL Problem(of vanishing/exploding gradients)discovered in1991.Sec.5.10explains how a deep RNN stack of1991(the History Compressor) pre-trained by UL helped to solve previously unlearnable DL benchmarks requiring Credit Assignment Paths(CAPs,Sec.3)of depth1000and more.Sec.5.11discusses a particular WTA method called Max-Pooling(MP)important in today’s DL FNNs.Sec.5.12mentions aﬁrst important contest won by SL NNs in1994.Sec.5.13describes a purely supervised DL RNN(Long Short-Term Memory,LSTM)for problems of depth1000and more.Sec.5.14mentions an early contest of2003won by an ensemble of shallow NNs, as well as good pattern recognition results with CNNs and LSTM RNNs(2003).Sec.5.15is mostly about Deep Belief Networks(DBNs,2006)and related stacks of Autoencoders(AEs,Sec.5.7)pre-trained by UL to facilitate BP-based SL.Sec.5.16mentions theﬁrst BP-trained MPCNNs(2007)and GPU-CNNs(2006). Sec.5.17-5.22focus on ofﬁcial competitions with secret test sets won by(mostly purely supervised)DL NNs since2009,in sequence recognition,image classiﬁcation,image segmentation,and object detection. Many RNN results depended on LSTM(Sec.5.13);many FNN results depended on GPU-based FNN code developed since2004(Sec.5.16,5.17,5.18,5.19),in particular,GPU-MPCNNs(Sec.5.19).5.11940s and EarlierNN research started in the1940s(e.g.,McCulloch and Pitts,1943;Hebb,1949);compare also later work on learning NNs(Rosenblatt,1958,1962;Widrow and Hoff,1962;Grossberg,1969;Kohonen,1972; von der Malsburg,1973;Narendra and Thathatchar,1974;Willshaw and von der Malsburg,1976;Palm, 1980;Hopﬁeld,1982).In a sense NNs have been around even longer,since early supervised NNs were essentially variants of linear regression methods going back at least to the early1800s(e.g.,Legendre, 1805;Gauss,1809,1821).Early NNs had a maximal CAP depth of1(Sec.3).5.2Around1960:More Neurobiological Inspiration for DLSimple cells and complex cells were found in the cat’s visual cortex(e.g.,Hubel and Wiesel,1962;Wiesel and Hubel,1959).These cellsﬁre in response to certain properties of visual sensory inputs,such as theorientation of plex cells exhibit more spatial invariance than simple cells.This inspired later deep NN architectures(Sec.5.4)used in certain modern award-winning Deep Learners(Sec.5.19-5.22).5.31965:Deep Networks Based on the Group Method of Data Handling(GMDH) Networks trained by the Group Method of Data Handling(GMDH)(Ivakhnenko and Lapa,1965; Ivakhnenko et al.,1967;Ivakhnenko,1968,1971)were perhaps theﬁrst DL systems of the Feedforward Multilayer Perceptron type.The units of GMDH nets may have polynomial activation functions imple-menting Kolmogorov-Gabor polynomials(more general than traditional NN activation functions).Given a training set,layers are incrementally grown and trained by regression analysis,then pruned with the help of a separate validation set(using today’s terminology),where Decision Regularisation is used to weed out superﬂuous units.The numbers of layers and units per layer can be learned in problem-dependent fashion. This is a good example of hierarchical representation learning(Sec.4.4).There have been numerous ap-plications of GMDH-style networks,e.g.(Ikeda et al.,1976;Farlow,1984;Madala and Ivakhnenko,1994; Ivakhnenko,1995;Kondo,1998;Kord´ık et al.,2003;Witczak et al.,2006;Kondo and Ueno,2008).5.41979:Convolution+Weight Replication+Winner-Take-All(WTA)Apart from deep GMDH networks(Sec.5.3),the Neocognitron(Fukushima,1979,1980,2013a)was per-haps theﬁrst artiﬁcial NN that deserved the attribute deep,and theﬁrst to incorporate the neurophysiolog-ical insights of Sec.5.2.It introduced convolutional NNs(today often called CNNs or convnets),where the(typically rectangular)receptiveﬁeld of a convolutional unit with given weight vector is shifted step by step across a2-dimensional array of input values,such as the pixels of an image.The resulting2D array of subsequent activation events of this unit can then provide inputs to higher-level units,and so on.Due to massive weight replication(Sec.2),relatively few parameters may be necessary to describe the behavior of such a convolutional layer.Competition layers have WTA subsets whose maximally active units are the only ones to adopt non-zero activation values.They essentially“down-sample”the competition layer’s input.This helps to create units whose responses are insensitive to small image shifts(compare Sec.5.2).The Neocognitron is very similar to the architecture of modern,contest-winning,purely super-vised,feedforward,gradient-based Deep Learners with alternating convolutional and competition lay-ers(e.g.,Sec.5.19-5.22).Fukushima,however,did not set the weights by supervised backpropagation (Sec.5.5,5.8),but by local un supervised learning rules(e.g.,Fukushima,2013b),or by pre-wiring.In that sense he did not care for the DL problem(Sec.5.9),although his architecture was comparatively deep indeed.He also used Spatial Averaging(Fukushima,1980,2011)instead of Max-Pooling(MP,Sec.5.11), currently a particularly convenient and popular WTA mechanism.Today’s CNN-based DL machines proﬁta lot from later CNN work(e.g.,LeCun et al.,1989;Ranzato et al.,2007)(Sec.5.8,5.16,5.19).5.51960-1981and Beyond:Development of Backpropagation(BP)for NNsThe minimisation of errors through gradient descent(Hadamard,1908)in the parameter space of com-plex,nonlinear,differentiable,multi-stage,NN-related systems has been discussed at least since the early 1960s(e.g.,Kelley,1960;Bryson,1961;Bryson and Denham,1961;Pontryagin et al.,1961;Dreyfus,1962; Wilkinson,1965;Amari,1967;Bryson and Ho,1969;Director and Rohrer,1969;Griewank,2012),ini-tially within the framework of Euler-LaGrange equations in the Calculus of Variations(e.g.,Euler,1744). Steepest descent in such systems can be performed(Bryson,1961;Kelley,1960;Bryson and Ho,1969)by iterating the ancient chain rule(Leibniz,1676;L’Hˆo pital,1696)in Dynamic Programming(DP)style(Bell-man,1957).A simpliﬁed derivation of the method uses the chain rule only(Dreyfus,1962).The methods of the1960s were already efﬁcient in the DP sense.However,they backpropagated derivative information through standard Jacobian matrix calculations from one“layer”to the previous one, explicitly addressing neither direct links across several layers nor potential additional efﬁciency gains due to network sparsity(but perhaps such enhancements seemed obvious to the authors).。

mobilenetv3 激活函数

MobileNetV3激活函数详解MobileNetV3是一种轻量级的卷积神经网络模型，广泛应用于移动设备和嵌入式系统中。

激活函数在神经网络中起到了至关重要的作用，能够引入非线性因素，增加模型的表达能力。

MobileNetV3中使用了一些特定的激活函数，本文将详细解释这些函数的定义、用途和工作方式。

1. Hard SwishHard Swish是MobileNetV3中最常用的激活函数之一。

它是由Google提出的一种非线性激活函数，结合了ReLU和Swish两种激活函数的优点。

Hard Swish函数定义如下：f(x) = x * relu6(x + 3) / 6其中relu6(x)表示输入x与0之间取较大值，并将结果限制在[0, 6]范围内。

Hard Swish可以看作是ReLU和Swish的结合体，具有ReLU快速收敛和Swish平滑非线性特点。

Hard Swish在MobileNetV3中被广泛应用于卷积层之后，能够有效地提高模型的准确率，并且具有计算效率高、参数量少等优势。

2. Hard SigmoidHard Sigmoid是另一种常见的激活函数，也是MobileNetV3中使用的激活函数之一。

它是对Sigmoid函数的一种近似，能够在保持计算效率的同时引入非线性因素。

Hard Sigmoid函数定义如下：f(x) = relu6(x + 3) / 6其中relu6(x)同样表示输入x与0之间取较大值，并将结果限制在[0, 6]范围内。

Hard Sigmoid可以看作是对Sigmoid函数进行了截断，去掉了曲线两端的平缓区域，使其更加接近于阶跃函数。

Hard Sigmoid在MobileNetV3中主要被用于深度可分离卷积层(Depthwise Separable Convolution)，能够有效地提高模型的准确率，并且具有计算效率高、参数量少等优势。

3. MishMish是一种新颖的激活函数，由Misra提出。

提示工程(prompt)

客户问题：你好，我有一个关于我的汽车保险的问题。分类主题：1
客户问题：{在此插入新问题} 分类主题：
之前
单个家庭的年用水需求是多少？包含四个在家的人的家庭每年平均200天，使用量平均为100 每天多少升水？
单户家庭的年用水需求包含四个人，他们平均在家的时间为每年200天，平均使用100升的每天的用水量为每年800万升。
2nd prompt—答案提取
一个普通人在一分钟内能打出25拳。一场比赛持续5个回合，每回合3分钟。有多少他投掷了多少拳？
让我们一步一步地思考并解释这个计算过程。一步一步地。
在一分钟内，乔打了25拳。
在三分钟内，乔打了3 * 25 = 75拳。
在五轮中，乔投掷了5*75 = 375次拳。
在一分钟内，乔打了25拳。在三分钟内，乔打了3 * 25 = 75拳。在五轮中，乔投掷了5*75 = 375次拳。
提示链
摘要
请总结信息关于产品 [{'text': 'iPhone', 'type': '产品'}], {'text': '苹果', 'type': 'ORGANIZATION’}]
iPhone是一系列智能手机。由苹果设计并销售 Inc. 这些设备结合了计算机、相机和互联网单一设备中的连接性，以及以其简洁的设计而闻名用户友好的界面。
人工智能机器学习深度学习
生成式人工智能
1956
人工智能
计算机科学领域中致力于创造的分支能够复制或超越人类智能的智能机器人类智能
1997
机器学习
人工智能的子集，使机器能够从现有数据，并在该数据的基础上进行改进，以使其决策或预测

语义三元组提取-概述说明以及解释

语义三元组提取-概述说明以及解释1.引言1.1 概述概述:语义三元组提取是一种自然语言处理技术，旨在从文本中自动抽取出具有主谓宾结构的语义信息。

通过将句子中的实体与它们之间的关系抽取出来，形成三元组（subject-predicate-object）的形式，从而获得更加结构化和可理解的语义信息。

这项技术在信息检索、知识图谱构建、语义分析等领域具有广泛的应用前景。

概述部分将介绍语义三元组提取的基本概念、意义以及本文所要探讨的重点内容。

通过对语义三元组提取技术的介绍，读者可以更好地理解本文后续内容的研究意义和应用场景。

1.2 文章结构本文将分为三个主要部分，分别是引言、正文和结论。

在引言部分，将从概述、文章结构和目的三个方面介绍本文的主题内容。

首先，我们将简要介绍语义三元组提取的背景和意义，引出本文的研究对象。

接着，我们将介绍文章的整体结构，明确各个部分的内容安排和逻辑关系。

最后，我们将阐明本文的研究目的，明确本文要解决的问题和所带来的意义。

在正文部分，将主要分为三个小节。

首先，我们将介绍语义三元组的概念，包括其定义、特点和构成要素。

接着，我们将系统梳理语义三元组提取的方法，包括基于规则的方法、基于统计的方法和基于深度学习的方法等。

最后，我们将探讨语义三元组在实际应用中的场景，包括知识图谱构建、搜索引擎优化和自然语言处理等方面。

在结论部分，将对前文所述内容进行总结和展望。

首先，我们将概括本文的研究成果和亮点，指出语义三元组提取的重要性和必要性。

接着，我们将展望未来研究方向和发展趋势，探索语义三元组在智能技术领域的潜在应用价值。

最后，我们将用简洁的语言作出结束语，强调语义三元组提取对于推动智能化发展的意义和价值。

1.3 目的本文的目的是介绍语义三元组提取这一技术，并探讨其在自然语言处理、知识图谱构建、语义分析等领域的重要性和应用价值。

通过对语义三元组概念和提取方法的讨论，希望能够帮助读者更好地理解和应用这一技术，提高对文本语义信息的理解和利用能力。

Perfect nizk with adaptive soundness

Perfect NIZK with Adaptive SoundnessMasayuki Abe1Serge Fehr2November17,20061Information Sharing Platform Laboratories,NTT Corporation,Japanabe.masayuki@lab.ntt.co.jp2CWI Amsterdam,The Netherlandsfehr@cwi.nlAbstractThe notion of non-interactive zero-knowledge(NIZK)is of fundamental importance in cryptography.Despite the vast attention the concept of NIZK has attracted since its intro-duction,one question has remained very resistant:Is it possible to construct NIZK schemesfor any NP-language with statistical or even perfect ZK?Groth,Ostrovsky and Sahai recentlypositively answers to the question by presenting a couple of elegant constructions.However,their schemes pose a limitation on the length of the proof statement to achieve adaptivesoundness against dishonest provers who may choose the target statement depending on thecommon reference string(CRS).In this work,weﬁrst present a very simple and eﬃcient adaptively-sound perfect NIZK argument system for any NP-language.Besides being theﬁrst adaptively-sound statisticalNIZK argument for all NP that does not pose any restriction on the statements to be proven,it enjoys a number of additional desirable properties:it allows to re-use the CRS,it canhandle arithmetic circuits,and the CRS can be set-up very eﬃciently without the need foran honest party.We then show an application of our techniques in constructing eﬃcientNIZK schemes for proving arithmetic relations among committed secrets,whereas previousmethods required expensive generic NP-reductions.The security of the proposed schemes is based on a strong non-standard assumption, an extended version of the so-called Knowledge-of-Exponent Assumption(KEA)over bilin-ear groups.We give some justiﬁcation for using such an assumption by showing that thecommonly-used approach for proving NIZK arguments sound does not allow for adaptively-sound statistical NIZK arguments(unless NP⊂P/poly).Furthermore,we show that theassumption used in our construction holds with respect to generic adversaries that do notexploit the speciﬁc representation of the group elements.We also discuss how to avoid thenon-standard assumption in a pre-processing model.1Introduction1.1BackgroundNon-Interactive Zero-Knowledge.The notion of non-interactive zero-knowledge(NIZK) captures the problem of proving that a statement is true by just sending one message and without revealing any additional information besides the validity of the statement,provided that a common reference string(CRS)has been properly set up.Since its introduction by Blum,Feldman and Micali in1988[6],NIZK has been a fundamental cryptographic primitive used throughout modern cryptography in essential ways.There is a considerable amount of literature dedicated to NIZK,in particular to the study of which languages allow for whatﬂavor of NIZK proof.As in case of interactive ZK it is well known that there cannot be statistical NIZK proofs(i.e.,both ZK and soundness are unconditional) for NP-complete languages unless the polynomial hierarchy collapses[22,2,30].Hence,when considering general NP-languages,this only leaves room for a NIZK proof with computational ZK or computational soundness(where the proof is also called an argument),or both.However, in contrast to interactive ZK where it has long been known that bothﬂavors can exist[8,7,23], all proposed NIZK proofs or arguments for general NP-languages have computational ZK(see e.g.[6,20,5,27,15]).Hence the construction of a statistically NIZK(NISZK)argument has remained an open problem(until very recently,see below).The question of the existence of NISZK arguments is in particular interesting in combination with a result by De Santis et al.[15],where they observe that for a strong notion of NIZK,called same-string NIZK,soundness can only be computational when considering NP-complete languages(assuming that one-way functions exist).Statistical NIZK Arguments.Recently,Groth,Ostrovsky and Sahai proposed an elegantconstruction for a perfect NIZK(NIPZK)argument for circuit-SAT[24]by using bilinear groups. This shows NIZK can come with perfect ZK for any NP-language.However,the scheme only provides security against a non-adaptive dishonest prover who chooses the target instance x∗∈L (for which it wants to fake a proof)independent of the CRS.In an application though,it is likely that the adversaryﬁrst sees the CRS and then chooses the false statement on which he wants to ing a counting argument,they argue that under some strengthened assumption their scheme is secure against an adaptive dishonest prover if the size of the circuit to be proven is a-priori limited.However,the bound on the size of the circuit is so restrictive that the circuit must be smaller than sublinear in the bit size of the CRS(as discussed in Section1.3).Groth et al.also proposed a perfect NIZK argument for SAT which is provably secure in Canetti’s Universal Composability(UC)framework[9].However,besides being much less eﬃcient than theirﬁrst construction,the scheme still does not guarantee unrestricted security against an adaptive dishonest prover who chooses the target instance x∗∈L depending on the CRS.For instance,the UC security does not exclude the possibility that a dishonest prover comes up with an accepting proof for the statement“the CRS is invalid or S is true”for an arbitrary false statement S.Since in a real-life execution the CRS is assumed to be valid,this is a convincing argument of the false statement S.Accordingly,the existence of an unrestricted statistical or perfect NIZK argument,which does not pose any restriction on the instances to be proven,is still an open problem.The Knowledge-of-Exponent rmally,the Knowledge-of-Exponent As-sumption(kea)says that for certain groups,given a pair g andˆg=g x of group elements with unknown discrete-log x,the only way to eﬃciently come up with another pair A andˆA such that ˆA=A x(for the same x)is by raising g andˆg to some power a:A=g a andˆA=ˆg a.kea wasﬁrst introduced and used by Damg˚ard in1991[12],and later,together with an extended version (kea2),by Hada and Tanaka[25].Recently,Bellare and Palacio[4]showed that kea2does not hold,and proposed a new extended version called kea3in order to save Hada and Tanaka’s results.kea3,which we call xkea for eXtended kea,says that given two pairs(g,ˆg)and(h,ˆh) with the same unknown discrete-log x,the only way to eﬃciently come up with another pair A andˆA such thatˆA=A x is by computing A=g a hαandˆA=ˆg aˆhα.Assumptions like kea and xkea are widely criticized in particular because they do not appear to be“eﬃciently falsiﬁable”, as Naor put it[28],though Bellare and Palacio showed that this is not necessarily the case.1.2Our ResultBased on xkea over bilinear groups,we construct an adaptively-sound NIPZK argument for circuit-SAT without any restrictions on the instances to be proven.Besides being theﬁrst un-restricted adaptively-sound NISZK argument for any NP-language,the proposed scheme enjoys a number of additional desirable properties:It is same-string NIZK,which allows to re-use the CRS.It is very eﬃcient:the CRS essentially consists of a few group elements,and a proof consists of a few group elements per multiplication gate;this is comparable(if not better)to the ﬁrst scheme by Groth et al.,which is the most eﬃcient general-purpose NIZK scheme known up to date(see the comparison in[24]).Furthermore,our scheme can also be applied to arithmetic circuits over Z q for a large prime q whereas known schemes are tailored to binary circuits;this often allows a more compact representation of the statement to be proven.Finally,the CRS does not need to be set-up by a trusted party.It can eﬃciently be set-up jointly by the prover and the veriﬁer.Furthermore,it can even be provided solely from a(possibly dishonest)veriﬁer without any correctness proof if we view the proof system as a zap[19]rather than a NIZK.We are not aware of any other NIZK arguments or proofs that enjoy all these desirable properties.Based on the techniques developed for the perfect NIZK argument for SAT,we also construct an eﬃcient NIPZK argument for arithmetic relations among committed secrets over Z q with large prime q.To the best of our knowledge,all known schemes only work for secrets from restricted domains such as Z[2]and have to rely on generic ineﬃcient reductions to NP-complete problems to handle larger secrets.Our approach in particular allows for additive and multiplicative relations among secrets committed to by standard Pedersen commitments.We give two justiﬁcations for using such a strong non-standard assumption like xkea.First, we give some indication that a non-standard assumption is unavoidable for adaptively-sound NISZK arguments.We prove that using the common approach for proving computational soundness,which has been used for all NIZK arguments(we are aware of),a non-standard assumption is necessary unless NP⊂P/poly(i.e.unless any NP-problem can be solved by an eﬃcient non-uniform algorithm).And,second,we prove that kea and xkea hold in the generic group model(even over bilinear groups).This suggests that if there exists an algorithm that breaks,say,kea in a certain group,then this algorithm must use the speciﬁc representation of the elements of that group,and it is likely to fail when some other group(representation)is used.A similar result was independently developed by Dent[18]for non-bilinear groups.Finally,we discuss how to avoid xkea in our NIZK arguments by allowing a pre-processing phase.Our scheme allows very eﬃcient pre-processing where the prover only need to make random commitments and prove its knowledge about the witness by using eﬃcient oﬀ-the-shelf zero-knoweldge schemes.1.3Related WorkIn order to make it easier for the reader to position our results,we would like to give a brief discussion about recently proposed NIPZK arguments.In[24]Groth et al.presented two schemes for proving circuit satisﬁability,where theﬁrst one comes in twoﬂavors.Let us name the resulting three schemes by the non-adaptive,the adaptive and the UC GOS scheme.These are theﬁrst(and so far only)NISZK arguments proposed in the literature.The non-adaptive GOS scheme is admitted by the authors to be not adaptively sound.The adaptive GOS scheme is adaptively sound,but it only allows for circuits that are limited in size,and the underlying computational assumption is somewhat non-standard in that it requires that some problem can only be solved with“sub-negligible”probability,like2−ǫκǫlogκnegl(κ)whereκis the bit size of the problem instance.The more one relaxes the bound on the size of the circuits,the strongerthe underlying assumption gets in terms of the assumed bound on the success probability of solving the problem;but in any case the size of the circuits are doomed to be sub-linear in the size of the CRS.Concerning the UC GOS scheme,weﬁrst would like to point out that it is of theoretical interest,but it is very ineﬃcient(though poly-time).Furthermore,it has some tricky weak soundness property in that if a dishonest prover should succeed in proving a false statement, then the statement cannot be distinguished from a true one.It is therefore claimed in[24]that the scheme“achieves a weaker,but suﬃcient,form of adaptive security.”This is true but only if some care is taken with the kind of statements that the(dishonest)prover is allowed to prove; in particular,soundness is only guaranteed if the statement to be proven does not incorporate the CRS.Indeed,the same example that the authors use to reason that theirﬁrst scheme is not adaptively sound can also be applied to the UC secure scheme:Consider a dishonest prover that comes up with an accepting proof for the statement“the CRS is invalid”,or for a statement like“the CRS is invalid or S is true”where S is an arbitrary false statement.In real-life, where the CRS is guaranteed to be correct,this convinces the veriﬁer of the truth of the false statement S.However,such a prover is not ruled out by the UC security:the simulator given in[24]does generate an invalid CRS so that the statement in fact becomes true;and thus the proof can obviously be simulated in the ideal-world(when given a corresponding witness,which the simulator has in case of the UC GOS scheme).We stress that this is not aﬂaw in the UC GOS scheme but it is the UC security deﬁnition that does not provide any security guarantees for statements that incorporate the CRS,essentially because in the ideal-life model there is no (guaranteed-to-be-correct)CRS.1In conclusion,UC NIZK security provides good enough security under the condition that the statements to be proven do not incorporate the CRS.This is automatically guaranteed in a UC setting,where the statements to be proven must make sense in the ideal-world model,but not necessarily in other settings.2Preliminaries2.1NotationWe consider uniform probabilistic algorithms(i.e.Turing machines)which take as input(the unary encoding of)a security parameterκ∈N and possibly other inputs and run in deterministic poly-time inκ.We thus always implicitly require the size of the input to be bounded by some polynomial inκ.Adversarial behavior is modeled by non-uniform poly-time probabilistic algorithms,i.e.,by algorithms which together with the security parameterκalso get some(poly-size)auxiliary input order to simplify notation,we usually leave the dependency onκ(and on auxκ)implicit.By y←A(x),we mean that algorithm A is executed(with a randomly sampled random tape)on input x(and the security parameterκand,in the non-uniform case,auxκ) and the output is assigned to y.We may also denote it as y←A(x;r)when the randomness r is to be explicitly noted.Similarly,for anyﬁnite set S,we use the notation y←S to denote that y is sampled uniformly from S,and y←x means that the value x is assigned to y.For two algorithms A and B,we write B◦A for the composed execution of A and B,where A’s output is given to B as input.Similarly,A B denotes the joint execution A and B on the same input and the same random tape,and we write(x;y)←(A B)(w)to express that in the joint execution on input w(and the same random tape)A’s output is assigned to x and B’s to y.Furthermore,P y=A(x) denotes the probability(taken over the uniformly distributed random tape)that A outputs y on input x,and we write P x←B:A(x)=y for the(average) probability that A outputs y on input x when x is output by B:P x←B:A(x)=y = x P y=A(x) P x=B .We also use natural self-explanatory extensions of this notion.An oracle algorithm A is an algorithm in the above sense connected to an oracle in that it can write on its own tape an input for the oracle and tell the oracle to execute,and then,in a single step,the oracle processes its input in a prescribed way,and writes its output to the tape. We write A O when we consider A to be connected to the particular oracle O.A valueν(κ)∈R,which depends on the security parameterκ,is called negligible,denoted by ν(κ)≤negl(κ)orν≤negl,if∀c>0∃κ◦∈N∀κ≥κ◦:ν(κ)<1/κc.Furthermore,ν(κ)∈R is called noticeable if∃c>0,κ◦∈N∀κ≥κ◦:ν(κ)≥1/κc.2.2DeﬁnitionLet L⊆{0,1}∗be an NP-language.Deﬁnition1.Consider poly-time algorithms G,P and V of the following form:G takes the security parameterκ(implicitly treated hereafter)and outputs a common reference string(CRS)Σtogether with a trapdoorτ.P takes as input a CRSΣand an instance x∈L together with an NP-witness w and outputs a proofπ.V takes as input a CRSΣ,an instance x and a proof πand outputs1or0.The triple(G,P,V)is a statistical/perfect NIZK argument for L if the following properties hold.Completeness:For any x∈L with corresponding NP-witness wP (Σ,τ)←G,π←P(Σ,x,w):V(Σ,x,π)=0 ≤negl. Soundness:For any non-uniform poly-time adversary P∗P (Σ,τ)←G,(x∗,π∗)←P∗(Σ):x∗∈L∧V(Σ,x∗,π∗)=1 ≤negl.Statistical/Perfect Zero-Knowledge(ZK):There exists a poly-time simulator S such that for any x∈L with NP-witness w,and for(Σ,τ)←G,π←P(Σ,x,w)andπsim←S(Σ,τ,x), the joint distributions of(Σ,π)and(Σ,πsim)are statistically/perfectly close.Remark2.The notion of soundness we use here guarantees security against an adaptive at-tacker,which may choose the instance x∗depending on the CRS.We sometimes emphasize this issue by using the term adaptively-sound.Note that this is a strictly stronger notion than when the adversary must choose x∗independent of the CRS.Remark3.In the notion of ZK we use here,P and S use the same CRS string.In[15],this is called same-string ZK.In the context of statistical ZK,this notion is equivalent(and not only suﬃcient)to unbounded ZK,2which captures that the same CRS can be used an unboundednumber of times.This is obviously much more desirable compared to the original notion of NIZK, where every proof requires a fresh CRS.In[15],it is shown that there cannot be a same-string NIZK proof with statistical soundness for a NP-complete language unless there exist no one-way functions.This makes it even more interesting toﬁnd out whether there exists a same-string NIZK argument with statistical security on at least one side,namely the ZK side.2.3Bilinear Groups and the Hardness AssumptionsWe use the standard setting of bilinear groups.Let BGG be a bilinear-group generator that(takes as input the security parameterκand)outputs(G,H,q,g,e)where G and H is a pair of groups of prime order q,g is a generator of G,and e is a non-degenerate bilinear map e:G×G→H, meaning that e(g a,g b)=e(g,g)ab for any a,b∈Z q and e(g,g)=1H.We assume the Discrete-Log Assumption,dla,that for a random h∈G it is hard to compute w∈Z q with h=g w.In some cases,we also assume the Diﬃe-Hellman Inversion Assumption, dhia,which states that,for a random h=g w∈G,it is hard to compute g1/w.Formally, these assumptions for a bilinear-group generator BGG are stated as follows.In order to simplify notation,we abbreviate the output(G,H,q,g,e)of BGG by pub(for“public parameters”).Assumption4(dla).For every non-uniform poly-time algorithm AP pub←BGG,h←G,w←A(pub,h):g w=h ≤negl.Assumption5(dhia).For every non-uniform poly-time algorithm AP pub←BGG,h←G,g1/w←A(pub,h):g w=h ≤negl.Furthermore,we assume xkea,a variant of the Knowledge-of-Exponent Assumption kea, (referred to as kea3respectively kea1in[4]).kea informally states that givenˆg=g x∈G with unknown discrete-log x,the only way to eﬃciently come up with a pair A,ˆA∈G such thatˆA=A x for the same x is by choosing some a∈Z q and computing A=g a andˆA=ˆg a. xkea states that givenˆg=g x∈G as well as another pair h andˆh=h x with the same unknown discrete-log x,the only way to eﬃciently come up with a pair A,ˆA such thatˆA=A x is by choosing a,α∈Z q and computing A=g a hαandˆA=ˆg aˆhα.Formally,kea and xkea are phrased by assuming that for every algorithm which outputs A andˆA as required,there exists an extractor which outputs a(andαin case of xkea)when given the same input and randomness.Assumption6(kea).For every non-uniform poly-time algorithm A there exists a non-uniform poly-time algorithm X A,the extractor,such thatP pub←BGG,x←Z q,(A,ˆA;a)←(A X A)(pub,g x):ˆA=A x∧A=g a ≤negl. Recall that(A,ˆA;a)←(A X A)(pub,g x)means that A and X A are executed on the same input (pub,g x)and the same random tape,and A outputs(A,ˆA)whereas X A outputs a. Assumption7(xkea).For every non-uniform poly-time algorithm A there exists a non-uniform poly-time algorithm X A,the extractor,such that:ˆA=A x∧A=g a hα ≤negl.P pub←BGG,x←Z q,h←G,(A,ˆA;a,α)←(A X A)(pub,g x,h,h x)It is well known that dla holds provably with respect to generic algorithms(see e.g.[32]), which operate on the group elements only by applying the group operations(multiplication and inversion),but do not make use of the speciﬁc representation of the group elements.It is not so hard to see that this result extends to groups G that come with a bilinear pairing e:G×G→H,i.e.,to generic algorithms that are additionally allowed to apply the pairing and the group operations in H.We prove in Section6that also kea and xkea hold with respect to generic algorithms.We would also like to point out that we only depend on xkea for“proof-technical”reasons: our perfect NIZK argument still appears to be secure even if xkea should turn out to be false (for the particular generator BGG used),but we cannot prove it anymore formally.This is in contrast to how kea and xkea are used in[25]respectively[4]for3-round ZK,where there seems to be no simulator anymore as soon as kea is false.3A Perfect NIZK Argument for SAT3.1Handling Multiplication GatesLet(G,H,q,g,e)be generated by BGG,as described in Section2.3above.Furthermore,let h=g w for a random w∈Z q which is unknown to anybody.Consider a prover who announces an arithmetic circuit over Z q and who wants to prove in NIZK that there is a satisfying input for it.Following a standard design principle,where the prover commits to every input value using Pedersen’s commitment scheme with“basis”g and h as well as to every intermediate value of the circuit when evaluating it on the considered input,the problem reduces to proving the consistency of the multiplication gates in NIZK(the addition gates come for free due to the homomorphic property of Pedersen’s commitment scheme).Concretely,though slightly informally,given commitments A=g a hα,B=g b hβand C= g c hγfor values a,b and c∈Z q,respectively,the prover needs to prove in NIZK that c=a·b. Note thate(A,B)=e(g a hα,g b hβ)=e(g,g)ab e(g,h)aβ+αb e(h,h)αβande(C,g)=e(g c hγ,g)=e(g,g)c e(g,h)γand hence,if indeed c=a·b,thene(A,B)/e(C,g)=e(g,h)aβ+αb−γe(h,h)αβ=e(g aβ+αb−γhαβ,h).(1) Say that,in order to prove that c=a·b,the prover announces P=g aβ+αb−γhαβand the veriﬁer accepts if and only if P is satisfying in thate(A,B)/e(C,g)=e(P,h).Then,by the above observations it is immediate that an honest veriﬁer accepts the correct proof of an honest prover.Also,it is quite obvious that a simulator which knows w can“enforce”c=a·b by“cheating”with the commitments,and thus perfectly simulate a satisfying P for the multiplication gate.Note that the simulator needs to know some opening of the commitments in order to simulate P;this though is good enough for our purpose.For completeness,though,we address this issue again in Section4and show a version which allows a full-ﬂedged simulation. Finally,it appears to be hard to come up with a satisfying P unless one can indeed open A,B and C to a,b and c such that c=a·b.Concretely,the following holds.Lemma8.Given openings of A,B and C to a,b and c,respectively,with c=a·b,and given an opening of a satisfying P,one can eﬃciently compute w.Proof.Let P=gρh̟be the given opening of P.Then,inheriting the notation from above, e(A,B)/e(C,g)=e(g a hα,g b hβ)/e(g c hγ,g)=e(g,g)ab−c e(g,h)aβ+αb−γe(h,h)αβ.ande(A,B)/e(C,g)=e(P,h)=e(gρh̟,h)=e(g,h)ρe(h,h)̟are two diﬀerent representations of the same element in H with respect to the“basis”e(g,g), e(g,h)=e(g,g)w,e(h,h)=e(g,g)w2.This allows to compute w by solving a quadratic equation in Z q.The need for an opening of P can be circumvented by basing security on dhia rather than dla as stated in the following lemma.Lemma9.Given openings of A,B and C to a,b and c,respectively,with c=a·b,and given a satisfying P,one can eﬃciently compute g1/w.Proof.For a satisfying P it holds thate(P,h)=e(A,B)/e(C,g)=e(g,g)ab−c e(g,h)aβ+bα−γe(h,h)αβand thus,when c=a·b as assumed,the following equalities follow one after the other.e(g,g)=e (P g−aβ−bα+γh−αβ)1/(ab−c),he(g1/w,g)=e (P g−aβ−bα+γh−αβ)1/(ab−c),gg1/w=(P g−aβ−bα+γh−αβ)1/(ab−c)It remains to argue that a(successful)prover can indeed open all the necessary commitments. This can be enforced as follows.Instead of committing to every value s by S=g s hσ,the prover has to commit to s by S=g s hσandˆS=ˆg sˆhσ,whereˆg=g x for a random x∈Z q andˆh=h x (with the same x).Note that the same randomnessσis used for computing S andˆS,such that ˆS=S x;this can be veriﬁed using the bilinear map:e(ˆS,g)=e(S,ˆg).xkea now guarantees that for every correct double commitment(S,ˆS)produced by the prover,he knows(respectively there exists an algorithm that outputs)s andσsuch that S=g s hσ.Based on the above observations,we construct and prove secure an adaptively-sound perfect NIZK argument for circuit-SAT in the next section.3.2The Perfect NIZK SchemeThe NIZK scheme for circuit-SAT is given in Figure1.Note that we assume an arithmetic circuit C over Z q(rather than a binary circuit),but of course it is standard to“emulate”a binary circuit by an arithmetic one.Theorem10.(G,P,V)from Fig.1is an adaptively-sound perfect NIZK argument for circuit-SAT,assuming xkea and dla.CRS Generator G`1κ´:G-1.(G,H,q,g,e)←BGG(1κ),w←Z q,ˆg←G,h←g w,ˆh←ˆg w,G-2.outputΣ←(G,H,q,g,h,ˆg,ˆh,e)andτ←w.Prover P`Σ,C,x=(x1,...,x n)´:pute commitments for every input value x i by X i=g x i hξi andˆX i=ˆg x iˆhξi.P-2.Inductively,for every multiplication gate in C for which the two input values a and b are committed upon(either directly or indirectly via the homomorphic property)by A=g a hαandˆA=ˆg aˆhαrespectively B=g b hβandˆB=ˆg bˆhβ,do the pute a(double)commitment C=g c hγandˆC=ˆg cˆhγfor the corresponding output value c=a·b,and compute the(double)commitment P=g aβ+αb−γhαβandˆP=ˆg aβ+αb−γˆhαβ.P-3.As proofπoutput all the commitments as well as the randomnessηfor the commitment Y=g C(x)hηfor the output value C(x)=1.Veriﬁer V`Σ,C,π´:Output1(i.e.“accept”)if all of the following holds,otherwise output0.V-1.Every double commitment(S;ˆS)satisﬁes e(ˆS,g)=e(S,ˆg).V-2.Every multiplication gate in C,with associated(double)commitments(A,ˆA),(B,ˆB),(C,ˆC)and(P,ˆP) for the two input values,the output value and the“multiplication proof”,satisﬁes e(A,B)/e(C,g)= e(P,h).V-3.The commitment Y for the output value satisﬁes Y=g1hη.Figure1:Perfect NIZK argument for circuit-SATpleteness is straightforward using observation(1).Also,perfect ZK is easy to see. Indeed,the simulator S can run P with a default input for x,say o=(0,...,0),and then simply open the commitment Y for the output value y=C(o)(which is likely to be diﬀerent from1) to1using the trapdoor w.Since Pedersen’s commitment scheme is perfectly hiding,and since P andˆP computed in step P-2.for every multiplication gate are uniquely determined by A,B, and C,it is clear that this simulation is perfectly indistinguishable from a real execution of P.It remains to argue soundness.Assume there exists a dishonest poly-time prover P∗,which on input the CRSΣoutputs a circuit C∗together with a proofπ∗such that with non-negligible probability,C∗is not satisﬁable but V(Σ,C∗,π∗)outputs1.By xkea,there exists a poly-time extractor X P∗such that when run on the same CRS and the same random tape as P∗,the extrac-tor X P∗outputs the opening information for all commitments in the proof with non-negligible probability.Concretely,for every multiplication gate and the corresponding commitments A, B,C and P,the extractor X P∗outputs a,α,b,β,c,γ,ρ,̟such that A=g a hα,B=g b hβ, C=g c hγand P=gρh̟.3If P∗succeeds in forging a proof for an unsatisﬁable circuit,then there obviously must be an inconsistent multiplication gate with inputs a and b and output c=a·b.(Note that since addition gates are processed using the homomorphic property,there cannot be an inconsistency in an addition gate.)But this contradicts dla by Lemma8.Remark11.The NIZK argument from Fig.1actually provides adaptive ZK,which is a stronger ﬂavor of ZK than guaranteed by Deﬁnition1.It guarantees that S cannot only perfectly simulate a proofπfor any circuit C,but when later given a satisfying input x for C,it can also provide。

Parallel Frequent Pattern Mining

PFP: Parallel FP-Growth for Query Recommendation
Google Beijing Research, Beijing, 100084, China
Haoyuan Li
Google Beijing Research, Beijing, 100084, China
பைடு நூலகம்
{ (f:3, c:3, a:3) } | m
Reduce inputs (conditional databases) key: value
Conditional FP−trees
p:
{fcam/fcam/cb}
{(c:3)} | p
abcflmo
fcabm
m: f c a b b: f c a a: f c c: f
m:
{fca/fca/fcab}
Map inputs (transactions) key="": value facdgimp
Sorted transactions (with infrequent items eliminated) fcamp
Map outputs (conditional transactions) key: value p: m: a: c: fcam fca fc f
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for proﬁt or commercial advantage and that copies bear this notice and the full citation on the ﬁrst page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior speciﬁc permission and/or a fee. ACM RS Copyright 200X ACM X-XXXXX-XX-X/XX/XX ...$5.00.

第四章生涯决策

第四章生涯决策
• 学习目标
了解金字塔模型的第二级水平---决策技能领域了解改善与生涯规划相关问题的解决和决策技能方面的信息
第一部分生涯的概念和应用
1．生涯规划导言．生涯规划导言
5．思考我的生涯决策．
2．了解自我．
4．生涯决策．
3．了解我的各种选择．
信息加工金字塔
执行加工领域
改进生涯决策
我正处在 CASVE 循环的哪个地方？
了解决策过程
改进生涯决策
改进决策制定技能
谢谢大家！
下周再见！
Self-Knowledge
Occupational Knowledge
Knowledge Domains
良好决策的重要性
个人决策的质量是评估生活有效性的指标之一关注过程,而非结果
良好决策的重要性
通识教育+CASVE循环
生涯决策者的类型
做出决策的
尚未决策的
生涯决策者的类型
无法决策的
决策风格
CIP – CASVE Cycle of Decision Making Skills
External or Internal Problem Signals Communication (Identifying a gap)
Execution (Forming means-ends strategies
Analysis (Interrelating problem components)
Valuing (Prioritizing Asis (Creating likely Alternatives)
CASVE循环循环
外部或内部的问题信号被问题淹没，被问题淹没，感觉非常不好 C－沟通－找出差距）（找出差距）

想要了解的事物英语作文

想要了解的事物英语作文Things I Yearn to Understand The world is an intricate tapestry woven with threads of knowledge, both known and unknown. While I find myself fascinated by the vast amount of information we’ve accumulated as a species, I am acutely aware of the vast, uncharted territories of understanding that lie before me. There are several key areas that spark a deep curiosity within me, areas I yearn to explore and grasp with greater clarity. Firstly, I am captivated by the complex workings of the human mind. The brain, a three-pound universe contained within our skulls, is a marvel of intricate networks and electrochemical signals that give rise to consciousness, emotion, and behavior. How do neurons fire in symphony to create our perceptions of the world? What are the mechanisms behind memory formation and retrieval? How does our unique blend of genetics and environment shape our personalities and predispositions? Unraveling the mysteries of the mind holds the key to understanding the very essence of what makes us human. The vast universe, with its swirling galaxies, enigmatic black holes, and the tantalizing possibility of life beyond Earth, also ignites my imagination. I long to understand the fundamental laws that govern the cosmos, from the delicate dance of subatomic particles to the majestic movements of celestial bodies. What is the true natureof dark matter and dark energy, the unseen forces shaping the universe's evolution? Are we alone in this vast cosmic expanse, or does life, in all its wondrous forms, exist elsewhere? The pursuit of answers to these questions is a quest to understand our place in the grand scheme of existence. Closer to home, the interconnected web of life on our planet fascinates me. The intricate ecosystems teeming with biodiversity, the delicate balance of predator and prey, theintricate cycles of energy and nutrients - these are all testament to the awe-inspiring power of evolution and adaptation. I yearn to understand the complex interactions within these ecosystems, the delicate balance that sustains them, and the impact of human activities on this delicate web. Understanding these complexities is crucial for our responsible stewardship of the planet and the preservation of its irreplaceable biodiversity. Furthermore, I am drawn to the intricacies of human history and its impact on our present reality. From the rise and fall of civilizations to the struggles for freedom and equality, historyoffers a lens through which we can examine the triumphs and failures of humankind.I crave a deeper understanding of the forces that have shaped our social,political, and economic systems, the ideologies that have fueled conflicts and cooperation, and the enduring legacies of past events. By studying history, wecan learn from our ancestors' mistakes and successes, equipping ourselves to navigate the challenges of the present and build a better future. The ever-evolving world of technology, with its rapid advancements in artificial intelligence, biotechnology, and space exploration, also holds a powerful allure.I am driven to understand the principles behind these innovations, their potential to address global challenges, and the ethical implications that accompany them. How can we harness the power of artificial intelligence for the betterment of society while mitigating potential risks? What are the ethical considerations surrounding genetic engineering and its impact on future generations? How can space exploration contribute to scientific advancements and inspire future generations? Exploring these frontiers of technology is essential for shaping a future where innovation serves humanity and the planet. Finally, I yearn to understand the very essence of creativity and its power to inspire, challenge, and transform. From the evocative brushstrokes of a painter to the soaring melodiesof a composer, creativity speaks a universal language that transcends cultural boundaries. What are the cognitive processes that underpin artistic expression? How does creativity foster innovation and problem-solving across disciplines? How can we nurture and cultivate our own creative potential to contribute to the world in meaningful ways? Understanding the nature of creativity is key to unlockingour own potential and enriching the human experience. In conclusion, the pursuit of knowledge is a lifelong journey, an insatiable thirst for understanding that fuels my curiosity and motivates my exploration. From the inner workings of the human mind to the vast expanses of the cosmos, from the intricate web of life on Earth to the enduring legacies of human history, from the frontiers of technology to the power of creative expression - these are the areas I yearn to understand with greater depth and clarity. This quest for knowledge is not merely an academic pursuit but a fundamental aspect of what makes us human - the desire to learn, grow, and contribute to the betterment of ourselves and the world around us.。

博士复试英文PPT

2. PTBP1 promotes migration and invasion of lung cancer cells
3. PTBP1 enhances exon11a skipping of Mena premRNA in lung cancer cells
Results
1. PTBP1 is highly expressed in lung adenocarcinoma (LUAD) tissues and 95-D cells and upregulation of PTBP1 is associated with EMT progress
2. PTBP1 promotes migration and invasion of lung cancer cells
Master Research
PTBP1 enhances exon11a skipping in Mena premRNA to promote migration and invasion in lung
carcinoma cells
Background Objectives Technology Methods Results Conclusions
5. PTBP1-mediated migration and invasion of 95-D cells are partially dependent on MenaINV
Results
2.1. Overexpressed PTBP1 promotes levels of EMT-related proteins in lung cancer cells
Technology Methods
Results
1. PTBP1 is highly expressed in lung adenocarcinoma (LUAD) tissues and 95-D cells and upregulation of PTBP1 is associated with EMT progress

TA-Lib函数对照

TA-Lib函数对照Overlap Studies 重叠研究指标BBANDS Bollinger Bands 布林带DEMA Double Exponential Moving Average 双指数移动平均线EMA Exponential Moving Average 指数移动平均线HT_TRENDLINE Hilbert Transform - Instantaneous Trendline 希尔伯特变换——瞬时趋势线KAMA Kaufman Adaptive Moving Average 考夫曼⾃适应移动平均线MA Moving average 移动平均线MAMA MESA Adaptive Moving Average 梅萨⾃适应移动平均线(MAVP Moving average with variable period 变化周期移动平均线MIDPOINT MidPoint over period 周期中点MIDPRICE Midpoint Price over period 周期中点价SAR Parabolic SAR 抛物线停损转向操作点指标SAREXT Parabolic SAR - ExtendedSMA Simple Moving Average 简单移动平均线T3 Triple Exponential Moving Average (T3) 三重指数移动平均线(T3)TEMA Triple Exponential Moving Average三重指数移动平均线TRIMA Triangular Moving Average 三⾓移动平均线WMA Weighted Moving Average 加权移动平均Momentum Indicators 动量指标ADX Average Directional Movement Index 平均趋向指标ADXR Average Directional Movement Index Rating 平均趋向指标评估APO Absolute Price Oscillator 绝对价格振荡器AROON Aroon 阿隆指标AROONOSC Aroon Oscillator 阿隆震荡线BOP Balance Of Power 能量均衡CCI Commodity Channel Index 顺势指标CMO Chande Momentum Oscillator 钱德动量摆动指标DX/DMI Directional Movement Index 动向指标MACD Moving Average Convergence/Divergence 指数平滑移动平均线MACDEXT MACD with controllable MA type 可控MA类型的MACDMACDFIX Moving Average Convergence/Divergence Fix 12/26 MACD固定值12/26MFI Money Flow Index 资⾦流量指标MINUS_DI Minus Directional Indicator 负⽅向指⽰器MINUS_DM Minus Directional Movement 负⽅向移动MOM Momentum 动量PLUS_DI Plus Directional Indicator 正⽅向指⽰器PLUS_DM Plus Directional Movement 正⽅向移动PPO Percentage Price Oscillator 价格震荡百分⽐指数ROC Rate of change : ((price/prevPrice)-1)*100 变动率指标ROCP Rate of change Percentage: (price-prevPrice)/prevPrice 变动率百分⽐ROCR Rate of change ratio: (price/prevPrice) 变动率⽐例ROCR100 Rate of change ratio 100 scale: (price/prevPrice)*100 变动率⽐例 100刻度RSI Relative Strength Index 相对强弱指标STOCH Stochastic 随机指标STOCHF Stochastic Fast 随机快指标STOCHRSI Stochastic Relative Strength Index 随机相对强弱指标TRIX 1-day Rate-Of-Change (ROC) of a Triple Smooth EMA 三重平滑EMA的1⽇变化率ULTOSC Ultimate Oscillator 终极指标WILLR Williams' %R 威廉指标Volume Indicators Volume Indicators 成交量指标AD Chaikin A/D(Accumulation/Distribution) Line 累积/派发线ADOSC Chaikin A/D Oscillator 累积/派发摆动指标OBV On Balance Volume能量潮Volatility Indicators 波动率指标ATR Average True Range 平均真实波幅指标NATR Normalized Average True Range 归⼀化平均真实波幅指标TRANGE True Range 真实波幅指标Price Transform 价格变换AVGPRICE Average Price 平均价格函数MEDPRICE Median Price 中位数价格TYPPRICE Typical Price 代表性价格WCLPRICE Weighted Close Price 加权收盘价Cycle Indicators 周期指标HT_DCPERIOD Hilbert Transform - Dominant Cycle Period 希尔伯特变换-主导周期HT_DCPHASE Hilbert Transform - Dominant Cycle Phase 希尔伯特变换-主循环阶段HT_PHASOR Hilbert Transform - Phasor Components 希尔伯特变换相量分量HT_SINE Hilbert Transform - SineWave 希尔伯特变换-正弦波HT_TRENDMODE Hilbert Transform - Trend vs Cycle Mode 希尔伯特变换-趋势与周期模式Pattern Recognition 模式识别CDL2CROWS Two Crows 两只乌鸦CDL3BLACKCROWS Three Black Crows 三只乌鸦CDL3INSIDE Three Inside Up/Down 三内部上涨和下跌CDL3LINESTRIKE Three-Line Strike 三线打击CDL3OUTSIDE Three Outside Up/Down 三外部上涨和下跌CDL3STARSINSOUTH Three Stars In The South 南⽅三星CDL3WHITESOLDIERS Three Advancing White Soldiers 三个⽩兵CDLABANDONEDBABY Abandoned Baby 弃婴CDLADVANCEBLOCK Advance Block ⼤敌当前CDLBELTHOLD Belt-hold 捉腰带线CDLBREAKAWAY Breakaway 脱离CDLCLOSINGMARUBOZU Closing Marubozu 收盘缺影线CDLCONCEALBABYSWALL Concealing Baby Swallow 藏婴吞没CDLCOUNTERATTACK Counterattack 反击线CDLDARKCLOUDCOVER Dark Cloud Cover 乌云压顶CDLDOJI Doji ⼗字CDLDOJISTAR Doji Star ⼗字星CDLDRAGONFLYDOJI Dragonfly Doji 蜻蜓⼗字/T形⼗字CDLENGULFING Engulfing Pattern 吞噬模式CDLEVENINGDOJISTAR Evening Doji Star ⼗字暮星CDLEVENINGSTAR Evening Star 暮星CDLGAPSIDESIDEWHITE Up/Down-gap side-by-side white lines 向上/下跳空并列阳线CDLGRAVESTONEDOJI Gravestone Doji 墓碑⼗字/倒T⼗字CDLHAMMER Hammer 锤头CDLHANGINGMAN Hanging Man 上吊线CDLHARAMI Harami Pattern 母⼦线CDLHARAMICROSS Harami Cross Pattern ⼗字孕线CDLHIGHWAVE High-Wave Candle 风⾼浪⼤线CDLHIKKAKE Hikkake Pattern 陷阱CDLHIKKAKEMOD Modified Hikkake Pattern 修正陷阱CDLHOMINGPIGEON Homing Pigeon 家鸽CDLIDENTICAL3CROWS Identical Three Crows 三胞胎乌鸦CDLINNECK In-Neck Pattern 颈内线CDLINVERTEDHAMMER Inverted Hammer 倒锤头CDLKICKING Kicking 反冲形态CDLKICKINGBYLENGTH Kicking - bull/bear determined by the longer marubozu 由较长缺影线决定的反冲形态CDLLADDERBOTTOM Ladder Bottom 梯底CDLLONGLEGGEDDOJI Long Legged Doji 长脚⼗字CDLLONGLINE Long Line Candle 长蜡烛CDLMARUBOZU Marubozu 光头光脚/缺影线CDLMATCHINGLOW Matching Low 相同低价CDLMATHOLD Mat Hold 铺垫CDLMORNINGDOJISTAR Morning Doji Star ⼗字晨星CDLMORNINGSTAR Morning Star 晨星CDLONNECK On-Neck Pattern 颈上线CDLPIERCING Piercing Pattern 刺透形态CDLRICKSHAWMAN Rickshaw Man 黄包车夫CDLRISEFALL3METHODS Rising/Falling Three Methods 上升/下降三法CDLSEPARATINGLINES Separating Lines 分离线CDLSHOOTINGSTAR Shooting Star 射击之星CDLSHORTLINE Short Line Candle 短蜡烛CDLSPINNINGTOP Spinning Top 纺锤CDLSTALLEDPATTERN Stalled Pattern 停顿形态CDLSTICKSANDWICH Stick Sandwich 条形三明治CDLTAKURI Takuri (Dragonfly Doji with very long lower shadow) 蜻蜓⼗字下影线长CDLTASUKIGAP Tasuki Gap 跳空并列阴阳线CDLTHRUSTING Thrusting Pattern 插⼊CDLTRISTAR Tristar Pattern 三星CDLUNIQUE3RIVER Unique 3 River 奇特三河床CDLUPSIDEGAP2CROWS Upside Gap Two Crows 向上跳空的两只乌鸦CDLXSIDEGAP3METHODS Upside/Downside Gap Three Methods 上升/下降跳空三法Statistic Functions 统计功能BETA BetaCORREL Pearson's Correlation Coefficient (r)⽪尔逊相关系数LINEARREG Linear Regression 线性回归LINEARREG_ANGLE Linear Regression Angle 线性回归⾓LINEARREG_INTERCEPT Linear Regression Intercept 线性回归截距LINEARREG_SLOPE Linear Regression Slope 线性回归斜率STDDEV Standard Deviation 标准偏差TSF Time Series Forecast 时间序列预测VAR Variance ⽅差Math Transform 数学变换ACOS Vector Trigonometric ACos 反余弦函数ASIN Vector Trigonometric ASin 反正弦函数ATAN Vector Trigonometric ATan 反正切CEIL Vector Ceil 向上取整数COS Vector Trigonometric Cos 余弦函数COSH Vector Trigonometric Cosh 双曲正弦函数EXP Vector Arithmetic Exp 指数曲线FLOOR Vector Floor 向下取整数LN Vector Log Natural ⾃然对数LOG10 Vector Log10 对数函数logSIN Vector Trigonometric Sin 正弦函数SINH Vector Trigonometric Sinh 双曲正弦函数SQRT Vector Square Root ⾮负实数的平⽅根TAN Vector Trigonometric Tan 正切函数TANH Vector Trigonometric Tanh 双曲正切函数Math Operators 数学操作ADD Vector Arithmetic Add 向量加法运算DIV Vector Arithmetic Div 向量除法运算MAX Highest value over a specified period 周期内最⼤值（未满⾜周期返回nan）MAXINDEX Index of highest value over a specified period 周期内最⼤值的索引MIN Lowest value over a specified period 周期内最⼩值MININDEX Index of lowest value over a specified period 周期内最⼩值的索引MINMAX Lowest and highest values over a specified period 周期内最⼩值和最⼤值MINMAXINDEX Indexes of lowest and highest values over a specified period 周期内最⼩值和最⼤值索引MULT Vector Arithmetic Mult 向量乘法运算SUB Vector Arithmetic Substraction 向量减法运算SUM Summation 周期内求和。

1、下载文档前请自行甄别文档内容的完整性，平台不提供额外的编辑、内容补充、找答案等附加服务。
2、"仅部分预览"的文档,不可在线预览部分如存在完整性等问题,可反馈申请退款(可完整预览的文档不适用该条件!)。
3、如文档侵犯您的权益，请联系客服反馈,我们会尽快为您处理(人工客服工作时间：9:00-18:30)。

1. PERMUTATIONSuppose n objects are to be ordered from 1st to nth, each order is called a permutation. Apply the multiplication principle to count the number of permutations of n objects, i.e. n(n-1)(n-2)(n-3)....(3)(2)(1), or n!, called n factorial.e.g. Suppose that 10 students are going on a bus trip, and each of the students will be assigned to one of the 10 available seats. What is the number of possible different seating arrangements of the students on the bus?Notice: n objects should be distinguishable and they are always ordered in a line. If the objects are not ordered in a line but in other shapes, such as a circle or a square, how to calculate the number of permutations?e.g. Five students are going to sit around a table, how many arrangement can there be? (If the relative position of two students is the same, then we view it as one arrangement.)formula: (n-1)!If there are some objects are exactly the same, the number of permutations should be calculated in another way.e.g. How many different five-letter words can be formed when all letters in the word ENTER are used each time.formula: n!/(number of repeated objects)!Suppose that k objects will be selected from a set of n objects, where k<=n, and the k objects will be placed in order from 1st to kth. The number of permutation is n(n-1)(n-2)....(n-k+1).e.g. How many different five-digit positive integers can be formed using the digits 1, 2, 3, 4, 5, 6, and 7 if none of the digits can occur more than once in the integer?2. CombinationGiven the five letters A, B, C, D, and E, determine the number of ways of selecting 3 of the 5 letters, but unlike before, you do not want to count different orders for the 3 letters. i.e.Note that n choose k is always equal to n choose n-ke.g. You should choose 3-person committee from a group of 9 students. How many ways are there to do this?The difference between permutation and combination is whether order is considered. If different order makes different arrangement, it is permutation, and if different order makes no difference, it is combination. Most problems require us to combine these two.e.g. Suppose you want to choose 5 members from a group of 8 to watering 5 different gardens. How many ways to do this?3. Permutation& combinationMultiplication Principle:If we need N steps to do a job and in the first step we have M1 different ways, second M2, third M3...., then we apply multiplication principle to count the total ways of doing this job, i.e. N=M1*M2*M3....*MnAddition Principle:If we have N different ways to do a job and in the first way we have M1 different ways, second M2, third M3..., then we apply addition principle to count the total ways of doing this job, i.e. N=M1+M2+M3...+MnNotice: In multiplication principle steps are interdependent, while in addition principle each way can independently make the job done.General method to solve permutation & combination problems:1. Understand what needs to be done2. Decide whether to take steps or divide into different ways, or both. When taking steps, how many steps are there; when dividing into different ways, how many ways in total.4. Make sure of the requirements in each step or way, permutation or combination, and calculate the number of arrangements in each step or each way.5. Count the total ways using multiplication principle or addition principle.3.1 Give priorities to special elements and placese.g. How many different 5-digit odd numbers can be formed when choosing from 0, 1, 2, 3, 4, 5 and each number can be used only once? (key: 288)3.2 Bundling Strategye.g. There are 7 students to stand in a straight line. If student A and B should always stand side by side, so should student D and G, how many ways to line those 7 students? (key: 480)Remember to make permutation between bundling elements3.3 Slot strategye.g. There is going to be 2 dances, 2 cross talks, and 1 solo in a party. If the 2 dances cannot be performed in a row, how many different program lists can we make for this party? (key: 72)Permutation first and then plug in the special elements3.4 More than one line problemse.g. Suppose you are going to arrange 8 students into two rows with four students in each row. Student A and B must be in the first row and Student C must in the second row. How many ways to arrange? (key:5760)Turn it into one straight line problem3.5 Mixture problemse.g. Suppose we need to put 5 different balls into 4 different boxes and each box should contain at least one ball. How many ways to make this done? (key: 240) First choose and then order3.6 Plank strategye.g. An elementary school has 10 athlete quota to be assigned to 7 classes. If every class should have at least one quota, how many ways are there to assign? (key: 84)Pay attention to the differences between 3.5 and 3.64. ProbabilityProbability is a way of describing uncertainty in numerical terms.Probability experiment (random experiment)The set of all possible outcomes of a random experiment is called the sample space, and any particular set of outcomes is called an event.P(E)=the number of outcomes in the event/the number of possible outcomes in the experimentGeneral facts about probability:●If an event E is certain to occur, then P(E)=1●If an even E is certain not to occur, then P(E)=0●If an even E is possible but not certain to occur, then 0<P(E)<1●The probability that an event E will not occur is equal to 1-P(E)●If E is an event, then the probability of E is the sum of the probabilities ofoutcomes in E.●The sum of the probabilities of all possible outcomes of an experiment is 1. If E and F are two events of an experiment, we consider two other events relatedto E and F.●The event that both E and F occur, that is, all outcomes in the set●The event that E or F, or both, occur, that is , all outcomes in the set●P(E or F)=P(E)+P(F)-P(both E and F)●P(E or F)=P(E)+p(F) if E and F are mutually exclusive●P(E and F)=P(E)P(F) if E and F are independent●the probability of one event occurring k times during n independentexperiments P(E)=e.g. A fair 6-sided die is rolled nine times and what's the probability that two of the nine are odd number?If a fair 6-sided die is rolled once, let E be the event of rolling a 3 and let F be the event of rolling an odd number. What is the probability of both E and F occurring?A 12-sided die, with faces numbered 1 to 12, is to be rolled once, and each of the12 possible outcomes is equally likely to occur. What is the probability of rollinga number that is either a multiple of 5 or an odd number?Consider an experiment with events A, B, and C for which P(A)=0.23, P(B)=0.40, and P(C)=0.85. Suppose that events A and B are mutually exclusive and events B and C are independent. What are the probabilities P(A or B) and P(B or C)?Suppose that there is a 6-sided die that is weighted in such a way that each time the die is rolled, the probabilities of rolling any of the numbers from 1 to 5 are all equal, but the probability of rolling a 6 is twice the probability of rolling a 1. When you roll the die once, the 6 outcomes are not equally likely. What are the probabilities of the 6 outcomes?Suppose that you roll the weighted 6-sided die from above example twice. What is the probability that the first roll will be an odd number and the second roll will be an even number?A box contains 5 orange disks, 4 red disks, and 1 blue disk. You are to select two disks at random and without replacement from the box. What is the probability that the first disk you select will be red and the second disk you select will be orange?There are going to be 5 different songs and 3 different dances in a particular party. How many ways to arrange the programs as follows?(1) three dances should be shown in a row(2) three dances should be separatedHow many ways to divide 6 different books into three piles?How many ways to assign 6 different books to 3 persons and each has exactly two books?How many different 5-digit even numbers can be formed from 0, 2, 3, 6, and 9? (Each number should be used only once.)A. 120B. 72C. 60D, 48E. 106Xavier, Yvonne, and Zelda each try independently to solve a problem. If their individual probabilities for success are 1/4, 1/2 and 5/8, respectively, what is the probability that Xavier and Yvonne, but not Zelda, will solve the problem?(A)11/8(B)7/8(C)9/64(D)5/64(E)3/64Among a group of 2,500 people, 35 percent invest inmunicipal bonds, 18 percent invest in oil stocks, and7 percent invest in both municipal bonds and oilstocks. If 1 person is to be randomly selected fromthe 2,500 people, what is the probabilitythat theperson selected will be one who invests in municipalbonds but NOT in oil stocks?A certain club has 10 members, including Harry. Oneof the 10 members is to be chosen at random to bethe president, one of the remaining 9 members is tobe chosen at random to be the secretary, and one ofthe remaining 8 members is to be chosen at randomto be the treasurer. What is the probability that Harrywill be either the memberchosento be the secretaryor the member chosen to be the treasurer?A={2, 3, 4, 5}, B={4, 5, 6, 7, 8}Two integers will be randomly selected from the setsabove, one integer from set Aand one integer fromset B. What is the probability that the sum of the twointegers will equal 9 ?。