神经网络学习文章

9.3 Neural NetworksNeural networks are an interesting topic for artificial intelligence researchers. Since they are related to man’s nervous system construction, the results of their investigations form a foundation on which hypotheses describing the working of a brain can be built [1].Neural Network ObjectivesAlthough various neural network models to a greater or a lesser extent remind one of the actual neural network, nevertheless they do differ from it (apart from cases where such models are constructed with the intention of an exact simulation). The purpose of the limits imposed is to simplify the reasoning, and to enable the executability of algorithms devised [2].Artificial neural networks enable the execution of certain intelligent operations, e.g. associations.The analysis of these networks also provides interesting conclusions on parallel processing. Neural cells are characterized by a very long response (reaction) time (i.e. they work slowly), but because of [3] the massively parallel processing man is capable of instant execution of tasks for which conventional sequential machines require significantly more time; for instance, image recognition. Investigations of such processes will eventually lead to the design of fast algorithms that use the parallelism of artificial neurons. Parallel processing is especially important in building search algorithms employing neural networks.In the last couple of years electronic cubes containing more and more densely packed neural networks have appeared.Neural Network StructureA neural network is a set of processing units (nodes) joined by links through which they communicate. Each unit is characterized by its activation state which changes in time. From the current unit's activation state a signal it sends into the network is calculated [4]. This signal is carried over the links to other nodes. During the transmission it may be weakened or strengthened, depending upon the link's characteristics. Signals reaching a unit from its neighbors are combined into an input signal, from which the next activation state of that unit is computed [5].Network definition comprises descriptions of the following elements:⏹set of nodes⏹links⏹rules for calculating the input signal⏹activation function⏹output signal functionWe should start the characterization of processing units (neurons) by first indicating the significance of each unit. In some of the systems, single nodes clearly represent defined objects, features or concepts. This representation method is called local. Its opposite is a distributed representation, where single units denote not whole concepts but rather their abstract parts which do not have a representation in the language in which the working of a network is interpreted [6]. Only subsets of the whole network form full concept descriptions.A processing unit executes a simple sequence of operations: receives signals from neighbors, computes its activation state, and sends that state into the network. Not all units have to have the same characteristics (activation function, input signal), although networksare usually homogeneous.From the standpoint of network topology, units are classified as input, output and hidden. Input units are those which receive signals directly from the environment in which the network operates. Output units send the signal outside the network, and hidden units do not communicate directly with the environment. Sets of input and output units may contain common elements.Let us assume that a network consists of N units. Each of these units we denote by ui(1≤ i ≤ N ). A unit ui’s activation state at time t is denoted by ai(t). The whole network activation state is described, then, by a vectora(t)= (a1(t), a2(t),… , an(t))For various network models various assumptions regarding the values that an activation state can take are made [7]. These values can be continuous or discrete. A continuous value set can be bounded or unbounded [8], i.e. an activation state can be expressed by any real number or a real number from a bounded interval, e.g. [0, 1].The most common case of a discrete set are binary values, e.g. 0,1 or -1,1. The 0 (-1) state means that a unit is inactive, and 1 that it is active. Another set of values is -1, 0, 1. There are also networks possible where these values are limited to a certain well defined set, for instance 0, 1, 2, 3, 4.Units are joined by links. A link transmits a signal from one unit to the other, either strengthening or weakening it [9]. Some of the links may be excitatory. In that case, the signal transmitted [10] may increase the activation state of the receiver. Other links may be inhibitory. The signal transmitted may then lower the receiver's activation state. The power of signal transformation by a link is called the weight of that link. A weight of a link joining units ui and uj is denoted by wij [11]. Excitatory links have a positive weight while the inhibitory ones have a negative one. The lack of a link is equivalent to a link of weight 0. A signal input to a link from a unit ui's side reaches a uj unit multiplied by wij [12]. A link schema of a neural network is represented as a square weight matrix W(N×N) containing all the links' weights.In this approach it is assumed that a receiving unit treats all the incoming links the same. If this assumption cannot be upheld, the links are divided into link types, and for each link type a separate weight matrix W is given. In this section we consider only the networks with homogeneous links. Weights associated with links may change during a network learning process.A pattern of links between units determines the network topology and its capabilities. Cycle-free networks work differently to cyclic networks. Networks with just one level of links between the input and output units are inferior to multi-level networks with respect to a set of representable (recognizable) functions. The biggest capacity for function representation have fully connected networks (each node linked with every other node). The network link model is chosen depending upon its intended use.A unit's input signal is calculated from the signals sent to it and from the incoming links’weights. A unit ui’s incoming input signal at time t, denoted by neti(t), is calculated as a weighed sum of signals sent.where ok(t) is an output signal of unit uk at time t. Globally, an input signal is represented b y a vector net(t): net(t)=wo(t), where o(t)=( o1(t), o2(t), …, oN(t)) is a vector of output signals at time t. The reasoning above is applicable to networks with homogeneous links. In other cases, the value of neti(t) would be calculated by a more complicated procedure. The method of input signal calculation for a given network is constant and the same for all units.Each unit's activation state changes in time. A new state is calculated from the previous state and an input signal ai(t+1)=F(ai(t), neti(t)), and is generally expressed by the formula a(t+1)=F(a(t), net(t)), where F is an activation rule.In the simplest of cases, F does not take into account the previous activation state, and is an identity ai(t+1) = neti(t). For binary activation states, F is often given as F(x)=0 if x<0, F(x)=1 if x≥0.This means that if an input signal exceeds some threshold value a unit becomes active. Otherwise it becomes passive (inactive).For continuous activation states sets F assumes a form of sigmoidal function close [13] to a threshold function.A popular class of activation functions are quasi-linear functions, i.e. non-decreasing functions of one argument —single type input signal’s:Sometimes differentiability of the quasi-linear function is assumed. In that case it is called a semi-linear function.In some of the models an activation function is stochastic. From the input signal the probability of taking on a set value is calculated.In most models activation functions do not change in time. Exceptions are thermodynamic models, where some of the function parameters are changed. It should be noted that even unchangeable stochastic functions can, for the same data, return different values.。