From Tom Thumb to the Dockers Some Experiments with Foraging Robots

From Tom Thumb to the Dockers: Some Experiments with Foraging RobotsAlexis Drogoul, Jacques Ferber LAFORIA, Boîte 169,Université Paris VI, 75252 PARIS CEDEX O5 FRANCE drogoul@laforia.ibp.fr, ferber@laforia.ibp.frAbstractIn this paper, we experiment, from the point of view of their efficiency, different implementations of the "explorer robots application". Three types of "Tom Thumb robots", whose behavior is based on the foraging behaviors of ants are proposed and their results are critically examined. We then introduce chain-making robots (the "dockers"), governed by local perceptions and interactions. This helps us to show that only a few changes in the robots' behavior may greatly improve the efficiency of the population.IntroductionIn the research conducted in the field of swarm intelligence, the "explorer robots application" appears to be one of the most common examples found to illustrate the capacity of a population of poorly intelligent creatures to handle with a global goal. In this case study, the goal is to make a team of robots find and collect samples in an unpredictable environment and take them back to a home base. These robots usually operate independently and behave in a quite simple way. They can sense the samples, the home base and some other environmental signs (possibly left by other robots) and, of course, pick up and put down samples. This application appears to be very popular because of its use in the three most influent fields of Artificial Life, namely the simulation of animal's behaviour, with the works of (Deneubourg & al. 1986), GA's and genetic programming (see e.g., (Koza 1990) with the Santa-Fe trail) and robotics, where it has been used to introduce the idea of using small cheap robots for exploring planets or dangerous locations (Brooks 1990). Furthermore, it is now widely accepted as the best illustration of "swarm intelligence", along with the collective sorting example (Deneubourg & al. 1991). Recent papers have explored the ability of these multirobot systems to have useful behaviors, but none has emphasized on their efficiency. Moreover, although all the individual mechanisms of the robots are accounted for, stillthere is no understanding and explanation of how the overall patterns of foraging results from the interactions between these mechanisms. Consequently, even if the general idea is appealing, it is still difficult to choose between all the proposed solutions when one wants to use them in a real application. That is why we are interested in the comparisons between these systems in terms of efficiency. We hope that such comparison would significantly advance the understanding of these systems and the design of new ones. In this paper, we then intend to review some of the frameworks used so far for the implementation of this example. All the choices will be discussed and their results compared with respect to the needs of robustness, adaptivity and efficiency required. We will also compare them to our new system of "chain-making" robots. The plan of the paper is as follows: in Section 1, we introduce the application and the framework shared by all the robots. We then provide in Section 2 experimental results obtained with non-social robots, namely robots that do not interact directly with each other. Section 3 presents our "chain-making" robots and contains some numerical results and a few comparisons with the previous ones. We close in Section 4 by comparing all the systems presented so far and discussing some perspectives in swarm intelligence.1General FrameworkThe first report about the "collecting samples" application has been made in (Steels 89). The key idea is to make a set of little robots collect rock samples in an unknown environment. In all the experiments we describe in the next sections, our world (see Fig. 1) will be constituted by a variable population of robots, a home base that emits a signal decreasing with the distance and three heaps of one hundred samples each. The distance between the heaps and the base is 40 meters and the speed of the robots is 1 meter/cycle. The robots can sense samples from a short distance (2 meters). The field created by the home's signal defines an area some 80 meters in diameter.RobotsSamplesHome Baseway the robots are implemented. However, they simplify the understanding and the comparison of their behaviors. An example of such a diagram is shown on Fig. 2. It represents the behavior of the silliest foraging robot one can imagine: it simply searches randomly for samples, pick up one when it finds some, come back to the base, lay the sample and then returns to its random search. Whatever their lack of ability, these silly robots will be used as a basis for the next generations of robots. As a matter of fact, they are provided with the necessary autonomy and basic behaviors needed for completing the global task. We will see in the next sections that minor transformations at the individual level may lead to very important changes at the population level and to the emergence of cooperating processes.80 meters2Tom Thumb RobotsFigure 1 - The case studyThe robots are simulated using the EthoModelling Framework previously described in (Drogoul & al. 1992). They are built out of simple primitive behaviors that basically include: - obstacle and other robots avoidance - following of the home's gradient field - picking up and laying samples - random move These primitive behaviors are combined into concurrent tasks (see e.g. Drogoul & Ferber 1992), each of them being triggered by an internal or environmental stimulus. For more convenience, the behaviors of the robots will be described in the next sections with diagrams that represent the decision-making process performed by them at each step. Note that these diagrams do not exactly correspond to theThis kind of robots was inspired by the foraging behavior of ants. The basic idea is to make the robots that have found samples put down "crumbs" on their way back from the samples to the base. It is assumed that the other robots are attracted by the crumbs when they move onto them and then able to locate more quickly the samples. Roughly similar robots can be found in several papers including, but not limited to (Steels 1989, Deneubourg 1990). We decided to call them Tom Thumb, because of the resemblance their behavior shows with that of Charles Perrault's character. To provide the previous silly robots with this behavior, we only have to change one node and add one action (as shown in light-grey on Fig. 3). What is interesting is that this simple change now enables the robots to cooperate, even if it is a kind of non-intentional cooperation. They do not interact directly with one another, but they do transmit information that allow the other ones to minimize their search.Silly RobotFOUND SAMPLES?Primitives ActionsYESPICK UP SAMPLES FOLLOW STIMULUS YESNO NO CARRYING SAMPLES ? YES SENSED STIMULUS? NORANDOM MOVEGO HOME NO BACK HOME? YES LAY SAMPLESFigure 2 - The behavioral diagram of a silly foraging robotTom Thumb RobotFOUND YES SAMPLES? NOChangesPICK UP SAMPLES FOLLOW STIMULUSNO SENSED CARRYING SAMPLES ? YES CRUMB OR STIMULUS?YES NO RANDOM MOVE LAY 1 CRUMBGO HOME NO BACK HOME? YESLAY SAMPLESFigure 3 - The behavioral diagram of the first Tom Thumb robotsLet us now introduce the experimental results obtained with these robots. The case study has been described in Section 1. We measure the efficiency of populations of Tom Thumb robots, by plotting the number of cycles needed to pick up all the samples against the number of robots in the population. As shown on the log plot on Fig. 4, the curve dramatically decreases in the first steps, downto a minimum of 1113 cycles (for 63 robots) and then regularly increases (up to a maximum, not shown here, of approximately 3500 cycles, where it stabilizes). What do these results mean ? First point, the increase in performance of the populations (from one to 63 robots) is clearly due to the crumb laying and following. This helps to focus the population on the task it has to accomplish. Yet, and that is the second point, the positive feedback suppliedby this behavior is not counterbalanced by any negative feedback that could depend on the amount of samples available on each site. That is, an empty heap of samples will continue to attract robots because the path between this heap and the base will remain stable. This explains the brutal variations of the curve, due to the random choice made by each robot between the three paths. If several robots chose a path leading to an empty heap, the overall efficiency of the population clearly decreases. We will see below that we need another factor to explain the decrease of efficiency observed for the most numerous populations. However, let us firstly try to implement a negative feedback at the population level by modifying the robots behaviors.Number of cycles100000Tom Thumb RobotsMinimum : 1113 cycles (63 robots) Average: 3351 cycles1000 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77 81 85 89 93 97 1 5 9Number of robotsFigure 4 - Results obtained with 100 populations of Tom Thumb robotsTom Thumb Robot IIFOUND SAMPLES? NO YESChangesPICK UP SAMPLES FOLLOW STIMULUS PICK UP 1 CRUMB RANDOM MOVE LAY 1 GO HOME CRUMB LAY SAMPLESNO SENSED CARRYING SAMPLES ? YES CRUMB OR STIMULUS?YES NONO BACK HOME? YESFigure 5 - The behavioral diagram of the second Tom Thumb robotsOnce again, the idea can be found in Charles Perrault's novel, where the path established by Tom Thumb is being destroyed by birds that pick up the crumbs. If we make robots pick up crumbs, too, when they are following a path, we enable them to build dynamical trails and to cancel them when they lead to an empty heap (as a matter of fact, a robot following the trail will erase it. If it does not find samples, it will not rebuild it by coming back to the base). This new behavior only induces a minor change in the Tom Thumb robots, as shown on Fig. 5. The experimental results obtained with the robots are plotted on Fig. 6. The dashed lines show the previous results and the solid lines the new ones. We see that the new behavior has suppressed the brutal variations of the curveand that the performance of a given population can be roughly predicted from the performance of the previous one. However, the evolution of this efficiency is not really satisfactory with respect to the previous results. For small populations (less than 60 robots), we have clearly lost in efficiency what we have gained in stability and predictability. Why is it so ? In fact, it seems that we have introduced too much dynamics in the system. Letting one robot destroy a path means that it hides a lot of information to the others during its trip, until it comes back to the base. On one hand, robots are no more attracted towards useless crumb paths, but on the other hand, they often temporarily lose already made paths.Number of cycles100000Tom Thumb Robots IIMinimum : 1607 cycles (85 robots) Average: 5315 cycles1000131721252933374145495357616569737781858993Number of robotsFigure 6 - Results obtained with 100 populations of Tom Thumb II robots97159Tom Thumb Robot IIIFOUND SAMPLES? NO YESChangesPICK UP SAMPLES FOLLOW STIMULUS PICK UP 1 CRUMB RANDOM MOVE LAY 2 CRUMBS LAY SAMPLESNO CARRYING SAMPLES ? YES SENSED CRUMB OR STIMULUS?YES NOGO HOME NO BACK HOME? YESFigure 7 - The behavioral diagram of the third Tom Thumb robotsSo the solution, already proposed in (Steels 1991), is to provide them with a mixed behavior. We need the robots to build paths that they can erase, but slower. Then, we just tell them to put down two crumbs when coming back to the base and to pick up one when following the path (see Fig. 7). The results, plotted on Fig. 8, clearly show two things. Firstly, the efficiency of this robot is much better than that of the previous one. Secondly, the curve remains very regular until the population reaches 85 robots. For most of the cases, this solution seems to be a good compromise between the two previous Tom Thumb robots, combining efficiency with regularity.Still a question has yet to be answered. Why is there an increase in the number of cycles in the last 15 results ? In fact, the answer is pretty simple. The 15 last populations of robots are facing important traffic jams. And, though it might be funny to imagine robots stuck in traffic jams, this raises a major problem. If we look more closely to the results, we can see that a population of 25 robots is as efficient as a population of 100 robots, and that a population of 60 robots is twice as efficient. So the cooperation, which emerges quite quickly for small populations, is counterbalanced in bigger populations by the competition between the robots.Number of cycles100000Tom Thumb Robots IIIMinimum : 1075 cycles (85 robots) Average: 3519 cycles1000159131721252933374145495357616569737781858993Number of robotsFigure 8 - Results obtained with 100 populations of Tom Thumb III robots97Fig 9a - Unorganized dockersFig 9b - Chain-making dockers3Chain-Making RobotsThe problem to solve is then to make the robots cooperate more deeply without competing too much. In fact, the previous robots were only cooperating by sharing information (namely the location of samples) but they were not able to cooperate by coordinating their actions. Let us take an example borrowed from everyday life. When a ship enters a harbor in order to be discharged, the dockers do not run all together towards the ship, competing for the access to the gangway (like in Fig. 9a). Instead, they organize themselves, making a chain from the ship to the wharves and thus minimizing their waste of energy and maximizing their efficacy (Fig. 9b). Why did we take this example? In fact, the first situation peculiarly looks like the situations created by our last robots: a huge crowd around the samples with everybody trying to escape the place to come back to the base. And, of course, we would evidently like to obtain something like the second situation. The differences between these two situations are quite obvious. In the first one, the dockers act as if they were alone, without paying attention to the others. In the second one, they coordinate their actions. They have then to be aware of the behaviors of other dockers. If we want this global behavior to emerge in our application, the challenge is to implement these relationships in robots that cannot communicate.In which way do we have to modify the behavior of Tom Thumb to let it become a docker? The first idea is to transform our robots in order to make them detect the samples carried by other robots (as if these samples were on the floor). A robot carrying a sample can, for example, switch on a light on its head. This signal will trigger in the other robots the same behavior than the samples detection. In that way, robots will be able to react upon the state of others. The second idea is to let the robots be able to pick up samples carried by others . Several solutions can be conceived and we will not get into further details about it. We just assume that they are provided with an arm and a skip and that their arm can pick up things from the skip of other robots (see Fig. 10).Figure 10 - A possible design for the chain-making robotsWith these two primitive actions, correctly arranged within the framework of the previous robots (see Fig. 11), we claim that it is possible to obtain chains from the samples to the base, whatever the number of robots.Docker RobotFOUND SAMPLES? NO FOLLOW NO CARRYING SAMPLES ? YES YESChangesPICK UP SAMPLES PICK UP 1 CRUMB RANDOM MOVE LAY 2 CRUMBS LAY SAMPLES SWITCH ON LIGHT SWITCH OFF LIGHTSTIMULUS/LIGHT SENSED CRUMB , LIGHT OR STIMULUS? GO HOME NO BACK HOME? YES YES NOFigure 11 - The behavioral diagram of the docker (or chain-making robots)Number of cyclesDocker Robots10000Minimum : 697 cycles (93 robots) Average: 1805 cycles100131721252933374145495357616569737781858993Number of robotsFigure 12 - Results obtained with 100 populations of docker (chain-making) robotsThis emergent functionality is illustrated on Fig. 13, which contains two snapshots of the simulation in progress. One can see, on the right-hand window, the three chains being build from the three heaps of samples to the base. The robots that make up these chains do not move all the way from the samples to the base, but only between one location where they pick up samples (a robot or a heap) and one location where they are discharged of them (by themselves in the nest or elsewhere by another robot). The consequence of this double cooperation (information sharing and action coordination) is a real improvement in the efficiency of the populations, from the first (one robot) to the last one (a hundred robots). The curve, plotted on Fig. 12, is very regular (at least with respect to the previous ones) and evolves towards the asymptotic direction Y = 700. In most cases, the results are twice as best as any of the results obtained with the former Tom Thumb robots. The explanation appears to be twofold: (1) From the robot's point of view, once it is involved in a chain, it has less distance to cover (to pick up samples and to put them down) than any of the Tom Thumbs robots. Those were actually obliged, at each time, to cover the whole distance between the nest and the heaps.In the best cases, the chain-making robots do not even need to move to get and give samples. This explains the very impressive efficiency observed in almost any of the populations. (2) This reduction of the number of moves appears to prevent the robots from going and disturbing the others, and thus avoids traffic jams around the nest and the samples. And when the number of robots is very important compared to the size of the environment, samples can of course be conveyed without difficulty, from robot to robot, without needing the robots that carry them to escape the jams. This is especially interesting for populations of great size, where the brutal decrease of performance does not arise as in the previous cases. Of course, one can argue that these chain-making robots are a little more complex than the Tom Thumb robots and perhaps, physically speaking, a little harder to build. So it could be difficult to rely on their behavior, which needs direct interactions between them. But the chain appears to be very robust and fault-tolerant. Even if some robots miss their goal, the chain will continue to work, evolving dynamically towards stable states.97159Number of cycles: 0Empty robot Robot carrying samplesNumber of cycles: 530BaseSamplesFigure 13 - Two snapshots of a simulation in progress4ConclusionWhat we have learned in these four experiments can be summarized in the two following points: (1) In these dynamical systems, small changes at the individual level can greatly modify the global behavior of the population. If we look at the three Tom Thumb robots, the differences between them are not really important (in term of capabilities). But the results at the population level are quite substantial. This implies to be very careful when designing a swarm intelligence based system or any distributed system in which intelligence is collectively exhibited by non-intelligent entities. The methodology we followed here is that of the "increasing complexity". The key idea is to firstly experiment solitary entities that do not have any social behavior and to progressively increase their capabilities in terms of communication and interaction. The first robots were only able to indicate something (namely the location of samples) to the others. The second generationadded the ability to manipulate this information, by simply cancelling it. The third one implemented a more advanced manipulation of this information, giving the robots the faculty to change it. And, finally, the chain-making robots were provided with direct interactions. A very interesting challenge would then be to supply these robots with means of (limited) communication in order to test the efficiency of the chain-making robots proposed by (Deneubourg & al. 92). (2) This application of chain-making robots is still on its early stages and it is somewhat difficult to foresee what is going to be done and experimented with it. However, some perspectives may be drawn. First, we are extending the application to cope with several bases that may move. It is easy to see the interest of such an application, for example in agriculture (where harvesting could be performed by dispatching many robots in the fields) or in the military domain (for managing the supply lines between different corps). Secondly, we are interested in the emergence of topological structures and the chains obtainedhere are nothing else but structures dynamically created that stay stable as long as the flow of energy that has created them (from the samples to the nest) remains. We hypothetize that more complex geometrical structures could be observed by multiplying the number of heaps and the number of bases. We hope to have soon results on this subject.AcknowledgementsWe wish to acknowledge the contribution of Steffen Lalande in programming most of the applications described here during his first year of doctorate. We are deeply grateful to him for his wise advices and stimulating discussions.BibliographyR. Brooks (1990) "Elephants Don't Play Chess" in "Journal of Robotics and Autonomous Systems", Volume 6, p. 3-15. J.L. Deneubourg, S. Aron, S. Goss, J . M . Pasteels, G. Duerinck (1986) "Random Behaviour, Amplification Processes and Number of Participants: Howthey Contribute to the Foraging Properties of Ants" in Physica 22D, North-Holland, Amsterdam, p. 176-186. J.L.Deneubourg, S.Goss, N.Franks, A.Sendova-Franks, C.Detrain, L.Chretien (1991) "The dynamics of collective sorting Robot-like Ants and Ant-like Robots" in "From Animals to Animats", MIT Press, p. 356. A. Drogoul, J. Ferber, B. Corbara, D . Fresneau (1992), "A Behavioral Simulation Model for the Study of Emergent Social Structures" in "Towards a Practice of Autonomous Systems, Proceedings of ECAL'91", MIT Press. A. Drogoul, J. Ferber (1992), "Multi-Agent Simulation as a Tool for Modeling Societies: Application to Social Differentiation in Ant Colonies", in Proceedings of MAAMAW'92 (forthcoming "Decentralized AI IV"). J.R. Koza (1990) "Evolution and Co-Evolution of Computer Programs to Control Independent-Acting Agents" in "From Animals to Animats", MIT Press. L. Steels (1989) "Cooperation between distributed agents Through self-organisation" in "Journal on robotics and autonomous systems", North Holland, Amsterdam. L. Steels (1991) "Towards a Theory of Emergent Functionality" in "From Animals to Animats", MIT Press, p. 451.。