A heuristic bidding strategy for multiple heterogeneous auctions

A Heuristic Bidding Strategy for Multiple Heterogeneous
Auctions
Patricia Anthony
Dept.of Electronics and Computer Science University of Southampton
Highfield,Southampton,SO171BJ,UK.
pa99r@
Nicholas R.Jennings
Dept.of Electronics and Computer Science University of Southampton
Highfield,Southampton,SO171BJ,UK.
nrj@
ABSTRACT
Online auctions are increasingly being used as a medium to procure goods and services.As the number of auction sites increases,however,consumers will inevitably want to track and bid in multiple auctions(with multiple protocols)in or-der to get the best deal for their desired goods.To this end, this paper reports on the development of a heuristic decision making framework that an autonomous agent can exploit to tackle the problem of bidding across multiple heterogeneous auctions.The framework enables the agent to adopt varying tactics and strategies that attempt to ensure that the user’s objectives are satisfied.Through empirical evaluation,the agent’s performance is shown to be effective even when there are multiple such agents in the environment at the same time and when the agent cannot accurately determine the type of environment that it is situated in.
Keywords
multiple auctions,bidding strategy,intelligent agents.
1.INTRODUCTION
Online auctions are one of the most popular and effective ways of trading goods over the Internet;indeed there are more than2500sites that operate online auctions1.These auction houses conduct many different types of auctions ac-cording to a variety of protocols including English,Dutch,first-price sealed bid and second-price sealed bid(Vickrey). Moreover,the auctions operate with varying start and end times and many of them are likely to be active at any one moment in time.For these reasons,consumers are faced with a difficult task when it comes to selecting and bidding in these auctions.In order to purchase a particular item, they need to monitor multiple auctions at multiple sites be-fore selecting the one to participate in.Even after making this selection,they still have to decide on the amount to bid in order to get the desired item under conditions that are consistent with their preferences.
To assist users,many sites now offer simple bidding proxies that will bid in a stated auction up to some predefined limit (e.g.eBay and Yahoo!Auction).However,such systems typically have the problem of only being able to operate 1 at a single auction site or only with a particular auction ers can also utilise the services of auction search engines(such as BidXS2and AuctionBeagle3).However, while these facilities allow users to monitor the progress of multiple concurrent auctions,all the bidding decisions are left to the human users.
To address these shortcomings,we believe it is necessary to develop an autonomous agent that can participate in multi-ple heterogeneous auctions,that is empowered with trading capabilities,and that can make purchases autonomously[3, 1].In more detail,the agent should monitor and collect information from the ongoing auctions,make decisions on behalf of the consumer and endeavour to guarantee the de-livery of the item.The agent must ensure that it never bids above the private valuation(the maximum amount that the consumer is willing to pay)and it should try to get the item in a manner that is consistent with the consumer’s prefer-ences(e.g.at the earliest time,at the lowest price,or with maximum chance of succeeding).
To this end,this paper reports on our work in developing such a bidding agent.The agent has a range of strategies that it can employ depending on the user’s aims and the environment in which the agentfinds itself.Here,we con-sider three different types of protocol:English,Dutch and Vickrey4The strategies themselves are heuristic in nature because the multiple heterogeneous auctions environment is highly complex,dynamic and unpredictable,making it im-possible tofind an optimal strategy that can be used in practical contexts[5].Moreover,our early investigations showed that the effectiveness of the strategies was heavily influenced by the nature of the environment[3].For this reason,we decided to have different strategies for different circumstances.As the range of potential strategies is huge, we decided to use a genetic algorithm(GA)to search for effective strategies for each of the various environments that we identified[2].We chose this particular method because GAs have been shown to perform well in areas where the search space is large and not well understood[9].Having evolved the strategies(offline),the agent adopts the one that is most appropriate to its prevailing context(online). 2
3
4Thefirst-price sealed bid is not considered here because of its similarities to the Dutch auction[8].
Against this background,this paper advances to the state of the art in the following ways.Firstly,we extend the work presented in[2]to evolve the complete set of strate-gies for our trading scenarios(in[2]we merely described the method and showed that it was successful in a very limited range of environments).These strategies consist of multi-ple evolved sub-behaviours that are appropriate in different environment settings with different objectives.This collec-tion of strategies for a single agent is termed the intelligent bidding strategy in the remainder of this paper.Secondly, we evaluate the intelligent bidding strategy and show that it is effective across a broad range of scenarios(including those where the agent cannot accurately assess the type of environment it is situated in).Thirdly,given the success of our bidding strategy,we believe that it may be widely adopted.Thus,we present results about what happened to the performance of the buyers and sellers and the market ef-ficiency when multiple agents adopt this strategy in a given marketplace.
The remainder of this paper is structured in the following manner.In the next section,we briefly outline the design of our intelligent bidding strategy.Section3reports on the evaluation of the strategy in various settings.Section4dis-cusses related work andfinally Section5concludes.
2.DESIGNING THE BIDDING STRATEGY Before describing the decision-making framework,it is nec-essary to detail our assumptions about the environment. Firstly,we consider three auction protocols:English,Dutch and Vickrey(three of the most common types).Secondly, all auctions have a known start time and English and Vick-rey auctions have a known end time.Thirdly,our bidding agent is given a hard deadline(t max)by when it must obtain the desired item.It is also told the consumer’s private valu-ation(p r)for this item and the consumer’s intention(either looking for a bargain,desperate for the item or a combina-tion of the two).Finally,the agent must not buy more than one instance of the desired item.We use a simulated elec-tronic marketplace to evaluate our bidding strategies(see[3, 1]for more details).This marketplace consists of a number of English,Dutch and Vickrey auctions that run concur-rently.There are several bidders participating in each auc-tion.They operate in a single auction,have the intention of buying the target item and possessing certain behaviour based on the type of auction that they are participating in. These bidders’strategies are based on the dominant strate-gies of the respective one shot single auctions[8].
2.1The Bidding Strategy Framework
The agent’s decision making model works in the following manner(as summarised in Figure1).The bidder agent builds an active auction list(auctions that have started but not reached their end times,denoted L(t))and gathers rel-evant information(e.g.start and end times,current bid values)about them.It then calculates the current maxi-mum bid it is willing to make at the current time.The current maximum bid,by definition,will always be less than or equal to the private valuation.To determine the current maximum bid,the agent considers several bidding constraints including the remaining time left,the remaining auctions left,the user’s desire for a bargain and the user’s level of desperateness.For each such bidding constraint,
Figure1:Bidding Agent’s Top-Level Algorithm there is a corresponding function that suggests the value to bid based on the constraint at that time.These functions (based on[6])are parameterized by two key values:k(range [0..1])is a constant that determines the value of the start-ing bid andβ(range[0.005..1000])defines the shape of the curve(and so the rate of concession to p r).As an example, when k≥0.5andβ≥1,the agent demonstrates a rea-sonable degree of desperateness and starts bidding close to p r and quickly reaches p r.At the other extreme,the agent can demonstrate hard bargaining behaviour(k<0.5and β<1),where it makes a low initial bid and only concedes in a very slow fashion.All behaviours in between are also possible by setting the parameters appropriately.At any given time,the agent may consider any of the bidding con-straints individually or it may combine them depending on the situation(what the agent sees as being important at that time).If the agent combines multiple bidding constraints, it allocates a weight to each of them to denote their relative importance.The set of functions is referred to as the tactics and the combination of these tactics are referred to as the strategy.
Figures2and3illustrate how these bidding constraints can be combined to produce a single bid value for a given time.In particular,Figure2shows the behaviour of each bidding constraint.This behaviour suggests how the agent should behave when only a single constraint in considered. For example,if the agent considers desire for a bargain as the only bidding constraint,the it should start bidding at a low bid value and should slowly move toward its private valuation.On the other hand,if the agent wishes to consider the remaining time left as the single criterion,then it should start bidding at a reasonably high value and slowly reach its private valuation.Figure3show various combinations of the bidding constraints using weighted combinations.In this case,(0.25,0.25.0.25,0.25)indicates that equal weighting has been applied to all the bidding constraints,(0.50,0.00, 0.50,0.00)indicates that the agent only considers bidding constraints of the remaining time left and the desire for a bargain and places equal weight of0.50for both bidding constraints.It can also be seen that by varying the weights of the bidding constraints,different bidding patterns can be generated as shown in Figure3.Based on the value of the current maximum bid,the agent selects the potential auctions in which it can bid and calculates what it should bid at this time in each such auction.The auction and the corresponding bid with the highest expected utility is then selected from the potential auctions as the target auction.
Figure2:The Bidding Constraints’Behaviour Finally the agent bids in the target auction.This process is repeated until the item is acquired or until the given time is
reached.
Figure3:Various Combinations of the Bidding Con-straints
Our early experiments in this domain led to several conclu-sions[3].Firstly,p r is one of the most important factors that needs to be considered when determining the strategy that should be employed by the agent.For example if p r is low,the probability of winning the auction is low and the agent is unlikely to be able to get a bargain,whereas if p r is high,the agent has a better chance of obtaining the item with a bargain.Secondly,the strategies to be used by the agent need to be tailored to the prevailing context, since not all strategies work well in all situations.Thus, a successful strategy in one situation may perform badly in another.Nevertheless,it was possible to determine that certain classes of strategy are effective in environments that have particular characteristics.In this case,the key defin-ing characteristics of an environment were found to be the number of auctions that are active before t max.Given this, it was decided to evolve strategies that are effective for en-vironments classified in this way.
2.2Evolving Strategies
The performance of the bidding agent is heavily influenced by the strategy employed,which,in turn,relates to the val-ues of k andβin a given tactic and the weights for each tactic when they are combined.The number of strategies that can
STLA STLA STLA
STMA STMA STMA
MTLA MTLA MTLA
MTMA MTMA MTMA
LTLA LTLA LTLA
LTMA LTMA LTMA
STLA STLA STLA
STMA STMA STMA
MTLA MTLA MTLA
MTMA MTMA MTMA
LTLA LTLA LTLA
LTMA LTMA LTMA
STLA STLA STLA
STMA STMA STMA
MTLA MTLA MTLA
MTMA MTMA MTMA
LTLA LTLA LTLA
LTMA LTMA LTMA
RP1: Low Private Valuation STLA: Short Time Less Auctions
RP2: Medium Private Valuation STMA: Short Time Many Auctions
RP3: High Private Valuation MTLA: Medium Time Less Auctions FE1: Fitness Equation I MTMA: Medium Time Many Auctions FE2: Fitness Equation II LTLA: Long Time Less Auctions
FE3: Fitness Equation III LTMA: Long Time Many Auctions
RP3
FE1
FE2
FE3
RP2
FE1
FE2
FE3
RP1
FE1
FE2
FE3
Table1:The Environments
be employed is infinite,so,therefore,is the search space. Here,we decided to use GAs to search offline for the most successful strategies in the predefined environments(see[2] for more details).The individuals in the populations are the bidding agents and their genes consist of the parameters for the four different tactics and the relative weight for each of them.To compute thefitness function,we consider three plausible alternatives.These are the individual success rate in obtaining the item(FE1)and two variations based on the average utility.In thefirst case(FE2),the agent gets a utility of0it if fails to obtain the item.Thefinal utility function(FE3)is similar to FE2but the individual is pe-nalised if it fails to get the item.FE1is used if delivery of the item is of utmost important,FE2if the agent is looking for a bargain,and FE3if delivery of the item and looking for a bargain are equally important.
In[2]we showed,through empirical evaluation,that suc-cessful strategies with the obvious behaviours(as dictated by the variousfitness equations),could be evolved for a very limited set of environments.The next step,and the main contribution of this paper,is to complete the evolutionary process for all of the target environments and to show that the ensuing agent,composed of the evolved sub-behaviours, would actually perform effectively across a wide range of environments.
2.3The Intelligent Bidding Strategy
Having shown that GAs can effectively evolve strategies for different environments,thefinal step is to combine this knowledge into a single intelligent bidding agent.The agent has at its disposal knowledge about which strategies are ef-fective in which environments and assuming it can assess the environment accurately,it simply has to deploy the ap-propriate strategy.
In more detail,we subdivided our trading environments into 54categories(shown in Table1)based on the key determi-nants that influenced the agent’s bidding decision(namely, the private valuation,the remaining time left,the remaining auctions left and the agent’s behaviour(whether looking for a bargain,desperate or a mixture of both)).This categorisa-tion of the classes of environment is important because it en-ables the agent to tune its bidding strategy to its prevailing circumstances.The agent’s private valuation can be broadly categorised by its value:low(RP1),medium(RP2)or high (RP3).These three categories are then refined further into three sub-categories based on the agent’s behaviour(FE1 is desperate,FE2is looking for a bargain and FE3aims for a balance of both).Finally,each sub-categorisation is further divided into groupings that consist of environments in which there are short time less auctions(STLA),short time many auctions(STMA),medium time less auctions (MTLA),medium time many auctions(MTMA),long time less auctions(LTLA)and long time many auctions(LTMA) (see[3]for more details).As an example,RP1FE2STMA represents a strategy that was evolved in an environment with a low private valuation,where the bidding time is short, where there are many auctions and where the consumer is interested in a bargain.
The categorisation of the private valuation is made based on the auction closing price distribution(here,the closing price mean is76and the standard deviation is5).Thus, 50%of the auctions should be won by bidders with medium private valuations,25%by bidders with low private valua-tions and the remaining25%by bidders with high private valuations.In real market settings,the price of each de-sired item will naturally vary depending on the type of the item itself(e.g.a diamond ring usually costs more than a book).However,the value of a given item can be directly mapped to the reservation price categorisation merely by obtaining the mean price of the item.This can be achieved using comparison price data from sites such as PriceSCAN5, DealTime6,and BottomDollar7.From this,the agent can calculate the mean price of a given item and regenerate price ranges for the low,medium and high private valuations.The subdivisions of the short time,medium time,long time,less auctions and many auctions can also be carried out in a similar manner.
We then used the search algorithm defined in Section2.2to evolve the best strategy for each environment.Thus,the agent gets the user’s private valuation,the item to be pur-chased,when it is required and the intention of the user (either looking for a bargain,desperate or some combina-tion of the two).With this knowledge,the agent enters the marketplace and determines the number of active auctions in which it can participate within the given time constraint. Based on this combination of information,the agent deter-mines which strategy to use in each auction round.This decision is captured in a rule base that maps the prevailing context to the strategy that has been evolved for that situ-ation.Upon selection of the appropriate strategy the agent proceeds as defined in Figure1.
5/
6
7/3.EXPERIMENTAL EV ALUATION
Here we are interested in two main types of experimenta-tion.Firstly to show that our intelligent bidding strategy performs effectively in a wide range of bidding contexts.Sec-ondly,given its success,we believe that it will be widely adopted and thus we want to understand what will happen when there are multiple such agents in the environment.
In thefirst set of experiments,the performance of the agent is measured in terms of success rate and average utility.The success rate is defined as the number of times,as a percent-age,the agent is successful in obtaining the item.The utility of winning in an auction i is computed as U i(v)=(p r−v
p r
)+c, where v is the winning bid and c is an arbitrary constant 0.001to ensure that the agent receives some value when the winning bid is equivalent to its private valuation.
In order to assess the robustness of our bidding agent,we wanted to consider how it would perform if it could not ac-curately determine the type of environment it is in(e.g.if the actual environment is RP1FE2STMA,what would hap-pen if the agent believed it was RP1FE2MTMA and played the corresponding evolved strategy).This agent we label as inaccurate(c.f.the accurate agent that correctly deter-mines its environment).In this particular experiment,the inaccurate agent always picks a strategy that is very close to the actual environment based on the bidding time8.As an example,if the actual environment is RP1FE2STMA,the inaccurate agent may believe that it is in RP1FE2MTMA and will use the evolved strategy for RP1FE2MTMA as its strategy.The mapping from the actual environments to the inaccurately classified environments is shown in Table2. Here,if the actual environment is categorised as a short time (e.g.RP1FE1STMA),then the inaccurate environment is mapped to a medium time environment(e.g.RP1FE1MT-MA).
As a control model,we use agent C that has a singlefixed strategy based on the user’s behaviour.In particular,we picked the strategy that was evolved for the environments RP2FE1MTMA,RP2FE2MTMA and RP2FE3MTMA9.Th-us,C has three different strategies;when a user is interested in a bargain,C employs a strategy that was evolved for RP2FE2MTMA,when a user is desperate for the item,C uses a strategy that was evolved for RP2FE1MTMA and when a user is looking for a combination of both,it utilises the strategy evolved based on RP2FE3MTMA.
The agents and the control model were run1000times in the marketplace(independently)10.Analysis of variance(AN-OVA)was used to test the hypothesis about the differences of the success rate means(when running the experiment200, 8It is also possible to select a strategy that is close to its private valuation or the number of active auctions or some combinations of the three.
9We chose these environments because the average time to procure the good in the marketplace is50,the average to-tal number of auctions(|L(t)|)is30and the average closing price is76and these values fall into the RP2MTMA cate-gory.
10This means that the intelligent agent and the control mod-els are not competing against each other.Rather,they are competing against the simulated auction participants as dec-sribed in Section2.
Actual Environment Inaccurate Prediction
Short Medium
Medium Short
Long Medium
Table2:The Actual Environments’s Mapping to the Inaccurate Environment
400,600,800and1000times)and the procedure revealed that for all experiments,the differences between means were not significant(F4,15=0.134,p>0.05)and thus the results obtained are statistically significant.The user’s requirement was randomly allocated where the user’s private valuation ranges from70to82and the time allocated ranges from 10to100.The number of auctions running in the market-place is between3and60and there are between2and10 participants in each such auction.
The performance of the intelligent agents(accurate and in-accurate)and the control model in terms of success rate is shown in Figure4.The experiment is divided into four groups.Thefirst three groups show the detailed perfor-mance of the agents and the control model based on a sin-gle behaviour(desperateness,bargain,and both)and the last group shows the overall performance when all three be-haviours are considered11.It can be seen that the accurate agent achieved the highest success rate in all groups.This shows that by having the ability to change the strategy ac-cording to the user’s preference and the environment it is situated in,our agent can maximise its chances of succeed-ing.This is different from C,in which afixed strategy is used and where the agent views the environment and the user’s preference as static.In this case,C was only suc-cessful in the environment for which it was evolved.Even though the inaccurate agent’s success rate is lower than the success rate achieved by the accurate agent,it is still better than that of C.This is because the strategy selected by the inaccurate agent is one that is usually fairly close to the actual environment and this proximity means the deployed strategy is likely to be reasonably effective.
Figure4:Success Rate Comparisons
11Here,the user intention is generated randomly.Figure5shows the average utility obtained by the intelli-gent bidding agents and the control model.Again,it can be seen that the accurate agent performed well compared to the other two models(except in the case of desperate behaviour where it obtained a lower average utility than the inaccurate one).This can be explained by the technique used by the inaccurate agent when selecting its strategy.As an example, when the actual environment is RP2FE1STMA,the inaccu-rate agent uses RP2FE1MTMA which is characterised by a low initial bid value and slowly reaching its p r.The strat-egy for RP2FE1STMA is similar,but the rate at which its reaches p r is faster.As a result,when the inaccurate agent obtains the item,the price it pays is low and this leads to a higher utility.Again,the inaccurate agent recorded a higher average utility than C but lower than the accurate agent. This indicates that,even when our agent cannot accurately assess the environment that it is in,by approximating its environment its performance is still better than an agent that uses afixed strategy.
Figure5:Average Utility Comparisons Based on the level of achievement attained by the inaccurate agent,we conducted another experiment that analyses the performance of our intelligent bidding agent when it makes varyingly accurate predictions of the environment that it is in.This is achieved by gradually increasing the percent-age of errors that it made when assessing its environment. Here,the set up of the experiment is as before but this time we focused on the overall behaviour of the agent.From Figure6,it can be seen that as the error’s percentage in-creases,the success rate and the average utility obtained by the intelligent agents decreases.With a50%error rate the agent’s success rate and average utility performances decrease by12%and13%and with a100%error,its per-formance deteriorated by more than25%.This result shows that even when our agent makes the wrong assessment half of the time,it is still able to achieve a reasonable success rate and a reasonable average utility(it obtained61%suc-cess rate and0.03291average utility,whereas C obtained 57%success rate and0.2934average utility in the previous experiment).
We now turn to the second set of experiments.The pur-pose of these is to observe the effect on buyers and sellers (in terms of market efficiency)of having multiple intelligent bidding agents in the system.Here,we take afixed environ-ment consisting of30auctions(10of each type)with10bid-
Figure6:Agent’s Performance with Varying Envi-ronment Prediction Accuracy
ders in each auction.Thefirst group(called the dummy bid-ders)are those that operate in a single auction and have the intention of buying the target item and possessing certain behaviour(bargain seeking,desperate,etc.).They main-tain information about the target item they wish to pur-chase,their private valuation of the item,the starting bid value and their bid increment.These values are generated randomly from a standard probability distribution.Their bidding behaviour is determined based on the type of the auction that they are participating in and are based on the dominant strategies of the respective one-shot single auc-tions(as described in Section2).The second group is made up of the intelligent agents.These agents have all their indi-vidual and environmental parameter set in the same way as the dummy agents.Specifically,Figure7shows the supply and demand curves for the market.The intersection of the two curves is called the equilibrium price(when the supply of goods meets the demand),P0,and the quantity traded at this price is the equilibrium quantity.In this particular ex-periment,the equilibrium price is at81and the equilibrium quantity is25.
Figure7:Supply and Demand
To measure the effectiveness of the market,we calculate its allocative efficiency and the Smith’s Alpha(α)coefficient [11].Allocative efficiency is defined as the total actual profit earned by all the sellers divided by the maximum total profit that could have been earned in an ideal market(expressed as a percentage).The Smith’s Alpha coefficient measures
Number of
Intelligent
Agents
Average
Utility
Average
Number of
Auctions
Won
Average
Utility
Average
Number of
Auctions
Won
00.00341926.85
300.007401 5.000.00280720.70
600.0068369.900.00259316.60
900.00618012.200.00250914.65
1200.00471613.000.00249011.80
1500.00458415.650.0022549.45
1800.00450718.500.002090 6.50
2100.00434319.300.001846 4.70
2400.00381920.100.001410 2.55
Intelligent Agents Dummy Bidders
Table3:The Bidder’s Average Utility
how close the actual trade prices are to the equilibrium,α=100∗σ0/P0,whereσ0is the standard deviation of trade prices around P0.We initially ran the marketplace with no intelligent agents and observed the average utility obtained by the dummy bidders.We then replaced30dummy bidders with30intelligent agents and observed the change in aver-age utilities for both the dummy bidders and the intelligent agents.To be more precise,at each stage of the experiment, one dummy bidder in each auction is removed from the mar-ketplace and is replaced with a single intelligent agent.This number is gradually increased in increments of30until there are60dummy agents and240intelligent agents12.
Given this setup,Table3shows the average number of auc-tions won13and the corresponding average utilities gained by the dummy bidders and the intelligent bidders for a total of20runs.It can be seen that the dummy bidders achieved the highest average utility when there were no intelligent agents(as expected).This value steadily decreases as more intelligent agents enter the system.The intelligent agents achieved the highest average utility when there were30such agents participating in the auctions.This is because there is only a small number of intelligent agents participating in the marketplace.This situation allows them to be in direct competition with the dummy bidders.After this point,the average utility gradually decreases as more intelligent agents enter the marketplace.These agents also obtain higher suc-cess rates as their number increases(as reflected in the num-ber of auctions won by them).This is to be expected since there are now more intelligent agents dominating the mar-ketplace leading to a scenario where they are now compet-ing against each other(rather than competing against the dummy bidders)to maximise their user’s satisfaction.It is interesting to note that even when there were240intelligent agents in the marketplace,the average utility obtained by these agents surpassed the highest average utility recorded by the dummy bidders.This implies that even when there 12Our auctions require a minimum of2agents to be present before they start and so we cannot move to only intelligent agents without distorting the experimental setting of run-ning a constant numbers of agents.
13Not all auctions result in a sale(because of a lack of ap-propriate bids)and so the number of auctions won does not always sum to30.。