developing internet e-commerce benchmarks

t
Recommended by Michael P. Papazoglou and Aphrodite Tsalgatidou 475
476
DAWN JUTLA et (11.
The business model determines processes and transactions and thus also the database and navigational designs. Because a benchmark must model these aspects, development of an e-commerce benchmark must be targeted to a specific business model. Other aspects of benchmark design include definitions of performance metrics, probability distributions for workload characterization, and reporting formats for results. The paper is organized as follows. Section 2 outlines the goals and suggests users of an e-commerce benchmarking application. Three classes of e-commerce business models are briefly described in Section 3. Section 4 details components of e-commerce benchmark design. Section 5 describes an example benchmark design. Section 6 briefly outlines an example implementation and raises other technological issues in the e-commerce environment. Section 7 presents example results. Related work is detailed in Section 8. Section 9 gives a summary and concluding remarks. 2. BENCHMARKING GOALS AND METRICS E-Commerce benchmark applications can aid in system sizing, capacity planning, system tuning, and quality of service measurement and validation for transactional ecommerce business models. System sizing refers to the short-term initial sizing activity such as selecting processor speeds, memory capacities and number of servers involved in hardware acquisition. Capacity planning covers what-if scenarios. It can occur many times in the business cycle; for example to handle peakedness, scalability issues, introduction of new services or on business expansion. System tuning refers to the “tweaking” of a system to get maximum performance out of the configuration. Acceptable quality of service refers to reasonable site response times, transaction throughput, availability of connections to the site and other site services, reliability as in persistence of connections and uptime, and acceptable levels of video stream and audio rates. “Everyone” is going online to support business activities. E-Commerce hardware vendors, commerce service providers (CSPs), IT departments of big businesses, and consultants can use the benchmark to do system sizing and capacity planning for their E-Commerce customers. Example management questions that the benchmarks can address are:
1. INTRODUCTION Critical success factors for an Internet e-commerce business channel can be categorized as strategic, technical and functional. Strategies embody benefits gained from first-to-market, brand establishment, customer focus, targeted marketing, outsourcing and development of a customer or user community. Technical issues encompass quality of service items such as response time, transactional throughput, site availability, reliability, video and sound quality. Scalability, the interoperability of technologies, and security are also in the technical category. The supported transaction set (e.g. buy, shopping cart, browse, search and order status, negotiation, delivery) overlap in both user and system application functions. Functional customer care features include facilitation of product customization, support for negotiation and access to a similar-interest user community. Strategic benefits are not quantified in any existing benchmark application; they are difficult to quantify and return on investment formulae/calculations for e-commerce are not yet fully developed. However metrics for quality of service items are relatively easy to define for an e-commerce benchmark application. Functionality is determined by the underlying e-commerce business model and can be captured in the benchmark e-commerce application itself. The diverse emerging e-commerce business models complicate e-commerce benchmark development. We identify the need to develop different e-commerce benchmarks for each individual e-commerce business model. Business models for e-commerce are classed into the following categories: the e-broker (cybermediary) model, the manufacturer model, and the auction model [ 111. The cybermediary model is being used successfully by companies such as l-8OOFLOWERS, and . Dell and Cisco are example successes of the manufacturer business model. The auction model can be found in companies such as priceline.cdm. Because of continuous change in e-commerce we expect hybrid and completely new models to evolve: the categorization is far from complete at this stage.