Abstract A feedback-based model for IO servicing

A feedback-based model for I/O servicingAthena Vakali *,G.I.Papadimitriou,A.S.PomportsisDepartment of Informatics,Aristotle University of Thessaloniki,P.O.Box 114,54006Thessaloniki,GreeceReceived 22February 2000;accepted 3May 2000AbstractModern I/O subsystems have increased their capacity and density but their performance has not been improved accordingly.To address this problem,we have developed a new model that is based on the feedback information provided to the I/O subsystem controller.The presented model is applied to a multiple disk drive subsystem,which serves requests in parallel.Under conventional servicing,each request refers to a speci®c drive and is placed on the corresponding disk drive queue in order to be serviced.The proposed feedback-based model redirects requests among disk drives towards performance gain.The feedback is evaluated by a metric identi®ed by the queue length per disk drive.Performance analysis proves that the feedback-based model results in e ective and improved load distribution.The request servicing in a parallel disk drive subsystem is simulated and simulation runs measure both conventional and feedback-based servicing.The simulation results validate the presented model and prove that it shows an impor-tant improvement in both seek and servicing times compared with the conventional request servicing model.Ó2001Elsevier Science Ltd.All rights reserved.Keywords:Load balancing;I/O subsystems;Parallel I/O;Secondary storage;Parallel disks performance1.IntroductionStorage subsystems are a vital component of modern computer systems.According to Ref.[5],the amount of storage sold has been almost doubling each year,and in this context,magnetic disks are the most dominant devices.Typical accesses to disks are much more slower than accesses to the main memory of a computer system.This fact has created the so called access gap ,which has gained a lot of attention and raised several issues regarding methods to overcome or to *Corresponding author.Tel.:+30-31-99-8415;fax:+30-31-99-8419.E-mail addresses:avakali@csd.auth.gr (A.Vakali),gp@csd.auth.gr (G.I.Papadimitriou),apombo@csd.auth.gr (A.S.Pomportsis).0045-7906/01/$-see front matter Ó2001Elsevier Science Ltd.All rights reserved.PII:S0045-7906(00)00027-6310 A.Vakali et al./Computers and Electrical Engineering27(2001)309±322minimize the di erence between processor speed and disk servicing time.As a consequence of the slow service time of secondary storage media,many applications become I/O bound and the storage system is often a bottleneck in many applications.The expansion of the distributed systems resulted in increased I/O requirements for parallel and distributed applications and the operating systems design has to deal directly with the I/O bottleneck[7,5].The most widely adopted solution to the I/O bottleneck problem is the use of storage subsystems with parallel functionality and capabilities.Multiple disks are used in parallel in order to increase data availability and data access parallelism.Data redundant storage has been implemented by several multiple disk schemes attached to one or more controllers and supported by identical or various disk drive con®gurations.Redundant inexpensive/independent disk arrays(RAID)have been proposed as a varying level methodology for disk redundancy [3,15].A considerable interest has been shown for the shadowed/mirrored disks(mirroring cor-responds to the RAID level1model),where all disks are identical and store the same data.In such systems,enhanced fault tolerance and improved disk performance are achieved at the ex-pense of storage space.In Refs.[1,2,8],analytic models have been developed in order to study the performance of shadowed/mirrored disks by the consideration of expected involved times, focusing on seeking.Storage system management is a major problem a ecting the overall functionality of the cur-rent systems and several approaches have been proposed towards a self-managing storage system. As indicated in Ref.[13]a self-con®guring,self-managing storage system is given and the goals of the system and speci®cations of the workload,the data and the storage devices are available and it decides on how to balance the load and the data among the disk drives.Storage management models have also been developed by identifying the attributes of the workload and/or the disk device(e.g.[6]).These attributes refer to the workload response times,bandwidth,data sizes and so on.Modern disk drives attributes and its characteristics have been identi®ed in Ref.[12]and their inclusion to the simulation model is discussed.Some of these attributes have been studied extensively in order to investigate their impact on the disk's servicing time(e.g.,the rotational delay[9]).Furthermore,the a ect of disk drives characteristics on storage performance has been investigated in connection with the advances in disk technology and the most signi®cant per-formance factors have been indicated in Ref.[10].Furthermore,modern I/O subsystems are reinforced with quite e cient mechanisms imple-mented as policies that perform scheduling,reordering of I/O requests or read-ahead.The current complicated storage systems infrastructure hardens the development of analytic as well as simulation models.Disk controller has been considered as the most suitable component for hosting storage systems policies and current technology provides e cient controllers with re-spect to the disk drive's functionality.Most disk controllers are reinforced with self-managing techniques through standard interfaces used on standard systems without software modi®ca-tions[4].This paper presents a model in solution to the problems of I/O bottleneck and I/O request servicing,as studied in Ref.[16].Our approach is based on the following important assump-tions:·most current storage systems support multiple drives and a queue of requests is associated with each drive,A.Vakali et al./Computers and Electrical Engineering27(2001)309±322311·the response time could be improved by redirecting requests to idle drives or to drives with lighter load,·the information provided by each disk drive's queue could be used as a feedback in order to perform the request redirection,·the storage system could be self-managed and the workload could be served more e ciently.The remainder of the paper is organized as follows:Section2presents the multiple disks I/O storage subsystem con®guration and identi®es the most crucial performance factors.Section3 introduces the proposed feedback model and analyzes the request servicing policies as well as the performance metrics and their impact on improving the storage system responsiveness and functionality.In this section performance analysis is carried out and theorem proofs are provided.Section4presents the simulation results and discusses the performance gain in the presented model.Finally,conclusions are summarized and future work areas are suggested in Section5.2.I/O subsystem with multiple disksSeveral I/O subsystems have been suggested in modern parallel and distributed systems.Most of these assumes the hierarchical memory model proposed in Ref.[17],where an abstract machine consists of a set of processors interconnected via a high-speed network and each processor access an appropriate I/O controller.Each of these controllers manages a set of disk drives.The con-troller is responsible for managing and directing read/write requests to the queues of the disk drives.Most modern magnetic disks have an embedded small computer systems interconnect (SCSI)controller.Here,we concentrate on a multiple disks subsystem where disk drives are managed by a common I/O controller.Disk drives are of the same type and have similar con-®guration requirements(Fig.1).2.1.Device and workload characteristicsA typical data storage hierarchy includes main memory,magnetic disks and possibly tape drives or tertiary storage devices.The host operating system guides an I/O request to the disk drive controller via the appropriate disk drivers.The controller manages the storage and re-trieval of data to and from the disk mechanism and performs mappings between incoming logical addresses and the physical disk location that stores the data[12].The storage subsystem comprises of N individual disk drives,which stores information and each one is considered as an independent drive.Each disk drive is equipped with one read/write head per disk surface and the head moves to the appropriate cylinder location in order to serve the current request.Reading data is satis®ed by accessing the disk which contains the requested data whereas writing is performed by the disk speci®ed by the®le system(as explained next).Disks serve requests in parallel in order to exploit the system's responsiveness.Disk drives have associated queues that contain requests waiting to be serviced.Current controller interfaces support commandqueueing,which permits the reordering of queued requests in order to improve disk perfor-mance(e.g.,SCSI-2).Requests arrive to the system randomly by various independent processes.Some requests arrive, while others are being serviced,and so queues are created in each disk drive.Requests arrival rate could be either constant or independent and exponentially distributed or bursty.The disk con-troller commands the drives to serve the requests posed by the®le system.A typical request consists of the following attributes:Device:the id of the disk drive to serve the request,Operation:either R(ead)or W(rite),Start Location:the physical address where the data are(will be)located,Size:the amount of data(in MBytes)to be read(written),Arrival Time:the time when the request arrived at the controller.According to the above request pattern,the controller will direct each request to the appropriate drive in order to be served.As depicted in Fig.1,each disk drive in the subsystem is an individual server''and has its own queue.Therefore,the disk controller is the common server whose service is required by all I/O operations.2.2.Performance metricsGiven the arrangement of disk surfaces and read/write heads,the time required for a particular I/O operation involves mainly the following actions(Fig.2)[10]:·wait in queue:time spent in queue waiting for the drive to be free for servicing the I/O,·wait for channel:time spent waiting for the channel to be free such that the seek and sectorinformation can be sent down,·seek time:time spent to move the appropriate head to the appropriate cylinder,·latency time:time spent for the required sector to rotate around to the location of the head,·transfer time:time to perform the actual data transfer.Disk performance is measured by speci®c metrics based on the above times spent at each phase of the request servicing process.The overall time for executing and completing a user request consists of command overhead,seek time,rotational delay and data transfer mand overhead has been reducing due to the acceleration of disk controller's chips and mechanisms.Therefore,the service time of a request in the disk mechanism is a function of the seek time (ST),the rotational latency (RL)and the transfer time (TT),whereas queue delay must be considered also for the evaluation of the overall service time [13,14].The most widely used formula for evaluating the expected service time involves these time metrics and is expressed by:E ServiceTime E ST E RL E TT ; 1 where E ST refers to the expected seek time,E RL refers to the expected rotational delay and E TT refers to the expected transferring time.Seeking is a major performance factor and several expressions have been suggested for the expected seek time evaluation.While seeking,the read/write head arm is involved in the opera-tions of speedup,coast,slowdown and settle,successively in order to reach the requested location.The speedup time will be the dominant factor for short seeks,whereas the coast is the dominant factor for long seeks.The following function has been used widely for the approximate evaluation of the seek time:Seek Time dist 0if dist 0a b dist p if 0<dist <cutoff ;c d dist if dist P cutoff 8<: 2where a ,b ,c ,d and cut-o are device-speci®c parameters and dist is the number of cylinders to be traveled.Furthermore,a closed formula has been derived for the expected seek time (E ST )under random uniform access [13].The expected rotational delay is evaluated by E RL % Revolution Time =2 for randomly distributed requests.The transfer time depends on the amount of data to be transferred and is evaluated by E [TT] Request Size =Transfer Rate under a constant TransferRate.3.The feedback model3.1.The request servicing processOur work is based on the idea of redirecting requests at the disk controller,based on provided feedback in order to improve disks performance.Controller is suggested as a better suited place for the task of reorganizing data[4].Here,the controller does not remain a static design,where data are placed as directed by the®le system.Instead,the controller becomes a dynamic tool which e ciently re-directs the requests to the physical medium according to the feedback information.Fig.3presents the structure of our feedback-based model,which revises the basic multiple disk drives model(presented in Fig.1) according to the proposed feedback.Each request is either a read or a write process guided to the controller.Our model is based on the distinction of the requests by their type(Read or Write).The feedback information is used in order to redirect the write requests from heavy loaded disk drive queues to other disk queues which remain idle or are lightly loaded.As presented in Fig.4,the feedback-based controller alters the original request pattern such that the attributes of device and start location are adapted to the systems load in case of writing.Therefore,write service is per-formed by redirecting request pattern within the controller to allow data relocation such that the service time is improved.3.2.The probability updating schemeAt each time slot t,the feedback-based controller contains a probability distribution P(t)over the set of disks.Thus,P t f p1 t ;...;p N t g,where N is the number of disks.The destination-disk of each newly arriving write request is randomly selected according to the probability dis-tribution P(t).The probability distribution P(t)is updated at each time slot by taking into accountthe estimated load of each disk drive.The load is estimated by using the disk queue length since the queue size is an indicative quantity for the disk load.De®nition.The load L i (t )of each disk d i (i 1;...;N )is evaluated in byL i t max f Q i t ; g ; 3 where Q i (t )is the number of requests waiting in the queue of disk d i ,while is a positive real number in the neighborhood of 0.The probability p i (t )of redirecting a write request to a disk d i is inversely proportional to the load L i (t )of this disk.At any time slot t ,for any two disks d i and d j ,we havep i t p j t L j t L i t A p i t p j t L j t L i t : 4 Since the choice between two disks i and j for servicing a write request will be made according to their loads L i t and L j t .The disk with the heavier load has smaller probability than the other disk with the lighter load.Since at any given time slot t ,it holds P N i 1p i 1,we express all p i s i 1;2;...;N )in terms of one speci®c disk,namely the i disk drive.Therefore,p 1 t p i tL i t L 1 t ;p 2 t p i t L i t L 2 t;...p n t p i t L i t L N t : 5From the above,we derive that the choice probability of each disk d i i 1;...;N will be updated according to the following probability updating scheme:p i t1L i tP Nk 11L k t: 63.3.Asymptotic analysisThe performance evaluation is based on the request servicing process and the probability up-dating scheme as discussed in the previous subsections.The following theorem is indicative for the proposed load-balancing model.Theorem.When the feedback-based model is used in a multiple disk system,there is a small slot size, such that for any two disks d i and d jlim t3I L i t limt3IL j t :Proof.For any two disks d i and d j,we de®ne:X i;j t L i t ÀL j t .Without loss of generality, assume that L i t >L j t .Let a be the overall arrival rate of the disk system(in requests/slot)and b is the service rate of each disk(in requests/slot).Since the disks are of identical con®guration b remains the same for all disks.a and b can be as small as desired by selecting a small slot size.We haveE D X i;j t E X i;j t 1 ÀX i;j t ;ap i t Àb À ap j t Àb ;a p i t Àp j t ;a1L i tÀ1L j t P Nk 11L k t;ÀaL i t L j tP Nk 11L k tX i;j t ÀrX i;j t ;7whereraL i t L j tP Nk 11L k t:Thus,E X i;j t 1 1Àr X i;j t : 8 Since r>0,it follows that:E X i;j t 1 <X i;j t : 9 316 A.Vakali et al./Computers and Electrical Engineering27(2001)309±322Furthermore,if we select a small enough slot size,such that a <L i t L j t P N k 11=L k t and consequently,r <1,we haveE X i ;j t 1 >0: 10 From relations (9)and (10),it is derived that ÀX i ;j t 1 is a submartingale [11]and consequently,it asymptotically converges to a random variable ÀX .Therefore,as t 3ID X i ;j t 30A a p i t Àp j t 30A L i t ÀL j t 30:11 Thus,lim t 3I L i t lim t 3IL j t :4.Simulation resultsWe have built an event-based simulator to validate the feedback-based I/O servicing model.In order to study the performance of the proposed model,we have implemented the conventional model and two types of models for the feedback-based model,namely the deterministic and the probabilistic model.According to the deterministic model,requests are redirected by deter-mining the minimum loaded disk (used as feedback)each time,whereas under the probabilistic model the disk choice depends on the disks probability distribution (as described in the previous section).Each simulation run considered arrivals of more than 500,000requests over the simulated time.The simulation model was studied for an I/O subsystem of 2;4;...;10disk drives.Each disk is con®gured by the characteristics proposed in Refs.[12,13]for the HP 97560disk drive.The workload is characterized by the arrival process,the request rate,bursty arrivals and the fraction of read and write processes.The read/write ratio was also a parameter for the simulation process and there are di erent arrival sets depending on the probability of having reads to vary within the range 0:1;...;0:9.The proposed feedback-based model showed to be bene®cial in all cases when compared to the conventional I/O servicing model.Figs.5and 6present the improvement rates of the deterministic and probabilistic over the conventional model,respectively.These rates refer to the improvement in service time as evaluated by Eq.(1).The curves represent the results when reading probability was 0.1,0.3,0.5,0.7,0.9,under I/O subsystems of 2,4,6,8,10disk drives.As it was expected from the performance analysis,the most bene®cial improvements in service time result when having low read ratio and many parallel disk drives,since there is an increased exploitation of the service parallelism and load balancing.More speci®cally,the improvement rates for the deter-ministic over the conventional model vary between 4%(2disks,0.9read ratio)and 19%(10disks,0.1read ratio),whereas the corresponding improvement rates for the probabilistic over the conventional model vary between 5%(2disks,0.9read ratio)and more than 20%(10disk,0.1read ratio).Seek time shows a similar improvement rate of the feedback-based model compared to the conventional one.Similarly,Figs.7and 8present the seek time improvement rates of the A.Vakali et al./Computers and Electrical Engineering 27(2001)309±322317deterministic and probabilistic over the conventional model,respectively.The improvement rates for the deterministic over the conventional model vary between 6%(2disks,0.9read ratio)and 18%(10disks,0.1read ratio),whereas the corresponding improvement rates for the probabilistic over the conventional model vary between 7%(2disks,0.9read ratio)and more than 22%(10disks,0.1read ratio).Figs.9and 10depict the load distribution under indicative loads serviced by both the con-ventional and the proposed feedback-based probabilistic model.More speci®cally,Fig.9depicts the load distribution for a parallel I/O subsystem of 10disk drives,under both the feedback-based and the conventional model under 0.1read ratio and Fig.10presents the same bars for a 0.7read ratio.These histograms depict the load variation for each of these models and they represent the bene®cial load distribution of the proposed feedback-based model.Thefeedback-based model exploits the disks with low loads and improves the overall requests ser-vicing process.5.Conclusions ±further researchThe presented paper provided a new I/O servicing model in a parallel multiple I/O subsystem.The proposed model has introduced a request servicing redirection,based on disk queue infor-mation used as feedback.The redirection concerns write requests and the I/O controller is re-sponsible for the new model implementation.The performance analysis proves that the proposed model improves the I/O servicing process and this is also documented by a developed simulation model.Simulation runs for heavy disk loads have been presented and indicative results aredemonstrated.Both service and seek times are considerably bene®ted by the feedback-based model at rates over than 22%.Future work could expand this model in order to include di erent disk technology con®gu-rations.This expansion could be quite useful for investigating the in¯uence of disk parameters to the feedback-based processing.Also,we could adopt di erent load estimations such as expected seek or rotational delays at the disk queues.Disk caches could also be added to the model structure in order to further study the caching in¯uence to the I/O servicing performed under the proposed feedback-based model.References[1]Bitton D,Gray J.Disk shadowing.Proceedings 14th VLDB Conference.1988.p.331±38.[2]Bitton D.Arm scheduling in shadowed disks.Proceedings IEEE COMPCON 89Conference.1989.p.132±36.[3]Chen P,Lee E,Gibson G,Katz R,Paterson D.RAID-high performance,reliable secondary storage.ACM Computer Surveys 1994;26(2):145±85.[4]English R,Stepanov A.Loge:a self-organizing disk controller.HPL-91-179,HP Labs,Technical Report.1991.[5]Gibson GA,Vitter JS,Wilkes J,et al.Strategic directions in storage I/O issues in large-scale computing.ACM Computer Surveys 1996;28(4):763±79.[6]Golding R,Shriver E,Sullivan T,Wilkes J.Attribute-managed storage.Hewlett-Packard Laboratories,Technical Report HPL-SSP-95-11,Storage Systems Program.1995.[7]Jain R,Werth J,Browne JC,editors.Input/output in parallel and distributed computer systems.Dordrecht:Kluwer Academic Publishers,1996.[8]Lo RW,Matlo NS.Probabilistic limit on the virtual size of replicated disc systems.IEEE Trans on Knowledge and Data Engineering 1992;4(1):99±102.[9]Ng SW.Improving disk performance via latency reduction.IEEE Trans on Computers 1991;40(1):22±30.[10]Ng SW.Advances in disk technology-performance issues.IEEE Computer 1998;31(5):75±81.[11]Tucker H.A graduate course in probability.New York:Academic Press;1967.[12]Ruemmler C,Wilkes J.An introduction to disk drive modeling.IEEE Computer 1994;27(3):17±28.[13]Shriver E.Performance modeling for realistic storage devices.PhD Thesis,Department of Computer Science,New York University,1997.[14]Shriver E,Merchant A,Wilkes J.An analytic model for disk drives with readahead caches and request reordering.ACM SIGMETRICS'98,Conference Proceedings.1998.p.182±91.[15]Wilkes J,Golding R,Staelin C,Sullivan T.The HP AutoRAID hierarchical storage system.ACM Trans on Computer Systems Feb 1996;14(1):108±36.[16]Vakali A,Papadimitriou GI.An adaptive model for parallel I/O processing.Proceedings of the IASTED International Conference.Parallel and distributed systems (PDCS'99)1999.p.13±42.[17]Vitter J,Shriver E.Algorithms for parallel memory I,II.Department of computer science,Brown University,Technical Report CS-90-21.1990.Athena I.Vakali received a B.S.degree in Mathematics from the Aristotle University of Thessaloniki,Greece and an M.S.degree in Computer Science from Purdue University,USA.In 1997,she received a Ph.D.degree in Computer Science from the Aristotle University of Thessaloniki.Since 1997,she is a Lecturer at the Department of Informatics,Aristotle University of Thessaloniki,Greece.Her research interests include de-sign,performance and analysis of storage subsystems and data placement schemes for multimedia and Web based information.She has published several papers in international journals and conferences.A.Vakali et al./Computers and Electrical Engineering 27(2001)309±322321Georgios I.Papadimitriou received the Diploma and Ph.D.degrees in Computer Engineering from the Uni-versity of Patras,Greece in 1989and 1994,respectively.From 1989to 1994,he was a Teaching Assistant at the Department of Computer Engineering of the University of Patras and a Research Scientist at the Computer Technology Institute,Patras,Greece.From 1994to 1996,he was a Postdoctorate Research As-sociate at theComputer Technology Institute.Since 1997,he has been a Lecturer at the Department of Informatics,Aristotle University of Thessaloniki,Greece.His research interests include design and analysis of broadband networks and learning automata.He has published several dozens of papers in international journals andconferences.Andreas S.Pomportsis received a B.S.degree in Physics and an MS.degree in Electronics and Communi-cations (both from the University of Thessaloniki),and a Diploma Degree in Electrical Engineering from the Technical Universityof Thessaloniki.In 1987,he received a Ph.D.degree in Computer Science from the University of Thessaloniki.Currently,he is a Professor in the Department of Informatics,Aristotle Universityof Thessaloniki,Greece.His research interests include computer architecture,parallel and distributed com-puter systems,and multimedia systems.322 A.Vakali et al./Computers and Electrical Engineering 27(2001)309±322。