Variance Estimation

– If averages reveal a smoother and more precise trend as the # of replications increases. – If averages can be smoothed further by plotting a moving average. – Cumulative average becomes less variable as more data are averaged. – The more correlation present, the longer it takes for to approach steady state.
IE680
Output Analysis: Variance Estimation
7
Initialization Bias
• Methods to reduce the point-estimator bias caused by using artificial and unrealistic initial conditions:
• Design of Shewhart Control Chart
– Control Limits
• UCL & LCL
ˆ UCL x 3 ˆ LCL x 3
• Results from incorrect control limits
– If overestimated detection delay – If underestimated high false alarm rate – Estimating process variance (standard deviation) is critical.
•
Nonstationary time series with an upward trend
IE680
Output Analysis: Variance Estimation
11
Stochastic Stationary Process (2)
• A discrete-time stationary process X = {Xi : i≥1} with mean µ and variance σX2 = Cov(Xi,Xi),
IE680 Output Analysis: Variance Estimation 9
Introduction to variance estimation
• Simulation output analysis
– Point estimator and confidence interval – Variance estimation confidence interval
• Consider a single run of a simulation model to estimate a steady-state or long-run characteristics of the system.
– The single run produces observations Y1, Y2, ... (generally the samples of an autocorrelated time series).
– Type of simulations – Steady-State Simulation – Stochastic Stationary Process
• Variance Estimation Methods
– Replication Method – Batch mean methods
• Non-overlapping vs. Overlapping
Output Analysis: Variance Estimation
Jong-hyun Ryu
IE680
Output Analysis: Variance Estimation
Output Analysis : Variance Estimation 1
Contents
• Motivation (Quality Control Chart) • Collecting output data
• The sample size is a design choice, with several considerations in mind:
– Any bias in the point estimator that is due to artificial or arbitrary initial conditions (bias can be severe if run length is too short). – Desired precision of the point estimator. – Budget constraints on computer resources.
IE680 Output Analysis: Variance Estimation 4
Type of simulations
• Non-terminating simulation
– Runs continuously, or at least over a very long period of time. – Examples: assembly lines that shut down infrequently, telephone systems, hospital emergency rooms. – Initial conditions defined by the analyst. – Runs for some analyst-specified period of time TE. – Study the steady-state (long-run) properties of the system, properties that are not influenced by the initial conditions of the model.
– Intelligent initialization. – Divide simulation into an initialization phase and data-collection phase.
• Intelligent initialization
– Initialize the simulation in a state that is more representative of longrun conditions. – If the system exists, collect data on it and use these data to specify more nearly typical initial conditions. – If the system can be simplified enough to make it mathematically solvable, e.g. queueing models, solve the simplified model to find longrun expected or most likely conditions, use that to initialize the simulation.
IE680
Output Analysis: Variance Estimation
3
Type of simulations
• Terminating vs. non-terminating simulations • Terminating simulations
– Runs for some duration of time TE, where E is a specified event that stops the simulation. – Starts at time 0 under well-specified initial conditions. – Ends at the stopping time TE. – Bank example: Opens at 8:30 am (time 0) with no customers present and 8 of the 11 teller working (initial conditions), and closes at 4:30 pm (Te) – The simulation analyst chooses to consider it a terminating system because the object of interest is one day’s operation.
– Standardized Time Series
• Additional Methods
IE680 Output Analysis: Variance Estimation 2
Motivation
• Quality Control Chart
– Simple Control Chart (Shewhart Control Chart)
IE680
Output Analysis: Variance Estimation
8
Initialization Bias
• No widely accepted, objective and proven technique to guide how much data to delete to reduce initialization bias to a negligible level. • Plots can be misleading but they are still recommended.
( X t1 t ,..., X tk t ) ( X t1 ,..., X tk )
• Stationary time series with positive autocorrelation
d
d
where " " denotes equality in distribution
•
Stationary time series with negative autocorrelation

2

2 X
2 Cov( X 1, X 1 j )
j 1

• Variance of the sample mean
n 1 1 2 j Var ( X n ) X 2 (1 )Cov( X1, X1 j ) n n j 1
• Terminating or non-terminating
– The objectives of the simulation study – The nature of the system.
IE680
Output Analysis: Variance Estimation
5
Output Analysis Steady-State Simulation
• Independent and identically distributed (IID)
– Suppose X1,…Xm are iid
IE680
Output Analysis: Variance Estimation
10
Stochastic Stationary Process
The stochastic process X is stationary for t1,…,tk, t∈ T, if
• Performance measures:
– Point estimator and confidence interval – Independent of the initial conditions
IE680
Output Analysis: Variance Estimation
6
Output Analysis for Steady-State Simulation