A Continuous Quality Improvement Approach to Evaluation

– Benefits – Creating a CQI approach to evaluation
• The Model for Improvement • Drawbacks 2•020/4C/28 onclusions
Smaller World Communications
• Performance measurement and evaluation • Primarily in the health care and health
2020/4/28
Evaluation
• Process - studies program implementation
- tracking quantity and description of people who are reached - tracking quantity and types of services - description of how services are provided - quality of services provided - was everything implemented the way you thought
2020/4/28
Presentation Outline
• Overview of evaluation and quality improvement
• Discussion…..how does this relate to your practice?
– Challenges experienced with traditional approach to program evaluation
• Although most text books state there are many purposes or uses to evaluation the underlying tone is still to demonstrate effectiveness or proper use of resources
of participants - ways of improving a program/service - demonstrating the effectiveness of a program to
funders and other stakeholder groups
2020/4/28
Evaluation
2020/4/28
Learning Objectives
•Tools we will cover today
–Needs assessment –SWOT analysis –Flow charts –Fishbone diagram –Affinity Diagram –Brainstorming –Prioritization Matrix –Various measurement tools
2020/4/28
Applied market research skills and program evaluation in hospital setting
Indicators Client and employee opinions Clinical outcomes
Balance Score Cards
The evolution of an evaluation practice
Organizational Development
Natural progression to increase utilization of performance measurement data
Facilitation skills Team building Business strategy Leadership Continuous Quality Improvement
2020/4/28
Evaluation
• “Evaluation research is the systematic application of social research procedures for assessing the conceptualization, design, implementation, and utility of social intervention programs.”
2020/4/28
Evaluation
Treasury Board Policy on the Evaluation of Programs….Guidelines
Program evaluation in federal departments and agencies should involve the systematic gathering of verifiable information on a program and demonstrate evidence on its results and costeffectiveness. Its purpose should be to periodically produce credible, timely, useful and objective findings on programs appropriate for resource allocation, program improvement and accountability.
• Evaluation’s focus is on ‘measurement’
• In order to collect valid and reliable information, evaluators utilize, as much as possible the scientific method for collecting information.
The evolution of an evaluation practice
Evaluation of PT Clinics/ QA Programs for Professional Colleges
Performance Measurement
Adapted to language of ‘standards’and RHPA
2020/4/28
Evaluation
• Formative - information you collect to help you plan or implement a program/service
- needs assessments - pre-testing program materials - audience analysis
(1981)
Treasury Board of Canada (1981 ). Guide on the Program Evaluation Function
2020/4/28
Evaluation
• Evaluation helps you to make decisions about:
- the optimal use of time and resources - determining if the program/service is meeting the needs
• Program evaluation - social sciences research
• CQI - organizational development/business
2020/4/28
Evaluation
• Application of social science methods
• Emerged from the Education and Public Health fields prior to WWI
2020/4/28
The evolution of an evaluation practice
Clinical Research
Program Evaluation
Controlled experiments in the lab Clinical trials in hospitals
Research Design Threats to validity/reliability Highly controlled Mutli-variate statistics
Move from QA to Continuous Learning
Demonstrate effectiveness for accountability (Insurance Co.)
Demonstrate evaluating practice for College Requirements
promotion field • Clients include (health units, hospitals, not
for profit organizations, professional colleges, funding agencies) • Some work in the private sector
– Rossi and Freeman, 1993
2020/4/28
Evaluation
• The increase in popularity of program evaluation emerged from the need for government programs to be accountable
2020/4/28
Applied research skills to evaluating health promotion programs
Evaluation Design Indicators Goals/objectives/outcomes Internal evaluation Lack of experimental control
2020/4/28
Learning Objectives
1. Participants will understand the similarities between evaluation and quality improvement. 2. Participates will be able to explain the benefits of approaching evaluation from a quality improvement perspective 3. Participants will understand how evaluation ‘fits-in’ to quality improvement 4. Participations will be able to describe the Model for Improvement 5. Participations will identify different methods and tools used in a CQI approach to evaluation
2020/4/28
OverБайду номын сангаасiew: Evaluation and CQI
• The two disciplines developed in parallel with the same goal “to improve our services and programs”.
• But, they stemmed from two different areas of study
• By 1930s, applied social science and program evaluation grew at a rapid rate
• Evaluation of government programs first tookoff in the U.S. with Canada not far behind
• It was quickly recognized what we need to measure depends on the programs stage of development
2020/4/28
Evaluation
• Types of Program Evaluation
- Formative - Process - Summative (outcome)