Downloaded from

Clinical Trials The other side of clinical trial monitoring; assuring data quality and procedural adherenceGeorge W Williams Clin Trials 2006; 3; 530 DOI: 10.1177/1740774506073104 The online version of this article can be found at: /cgi/content/abstract/3/6/530Published by:On behalf of:The Society for Clinical TrialsAdditional services and information for Clinical Trials can be found at: Email Alerts: /cgi/alerts Subscriptions: /subscriptions Reprints: /journalsReprints.nav Permissions: /journalsPermissions.nav Citations (this article cites 8 articles hosted on the SAGE Journals Online and HighWire Press platforms): /cgi/content/refs/3/6/530Downloaded from at PENNSYLVANIA STATE UNIV on April 11, 2008 © 2006 The Society for Clinical Trials. All rights reserved. Not for commercial use or unauthorized distribution.Clinical Trials 2006; 3: 530–537ARTICLEThe other side of clinical trial monitoring; assuring data quality and procedural adherenceGeorge W WilliamsBackground Data monitoring can mean different things. It can mean statistical methodologies for clinical trial monitoring, interim data analysis, monitoring for quality control or assurance or safety reporting to regulatory agencies. Purpose The various facets of data monitoring will be discussed and reviewed from primarily an industry perspective. Methods By careful attention to the design and conduct of a clinical trial, the expense of monitoring can be markedly reduced. Careful attention should be given to the qualifications of investigators in the selection of clinical sites and central facilities. Site personnel must be adequately trained. The sponsor should utilize appropriately qualified individuals to supervise the overall conduct of the trial. The monitor should visit the investigator at the site of the investigation frequently enough to ensure acceptable quality. The monitor is responsible for inspecting the case report forms at regular intervals. Quality control should be applied to each stage of data handling to ensure that all data are reliable and have been processed correctly. The auditor will assess whether the site is being monitored in accordance with the monitoring plan. The determination of the extent and nature of monitoring should be based on considerations such as the objective, design and complexity of the trial. Statistical sampling may be an acceptable method for selecting the data to be verified. The monitor should ensure that adverse events are reported. Study data will be monitored on an ongoing basis to ensure patient safety. The sponsor may utilize a Data Monitoring Committee to protect the validity of a trial. Conclusions Discussions between industry, academia and regulatory groups regarding the optimal extent and methods for monitoring of clinical trials are encouraged. Clinical Trials 2006; 3: 530–537. IntroductionMonitors are employed by sponsors to review the conduct of clinical studies to assure that clinical investigators abide by their obligations for the proper conduct of clinical trials. Hence, monitoring is the act of overseeing the progress of a clinical trial and ofensuring that it is conducted, recorded and reported in accordance with the protocol, standard operating procedures (SOPs), good clinical practice (GCP) and applicable regulatory requirements. The extent to which a clinical trial is conducted according to the protocol will have a major impact on the credibility of the results. Careful monitoring can assist in identi-Amgen, Inc., Thousand Oaks, California, USA Author for correspondence: George W Williams, Amgen, Inc., One Amgen Center Drive, MS # 24-2-C, Thousand Oaks, California 91320-1799, USA. E-mail: gewillia@ Based on a presentation at the 10th International Symposium on Long-Term Clinical Trials, Keble College, Oxford, UK, September 2005.© Society for Clinical Trials 200610.1177/1740774506073104Downloaded from at PENNSYLVANIA STATE UNIV on April 11, 2008 © 2006 The Society for Clinical Trials. All rights reserved. Not for commercial use or unauthorized distribution.Monitoring of clinical trialsfying difficulties early and minimizing their subsequent occurrence or recurrence. There are many groups with monitoring responsibility – eg, the sponsor, the investigators, the medical monitor, the regulatory authorities, Institutional Review Boards (IRBs) and Data Monitoring Committees. However, the final accountability for the conduct of a clinical trial is with the study sponsor and study investigators. Instead of endorsing a single approach to study monitoring, the International Conference on Harmonization (ICH) E-6 [1] indicates that every study needs an adequate monitoring/auditing plan, appropriate to the study. Onsite monitoring can range from the industry model (frequent, all sites) to no onsite monitoring at all with central monitoring in conjunction with procedures such as investigator training, investigator meetings and extensive written guidance to assure the appropriate conduct of the trial in accordance with GCP [2]. We will now consider many of these aspects in more detail and consider some of the cost implications of different approaches.531on. These complexities impact the monitoring of clinical trials.Investigator/site selectionCareful attention should be given to the qualifications of investigators in the selection of clinical sites and central facilities for participation in the clinical trial. Many factors need to be considered in the selection of investigators, including their expertise, scientific training, ability to recruit patients and time to devote to the trial. It is the responsibility of the investigator and his site personnel to ensure the adequacy of the research facility for the conduct of the trial and the accuracy of the collected data. All the different facilities involved in the study need to be considered, for example, laboratory areas and the pharmacy, as they may generate study data that need to be examined.Training and pre-investigation meetings Impact of design on monitoringAlthough it may be impossible to avoid all errors and inconsistencies in data generated in clinical trials, careful attention should be devoted during the design, conduct, and analysis stages of clinical trials to minimize these problems.The scientific integrity of the trial and the credibility of the data from the trial depend substantially on trial design. The complexity of the protocol bears directly on the quality of the data. By careful attention to the design and conduct of a clinical trial, the expense of monitoring can be markedly reduced [3,4]. Protocols should be realistic in the amount of data to be collected and the ability of patients and investigators to comply with protocol requirements. The quantity of data that must be collected in a study is flexible, depending on the stage of development of the drug and the nature of the trial [2]. The volume of noncritical data required from a clinical trial should be kept to a minimum. Only important information should be collected. Too much noncritical data only leads to confusion rather than helping the data management and statistical aspects of the trial and often does not contribute substantially to addressing the main objectives of the trial. The more data that are collected, the more cumbersome and complicated the case record forms become, and the greater the likelihood of confusion and error [5]. The increasing complexity of trials includes interactive voice response systems for randomization assignments, imaging centers, central laboratories, contract research organizations (CROs) and so Site personnel must be adequately trained to conduct a study in order to successfully produce quality data. Training of investigators, data managers and laboratory personnel is critical and should include appropriate attention to guidelines for good clinical practices. Training sessions and certification procedures will promote standardization and minimize errors. Periodic retraining and recertification is required. It is important to document training of trial personnel. The monitor’s responsibilities include verifying that the investigator receives the current investigator’s brochure, all trial documents and all trial supplies required to support proper trial conduct and compliance with regulatory requirements. Although a visit to each site can be productive, bringing the investigators and coordinators to a central location can encourage the concentration and interaction desirable to enact the protocol effectively [4]. Topics to be included are: protocol review, the efficacy and safety profile of the treatment, safety monitoring standards and adverse event reporting procedures, case report form (CRF) completion and query resolution, GCP/ICH compliance issues, monitoring issues/regulatory concerns, and drug accountability. Investigators are instructed regarding the importance of accurate/clean data, avoiding protocol violations, and keeping patients in the study.Monitors and training of monitorsThe sponsor should utilize appropriately qualified individuals to supervise the overall conduct of the Clinical Trials 2006; 3: 530–537Downloaded from at PENNSYLVANIA STATE UNIV on April 11, 2008 © 2006 The Society for Clinical Trials. All rights reserved. Not for commercial use or unauthorized distribution.532GW Williamsdiscrepancy reports for unresolved data queries). Documentation and training requirements for new staff should be determined. Reconciliation and discrepancy reports of investigational product shipments to and from the site should be determined. There should be follow-up with the investigators as appropriate relative to the status of IRB/IEC approval of the protocol/protocol amendments and informed consent forms. Typically, the first monitoring visit is to occur within two weeks of the first subject’s randomization. If no subjects are registered within the first month, a visit should occur within four to six weeks to assist with recruitment. Following the first monitoring visit, routine site visits will occur approximately every six to eight weeks. Monitoring requirements and frequency at each clinical site may be modified. When problems are sensed through either a site visit or information transmitted to the sponsor, an unplanned site visit is scheduled. Special laboratories (exercise testing facility, echocardiography lab) must be visited and assessed for compliance.trial, to handle the data, to verify the data, to conduct the statistical analyses, and to prepare the trial reports. A monitor need not be a person qualified to diagnose and treat the disease or other condition for the treatment under study, but review of the study data should include a medically credentialed individual with experience in clinical trials and the treatment population. The factors to be considered in determining the number of monitors and the education, training or expertise necessary should include: the number of investigators conducting the study, the number and location of the facilities in which the study is being conducted, the complexity of the study and the nature of the disease or other condition under study. Monitoring workshops will introduce the study to the monitors [6].Monitoring planThe monitoring plan is designed to ensure that the clinical team members monitor the study in a consistent manner and in accordance with principles of Good Clinical Practice, ICH guidelines and SOPs. A standardized, written procedure, sufficiently detailed to cover the general aspects of clinical investigations, may be used as a basic monitoring plan and supplemented by more specific or additional monitoring procedures tailored to the individual clinical investigation. The monitoring plan is dynamic and should be revised and updated as necessary.Source document verificationThe monitor is responsible for inspecting the case report forms at regular intervals throughout the study to verify adherence to the protocol; completeness, accuracy, and consistency of the data; and adherence to local regulations on the conduct of clinical research. The monitor should have access to subject medical records and other study-related records needed to verify the entities on the case report forms. Typically, for studies conducted to support registrational filings, CRF data will be source verified for all enrolled subjects. Monitors will verify 100% for each subject screened that an informed consent form is on file with appropriately dated signatures and that all pages are present. Monitors will verify 100% that all of the inclusion and exclusion criteria are met for each subject enrolled into the study and will verify primary endpoint data.Clinical site visitsAs noted in the Food and Drug Administration (FDA) guidances [6], the monitor should visit the investigator at the site of the investigation frequently enough to ensure that the facilities continue to be acceptable; the study protocol is being followed; changes to the protocol have been approved by the IRB; accurate, complete and current records are being maintained; accurate, complete and timely reports are being made to the sponsor and the IRB; the investigator is carrying out the agreed upon activities and has not delegated them to other previously unspecified staff; and that subjects’ rights are being protected through a thorough informed consent process. In preparation for the site visit, the monitoring plan should be reviewed. Safety monitoring reports from the safety reporting database or serious adverse events reported by the site or any applicable regulatory agency safety reports (eg, Investigational New Drug (IND) safety reports) since the last monitoring visit should be reviewed. The status of outstanding data should be assessed CRFs, clinical database Clinical Trials 2006; 3: 530–537Informed consentThe clinical manager will be responsible for reviewing the draft informed consent form (ICF). The sponsor must review all ICF drafts prior to submission to the IRB, including any changes requested by the sites’ IRB and verify that the latest version of the informed consent includes the essential elements and complies with the sponsor’s sample informed consent form requirements. The sponsor assures that IRB approval has been obtained prior to the enrollment of subjects Downloaded from at PENNSYLVANIA STATE UNIV on April 11, 2008 © 2006 The Society for Clinical Trials. All rights reserved. Not for commercial use or unauthorized distribution.Monitoring of clinical trialsin the study. The sponsor assures that informed consent was obtained from all subjects in the study.533Patient recruitmentIn trials with a long-time scale for the accrual of subjects, the rate of accrual should be monitored and, if it falls appreciably below the projected level, the reasons should be identified and remedial actions taken in order to protect the power of the trial and address any concerns about quality. If the rate of accrual is above projections at specific clinical sites, it is important to verify that eligible patients are enrolled and that the site has capacity to maintain patients on the study. The monitor should ensure that all withdrawals and dropouts of enrolled subjects from the trial are reported and explained on the CRFs.to be taken and/or actions recommended to secure compliance. The detection of fraud and misconduct is a matter of concern for everybody involved in clinical trials. It requires careful judgement by the monitor including scrutiny of all original documentation for correctness. As Walsh [7] has noted, trends and patterns are most easily identified, however, when all the data from a particular center are assembled and reviewed at the same time. It is important when designing monitoring systems that provision is made for this kind of statistical review.LaboratoriesIf local laboratories are used, their participation in internal and external quality control, quality assurance and accreditation schemes should be evaluated by the study monitors. Central laboratories are frequently used. There should be periodic review of documentation and results from internal and external quality control programs implemented at each core laboratory to ascertain compliance with standards specified in the protocol. Computer systems and interfaces between instruments and databases should be validated. Internal quality control often includes replicate values obtained from aliquots of the same laboratory specimens or repeat readings. External quality control may involve the submission of duplicate laboratory specimens or resubmission of records such as electrocardiograms. The external program may also include the submission of known standards for analyses or reading to detect secular trends [3].Drug accountabilityMonitors are responsible for verifying for the investigational product that: 1) storage times and conditions are acceptable, and that supplies are sufficient throughout the trial; 2) investigational product is supplied only to subjects who are eligible to receive it and at the protocol specified doses; 3) subjects are provided with the necessary instruction on properly using, handling, storing and returning the investigational product; 4) receipt, use and return of the investigational product at the trial sites are controlled and documented adequately; and 5) disposition of unused investigational product at the trial sites complies with applicable regulatory requirements and is in accordance with the sponsor requirements.Site visit reports and follow-upAt the conclusion of a monitoring visit, the monitor should: 1) communicate deviations from the protocol, GCP and regulatory requirements to the investigator; 2) report evidence of detected or suspected scientific misconduct immediately to the clinical manager and the clinical quality assurance group for recommended actions; and 3) ensure query clarifications are made and retained by the investigator. In follow-up to a monitoring visit, the monitoring visit report should be completed. A monitoring report is a written report from the monitor to the sponsor after each site visit and/or other trial related communication according to the sponsor’s SOPs. Reports should include a summary of what the monitor reviewed and the monitor’s statements concerning the significant findings/facts, deviations and deficiencies, conclusions, actions taken or Database cleaning and report preparationQuality control should be applied to each stage of data handling to ensure that all data are reliable and have been processed correctly. To ensure the quality of clinical data across all subjects and sites, a clinical data management review will be performed on subject data received by the sponsor. During this review, subject data will be checked for consistency, omissions and any apparent discrepancies. The methods apply to the accumulating data without any unblinding of treatment allocation in individual patients or treatment groups. As Knatterud et al. [3] have noted, there should be checks of the data analyses. There should be consideration of independently reproducing tabulations and statistical calculations including correct selection of variables from the analysis file, proper definition of variables and cut points and correct formulation of transformations of original variables. Clinical Trials 2006; 3: 530–537Downloaded from at PENNSYLVANIA STATE UNIV on April 11, 2008 © 2006 The Society for Clinical Trials. All rights reserved. Not for commercial use or unauthorized distribution.534GW Williamssize and composition of the sample needs to be determined. While the use of sampling may be acceptable when conducting SDV, there may be occasions when checking the data in totality is appropriate. For example, if the number of patients at a center is relatively small, it might be sensible to check all data during an on-site visit. Alternatively, data could be checked 100% until confidence in the quality has been gained, with subsequent sampling of the data. The sample can be dependent on the nature and volume of data generated. A common approach is often to identify different categories of data, for example, critical and non-critical data. Critical data are those that are central to the objective of the study, and which must be corrected. These would generally be few in number. The non-critical data should, of course, also be correct, but if an item of such data is in error, it is not critical to the study outcome. Non-critical data must be checked for a proportion of the CRFs, for example, 25% [5]. Data validation is the cleaning of trial data after they are entered into a computer database in order to ensure that they attained a reasonable quality level. While data quality is centrally important to the success of clinical trials, perfect data are unnecessary and indeed, may be difficult if not impossible to obtain. As Fong [9] has noted, it has been generally agreed that high quality data are data used to arrive at the same conclusion as perfect data. An acceptable data quality level is one that can be considered good enough to not materially effect the results or conclusions. However, an acceptable data quality level has never been defined in regulatory guidances or the literature. Nevertheless, it is critical to optimize data quality by identifying errors from different sources before any conclusions are drawn from statistical analyses. These comments made by Clarke in a textbook on clinical data management are relevant: ‘Defining, setting up, and applying validation is a time-consuming and labor intensive process, so it is important to assess the value of the effort put into validation against the resulting improvement in the data . . . Validation should ease when there is a fair degree of certainty that any remaining errors will not materially affect the results or conclusions of any analyses or the reliability of the trial’ [10]. As Eisenstein et al. [11] have noted, data measurement and capture errors (ie, errors not evident to investigators and monitors) are more frequent than data processing errors. Inter-observer variability in clinical observations consistently exceeds transcription variability (from the medical record to the CRF). Although data capture errors are more difficult to detect and correct, they may be reduced through education. Reducing a trial’s complexity may be an effective strategy to reduce costs and increase data integrity. AuditingIt is important to note the basic difference between quality control and quality assurance in clinical trials. All of the day-to-day process controls that are accomplished by the internal and on-site monitoring staff and directed toward the control of the clinical study collectively comprise the quality control of the study. On the other hand, quality assurance involves independent review or auditing of key processes or pivotal studies in order to establish that the trial has been conducted in accord with GCP, ICH and SOP [1]. Typically, the responsibility for routinely conducting source document verification (SDV) for all patients involved in a clinical study clearly rests with the study monitor (a quality control activity). The auditor’s responsibility is to look at a proportion of the source document verification that has been undertaken to ensure that the system works and is appropriately documented (a quality assurance activity). The auditor will assess whether the site is being monitored in accordance with the protocol specific monitoring plan and applicable SOPs, if the monitoring plan is adequate for the study and site (the frequency of monitoring visits to support the number of subjects enrolled), and if the monitor is effectively monitoring and managing the site to maintain protocol and GCP compliance [5]. Quality assurance audits are performed during enrollment and early in the trial so as to provide feedback to study teams regarding potential problems with the site, the protocol or the management of the study.Costs and extent of monitoring and extent of data cleaningThe determination of the extent and nature of monitoring should be based on considerations such as the objective, purpose, design, complexity, blinding, size and endpoints of the trial. In general, there is a need for on-site monitoring, before, during and after the trial. However, in exceptional circumstances the sponsor may determine that central monitoring in conjunction with procedures such as investigator training and investigator meetings, and extensive written guidance can assure appropriate conduct of the trial in accordance with GCP [1]. In a web model of clinical trial management, much of the study monitoring is performed electronically allowing monitoring of aspects of study conduct in real time. Problems are identified earlier making them easier to resolve. Auditors need considerably less time at sites, reducing cost and increasing study efficiency [8]. Statistically controlled sampling may be an acceptable method for selecting the data to be verified. The Clinical Trials 2006; 3: 530–537Downloaded from at PENNSYLVANIA STATE UNIV on April 11, 2008 © 2006 The Society for Clinical Trials. All rights reserved. Not for commercial use or unauthorized distribution.Monitoring of clinical trials535Adverse event reportingStudy investigators, of course, have the front-line responsibility for identifying potential adverse effects experienced by study participants, adjusting the intervention accordingly and reporting the experience to the sponsor [12]. Regulations require that United States FDA be promptly notified of serious and unanticipated adverse experiences with the use of investigational articles. The monitor should ensure that adverse events, concomitant medications and comorbidities are reported in accordance with the protocol on the CRFs.Monitoring for safetyStudy data will be monitored on an ongoing basis to ensure patient safety. These reviews will include all available data on incidence of adverse events, serious adverse events, fatalities and events leading to withdrawal. Ongoing review of interval and/or cumulative line listings of reported adverse experiences by medical monitors and/or independent experts are conducted.Data monitoring committeesContinuous monitoring of blinded safety data is the responsibility of the study team. In some cases blinded monitoring of safety data is not sufficient, and, in order to ensure the safety of clinical trial subjects, it is also necessary to periodically review clinical trial data (both efficacy and safety) in an unblinded fashion. When it is necessary for the investigator and sponsor to be blinded to treatment assignments, the sponsor may utilize a committee known as a Data Monitoring Committee (DMC) to protect the validity of a trial and its conclusions. The DMC members are individuals independent of the sponsor. A data monitoring committee is generally considered to be a group of experts in relevant subject matter who review the accumulating data from a clinical trial on an interim basis and who make recommendations to the trial sponsor/ organizer regarding the appropriateness of trial continuation and the need for modifications of any protocol design or trial conduct procedures [13]. Interim monitoring of the data from clinical trials is critical to ensure the safety of the individuals participating and to maintain vigilance over the conduct of the trial. Should, for example, the treatments being tested produce some unexpected toxic effect or an unexpectedly high rate of an anticipated adverse effect, the trial organizers would need to carefully consider whether to modify the trial in some way – reduce the dose, tighten the eligibility criteria, prohibit certain concomitant medications or even stop the trial altogether. Another major purpose of monitoring is to determine whether there is any value to continuing the trial. If the results are so overwhelmingly positive or negative at an interim point in the trial that the ultimate conclusion can be readily predicted, early termination of the trial could be considered. Similarly if it has become clear that obtaining a definitive answer to the trial question is unlikely – for example because the event rate is far lower than had been anticipated and increasing the sample size to account for this is not feasible – then termination could be considered in this situation as well. A final important reason for interim monitoring is to maintain the quality of the trial conduct. If the accrual is lagging or the number of ineligible patients is high or the dropout and or non-compliance rate are high the organizers will want to know this while there is still an opportunity to make adjustments [13]. An important advantage of an independent DMC has to do with maintaining the confidentiality of the interim data. Awareness of interim comparisons can damage the trial by affecting accrual rates, drop out rates, endpoint assessment and other aspects of trial conduct. Limiting this information to the DMC minimizes this danger while still providing assurance that the trial can be modified without introducing bias or stopped if concerns arise [13].Stopping rulesFormal statistical methods of interim analysis to avoid inflated type 1 errors have been developed. These include sequential analysis, group sequential designs and analyses possibly utilizing spending functions, stochastic curtailment, repeated confidence interval approaches, and Bayesian methods. Formal guidelines are not typically defined for toxicity and accrual monitoring. Because of the complexity of randomized clinical trials these statistical procedures are intended to provide helpful guidelines rather than rigid rules about whether early trial termination should occur. Recommendations about trial termination or continuation must be based on a global consideration of all available data from the trial including information on primary and secondary efficacy measures, the frequency and severity of adverse effects and quality of trial conduct, along with relevant information external to the trial.Risk assessment/pharmacovigilance/ pharmacoepidemiologyRisk assessment during product development should be conducted in a thorough and rigorous manner. Clinical Trials 2006; 3: 530–537Downloaded from at PENNSYLVANIA STATE UNIV on April 11, 2008 © 2006 The Society for Clinical Trials. All rights reserved. Not for commercial use or unauthorized distribution.。