Clinical forecasting in drug development

FROM THE ANALYST’S COUCH Clinical forecasting in drug developmentAsher D. Schachter and Marco F. Ramoni Deckchair by Alexander McQueen, part of the ‘Deckchair Dreams’ range. To view the other 24 designs, go to / deckchairdreamsThe cost of developing an innovative drug remains the subject of ongoing debate,with some estimates now exceeding the widely quoted figure of US$802 million1. Whatever the true cost, however, it is clear that late-stage clinical failures account fora large proportion of the expenses. Thiscan be as a result of both the large out-of-pocket investments in Phase III clinical trials and because unsuccessful trials tie up capital resources during their conduct, and potentially also for the time spent during any attempted recovery following regulatory rejection. So, there is an interest in strategies that could halt, as early as possible, the development of drugs that eventually fail. Even a small improvement could have a considerable impact; it has been estimated that terminating just 5% of Phase III failures in Phase I could reduce out-of-pocket clinical costs by 5.5–7.1%2.Obviously, clinical factors — drug efficacy and safety — have a major effecton attrition rates and the associated costs1,3. However, clinical forecasting to predict late-phase safety and efficacy based on earlier-phase clinical data with sufficient accuracy to facilitate early termination of eventual failures without stifling would-be successful drugs is a major challenge. Furthermore, although various market-forecasting approaches — which generally aim to assess the financial effect of future market share, competition and profitability — are established4, the working assumption of these approaches is that the drug in question will receive regulatory approval. Indeed,it has been noted that early-stage market-research efforts have almost always failedto predict blockbusters5, and so accurate clinical forecasts would also empower assumptions related to drug success that underlie market forecasts.With these issues in mind, here,we briefly discuss evidence related to the ability of clinical forecasting strategies to significantly reduce failure rates without impeding innovation, and highlight issues that need to be addressed toallow the widespread implementationof such strategies.Forecasting strategiesPharmacokinetic/pharmacodynamic(PK/PD) and trial simulation models helpinvestigators improve later-phase trial designsin terms of trial length, sample size, selectionof target populations and optimization ofdosing regimens6–10. For example, Hunt andcolleagues6 demonstrated the value of a specificPK/PD forecasting model, and De Ridder usedPhase II data from a single drug to developa dose–response model that approximatedthe actual Phase III dose–response curves10.PK/PD modelling and trial simulationusage will probably increase as adaptive trialmethodologies gain wider acceptance11.However, the comparison of a drug indevelopment with a broader range of drugsis beyond the aim and scope of conventionalmodels. As an initial step to tackling thischallenge, we considered clinical forecastingfrom the perspective of modelling populationsof drugs rather than populations of patientsin a given development programme. Our goalwas to use knowledge of prior successful andfailed new chemical entities (NCEs) stratifiedby therapeutic class (for example, antineoplasticand cardiovascular) to inform a predictivemodel for NCEs in development in a particularclass (BOX 1). The resultant Bayesian networkmodel predicted the probability of Phase IIIsuccess and New Drug Application (NDA)approval for an independent validation setof antineoplastic NCEs with 78% accuracy12.In this case, application of the model wasshown to reduce the mean capitalizedexpenditures by US$283 million per successfulnew drug compared with the pharmaceuticalindustry’s reported success rates12.Widespread implementation of predictivemodels with this degree of accuracy couldhave a significant positive effect not onlyon the pharmaceutical and biotechnologyindustries and their investors, but alsoon public health and health care delivery.Patients enrolled in Phase III studies wouldbe more likely to be administered effective,safer drugs. Companies could reduce drugdevelopment costs, and pass on some of thesesavings to the consumer while remainingcompetitive, and investors could place moreconfidence in market forecasting models.Challenges for forecasting modelsThe task of building our independentvalidation data-set highlighted a majorimpediment to widespread clinical forecasting:the lack of data sharing in the pharmaceuticalindustry and between industry and the public,particularly with respect to terminated orfailed drugs. Current models cannot addressnon-adherence to eligibility criteria by specificclinical trial centres, idiosyncratic adverseevents, or any other variables unrelated tosafety or efficacy. However, with access to high-quality data on past failures, these seemingly‘unpredictable’ factors could be modelledinformatively.If certain changes can be effected(TABLE 1), predominantly with respect to datasharing, the power of clinical forecastingapproaches could grow exponentially.It is understandable that incentives existfor a given company to keep clinical trialdata proprietary, but clinical forecastingapproaches could increasingly demonstratethe broad benefits of exploiting this gold-mineof information. Indeed, the PharmaceuticalResearch and Manufacturers of America(PhRMA) have launched a publicly accessibledatabase of clinical trial results, and theirsupporting position paper13 indicates awillingness to share information. Otherefforts to improve the transparency of clinicalresearch have resulted in the creation ofclinical trial databases such as ClinicalTrials.gov, which is currently a mechanism forconnecting potential study subjects withrelevant trials that are actively recruiting(trial results are not as yet available).However, although it seems there isgrowing acknowledgement by industry ofthe importance of releasing late-stage trialdata, the public release of early-stage trialdata, which is perceived as being potentiallyharmful to competitive firms, is much morecontroversial14. Less controversial is the rolethat Bayesian methods could have in thepost-marketing surveillance system recentlyproposed by the Institute of Medicine15.Indeed, it has been shown that Bayesiananalysis would have identified the cerivastatin–rhabdomyolysis association within 6 monthsof approval had this method been available16,and Bayesian methods are a part of the WHO ▶N E W S&A N A LY S I SCLINICAL FORECASTING IN DRUG DEVELOPMENT | MARKET INDICATORS▶N E W S & A N A LY S I S。