Crossing the “Valley of Death” Transitioning Research into Commercial Products

Crossing the “Valley of Death”: Transitioning Research into Commercial ProductsA Personal PerspectiveW. Douglas MaughanPotomac Institute for Policy StudiesArlington, Virginia USADouglas.Maughan@Abstract—Many researchers with innovative ideas ju st never seem to be able to bring those ideas to market. One of the biggest problems within the cybersecurity research community is transitioning technology into commercial produ ct. This paper discusses these technology transition activities from the view of a program manager and offers several examples of successful transition for consideration.Keywords- technology transition; cybersecurity; research; development; open source; small businessI.I NTRODUCTIONA consistent problem encountered in the world of research and development (R&D), which is especially true for the field of cyber security, is taking the research through a path of development, testing, evaluation, and transition into operational environments. Past experience shows that transition plans developed and applied early in the life cycle of the research program, with probable paths for the research product, are effective in achieving successful transfer from research to application and use. Transition plans are subject to change and must be reviewed often. Different technologies are better suited for different technology transition paths and in some instances the choice of the transition path will ultimately determine success or failure for the “research product” becoming a “commercial product”.A July 2007 Department of Defense Report to Congress on Technology Transition noted there is evidence that a chasm exists between the DoD Science and Technology (S&T) communities focused on demonstration of a component and/or breadboard validation in relevant environment and acquisition of a system prototype demonstration in an operational environment. That chasm, commonly referred to as the ‘valley of death,’ can be bridged only through cooperative efforts and investments by both communities -- R&D and operations.This paper provides a personal perspective over the past decade or so of the mechanisms that have worked when transitioning technology from government-funded R&D into the commercial marketplace. While this paper highlights successes, there are certainly many more failures than successes and their causes would be an excellent topic for another paper.II.G OVERNMENT F UNDED R ESEARCH &D EVELOPMENTThere are a number of transition paths for research funded by the Federal Government. These transition paths are affected by the nature of the technology, the intended end-user, participants in the research program, and other external circumstances. Success in research product transition is often accomplished by the dedication of the program manager through opportunistic channels of demonstration, partnering, and sometimes good fortune.H owever, no single approach is more effective than a proactive technology champion who is allowed the freedom to seek potential utilization of the research product.Transitioning research products is affected by several factors, including (a) lessons learned about the effects of time/schedule, (b) budgets, (c) customer or end-user participation, (d) demonstrations, (e) testing and evaluation, (f) product partnerships, and (g) other factors.There are several known technology transition paths. I have included a few below with examples to illustrate how these paths might be effectively used.1.Department/Agency to Industry:The Small Business Innovation Research (SBIR) program and the Small Business Technology Transfer (STTR) program directly support technology transfer associated with innovative small businesses. Eleven federal departments participate in the SBIR program (for agencies with $100 million or more in extramural research and development) and five departments participate in STTR (for agencies with $1 billion or more in extramural research and development). Overall, these programs award over $2 billion per year. It is worth noting that these programs are sometimes run within agencies as separate and independent from other programs and activities in the agency. It has also been my experience that many academic researchers are unaware of the STTR program. It is an opportunity for them to partner with a small business and receive at least 30% of the project funds. For a Phase II SBIR, this amounts to $225K. These STTR funds often go unused. Often these small businesses are acquired by larger companies and the technology sees a much broader deployment than would otherwise occur.2.Department/Agency to Academia to Industry(typically via a startup company):2010 IEEE Symposium on Security and PrivacyMany government agencies fund academic research, but do not provide the mechanisms to facilitate transition from the academic world to the commercial world. Successful academic research that has been commercialized has typically been through heroic efforts of the original professor or graduate student via a startup company. There are, of course, venture capitalists that have played a role by providing seed funding for the startup company. The only government entity that provides startup funding is In-Q-Tel, which does so for intelligence community products. 3.Department/Agency to Open Source Community: A number of government agency programs encourage or require software development to be released under open source licensing, as part of the research and developmentactivities. Open source availability is well documented as a powerful and effective means to bring software-basedcapabilities into adoption, use, and support by larger communities.A.Technology Assessment Many researchers, including private sector, academia, and government labs, perform research and development through the prototype phase. There remains an important open issue of how research prototypes are transitioned to full commercial support, documentation, and maintenance. The lifecycle issues are expensive, but essential and are often forgotten in planning funding for the R&D process. Many government agencies and commercial companies use a measure known as a Technology Readiness Level (TRL) to assess the maturity of evolving technologies (materials, components, devices, etc.) prior to incorporating that technology into a system or subsystem. Normally, new technologies are subjected to experimentation , refinement, and increasingly realistic operational testing. Once the technology is sufficiently proven, it can be deployed or incorporated into other systems with confidence of its ability to perform as expected. For more information about TRLs, see [1]. There is considerable debate about the use of TRLs for software and software systems. This continues to plague the entire software ecosystem and is of particular concern in the area of cybersecurity as we have no agreed upon “standards” for software assurance, metrics, and software evaluation. III.T HE D ARPA Y EARS A.The 1990’s I first became acquainted with DARPA in the mid-1990’swhile employed as an information security researcher at the National Security Agency (NSA). They invited information security researchers to participate on their Broad Agency Announcement (BAA) review boards. This led to more involvement in workshops, etc. and an eventual invitation forme to go to DARPA as a Program Manager from H ilarieOrman in 1998. David Tennenhouse, Director of the Information Technology Office (ITO), encouraged me to come to DARPA, but wouldn’t allow me to come until I finished my dissertation (which in retrospect was a verygood incentive to get it done). As I have examined the technology transitions from DARPA that occurred during the 1990’s, two particularly successful examples stand out. The first example is that of the Firewall Toolkit (FWTK). This was developed in the early 1990’s by researchers at Trusted Information Systems [2] and then “open sourced”. It created an entire firewall market that still thrives today. The second example is that of the area of intrusion detection systems (IDS). Research at DARPA that was funded by Michael St. Johns and TeresaLunt and led to several research prototypes. This technologytransitioned in various ways, including researchers thatbraved the entrepreneur world and some who open sourcedtheir work for the good of the community. Most, if not all, ofthe technology transition activities were not funded as part of the original research. B.1999-2003: The Maughan YearsH aving recently completed a Ph.D. from the University of Maryland, Baltimore County, I transferred from theNational Security Agency to DARPA in February 1999 and began working in the Information Technology Office (ITO).I took over a program known as “Active Networks” and quickly launched two programs known as “Dynamic Coalitions (DC)” and “Fault Tolerant Networks (FTN)”. In so doing, I quickly became immersed in the DARPA culture and very quickly came to realize the difficulties Program Managers face as they attempt to transition technology to operational environments and commercial products. In 2000, I successfully “pitched” a new program known as CHATS - Composable H igh Assurance Trusted Systems - that was focused on improving security in the more prevalent open source operating systems at that time, i.e., Linux, FreeBSD, and OpenBSD. The creation of the CH ATS program was particularly interesting. At the time, the Department of Defense was considering policies that would disallow DOD agencies from using open source products within their enterprise environments. As part of this process, MITRE did a study identifying over 200 open source packages in use within the DOD. Shutting down open source software would have shut down the DOD. It was against this landscape that I successfully convinced the Director of DARPA of the need for an R&D program into more secure open source software. If we were going to use it, then we might as well attempt to make it as secure as possible. There were two issues of political interest with respect tothe CH ATS program: 1) the fact that lots of open sourcedevelopers are outside the United States. This creatednumerous issues when it came to contracting and even moreconcerns when we tried to highlight the successful workbeing accomplished and 2) public affairs had real problems presenting the story that DARPA was funding international researchers to create this “new” phenomenon known as open source software. All of this made for a very interesting three year program that eventually made some importantcontributions to the technology transition story of DARPA. Many people did not realize that DARPA was not ideally suited to be an organization that enabled R&D to “cross the valley of death”. In fact, DARPA was responsible for many information security products that were developed to adegree of maturity and left to their own to commercialize or transition in some other fashion. This is where I began to see transition failures as really promising research fell on its face when the researcher/developer attempted to commercialize the technology. In order to explain these transitionroadblocks, I had the following graphic created:The important components on the graphic include:x DARPA (as a part of Research Land) – note that DARPA throws technology over the wall and it sitsuntil someone picks it upx Vendor Land – note that not much exists therex Technology Transition Bridge – note that there are many slats missing from the bridge (which was myattempt to show that few things ever cross it) x Technology Transitions – note that this is shown bya monitor in a rubber raft trying to go from ResearchLand to Vendor Land (attempting to show thedifficulty for a researcher to become an entrepreneurand be successful)C.DARPA SuccessesAs I prepared to leave DARPA in August 2003, I did a retrospective analysis and determined that I had personally been responsible for funding and managing over $200 million dollars of research during my time at DARPA. There were many successful projects, but very few that actually made it successfully into the commercial market. As mentioned previously, my assertion is that this is because DARPA wasn’t set up to assist the researcher across the ‘valley of death’. A decade later I can point to three companies from my days at DARPA that continue to be successful today. They are:x Arbor Networks – Distributed Denial of Service (DDOS) defense technology funded at theUniversity of Michigan. Farnam Jahanian took asabbatical at the university to start the company.x Mazu Networks – Also DDOS and network defense technology funded at MIT. Same story, FransKaashoek and ari Balikrishnan created thecompany and became entrepreneurs. The companywas eventually sold in 2009.x Voltage – Identity-Based Encryption (IBE) technology funded at Stanford University. DanBoneh and his students (Matt Pauker, Rishi Kacker,and Guido Appenzeller) won the Stanford BusinessPlan contest and commercialized the technologywith the help of several venture capitalists.Because of the small amounts of funding the SBIR program at DARPA often goes unused by program managers. It was only late in my tenure that I released the real value of SBIR. I did experience one successful transition from my DARPA SBIR portfolio, which was the acquisition of Mountain Wave, a company in northern Virginia, by Symantec, the anti-virus vendor.In addition to the companies mentioned above, one additional method was used to impact the information technology (IT) market. The CHATS program was focused on improving the security of open source operating systems. As such, if you look at the historic timelines of Linux, FreeBSD, and OpenBSD, you will see noteworthy improvements during the 2001-2004 timeframe that came from the CH ATS program. This also caused some security improvements within the proprietary vendor operating system community at that time. It is clear that using “open source” as a technology transition method can be useful.As mentioned by Dr. Carl Landwehr, the question of “what results have come from the U.S. Government’s investment?” would be an excellent study as we continue to try to improve the “bridge” from Research Land to Vendor Land.IV.T HE D EPARTMENT OF H OMELAND S ECURITYY EARSA.Getting StartedIn June 2003, Jane Alexander, former DARPA Deputy Director contacted me to see if I’d be interested in coming to the new Department of H omeland Security (DH S) Science and Technology (S&T) Directorate and offered me the opportunity to build the Cybersecurity R&D program within DHS S&T from scratch. As my time at DARPA was coming to a close, this was a perfect opportunity. I started at DH S S&T in October 2003.The mission of DHS S&T is to “conduct, stimulate, and enable research, development, test, evaluation and timely transition of homeland security capabilities to federal, state and local operational end-users.” This was ideally suited for the types of things I wanted to do that were different from what I had experienced at DARPA.Because of the strong customer focus on both internal D S components and external critical infrastructure providers, we designed an R&D execution model that included the full spectrum of what we thought was necessary – research, development, test, evaluation, and transition (RDTE&T) – to ensure that completed “research products” could actually be tested in the hands of an operational end-user and potentially result in a commercial product. This model is shown in the graphic below and is affectionately known as the “Circle of Life”.The important components on the graphic include:x Customers – most R&D funded by DHS S&T is inresponse to some customer, either internal to DHS or an external entity or national document x Pre-R&D – workshops, roadmaps, and solicitations(see below) ensure that the funded R&D is focused on the needs identified by the customers x R&D and Supporting Programs – this is where thework gets done and infrastructure is developed to support the R&D programx Post-R&D – this is where DH S S&T is verydifferent from any other government R&D program and provides experimental deployment and outreach activities to assist the researchers in the process of transitioning technology B.Changing Standard SolicitationsOne of the first things I felt was needed was a change in the Broad Agency Announcement (BAA) process at DH S S&T. It was very important to change the model to incorporate technology transition from the start. It has been very interesting to watch the R&D community adapt to this way of thinking and the transition requirement has also helped build partnerships that probably never would have been established otherwise. It has forced academic researchers to work with commercial companies, which is something they have not been accustomed to doing previously. The pertinent text from the DH S S&T BAA is included below:“Proposals shall be structured in phases per one of the three Type classifications described below, depending on thematurity of the proposed technology. The deployment phase sha ll outline the test, eva lua tion, a nd deployment of the technology, a nd sha ll be included a s a n option in the proposal for Type I and II technologies:• Type I (New Technologies)—Proposa ls requestingfunding for new technologies shall have an applied researchphase, a development phase, and a deployment phase, withthe emph sis on rese rch nd development, nd thedeployment pha se included a s a n option. Proposa ls ma yrequest funding for a timefra me not to exceed 36 months, including the deployment phase. • Type II (Prototype Technologies)—Propos a ls requesting funding for more ma ture prototype technologies sha ll ha ve a development pha se a nd a deployment pha se included as an option. Proposals may request funding for a timeframe not to exceed 24 months, including the deployment phase. • Type III (Mature Technologies)—Proposals requesting funding for a ma ture technology sha ll ha ve a deploymentphase only. Proposals may request funding for a timeframe not to exceed 12 months.The objective of this structure is to support immedia tetechnology tra nsition wherever possible, a nd to crea tetr nsition p ths for new c p bilities from the outset.Offerors must a lso include a description in the proposa l oftheir plan for commercializing the technology or other plansfor getting the technology into esta blished tra nsition pa ths. Technology tr a nsition pl a ns th a t include commerci a l pa rtnerships a re preferred, but tra nsition into the open source community is a lso a ccepta ble. The intent is for offerors to provide evidence tha t, a s pa rt of the technica l pla n development, considera tion ha s been given to the ultima te commercia liza tion of the outputs of DHS-funded programs. Such considerations would include expected user ba se, how the technology will be used, a nd how it will tra nsition in to broa d use. Of key importa nce a re the identific a tion of technology diffusion p a ths th a t a re a ppropria te for the type a nd ma turity of the technology involved, and any additional factors that might increase the likelihood of it being commercia lized. Proposa ls will be eva lua ted with other proposa ls tha t a re submitted to the same Type (I-III) classification.”The BAA text was a significant difference from anything the cybersecurity R&D community had experienced previously. DHS S&T has published two BAAs and funded over 30 efforts over the past 6 years. Many of the winning teams consisted of joint academic and industry partners with the industry partner on-board as the technology transition part of the team. There is still plenty of room to improve, but I believe this is an important mechanism for ensuring technologies make it out of the lab and into the marketplace.C.DHS S&T SuccessesH aving completed 6 years at DH S S&T, I decided to review the progress and results of the program and determined that I had been responsible for funding andmanaging over $180 million dollars of research during thepast 6 years. There have been many successes and far fewerfailures, which I attributed to the change to the RDTE&Tmodel. I believe we are doing a much better job of helping the government-funded performer cross the ‘valley of death’. This is still a work in progress, the following technology transitions have occurred within the DHS S&T portfolio:x IronKey, Inc. was funded to develop a secure universal serial bus (USB) device that could provide a more secure environment for data protection. The IronKey device provides secure Web browsing, cryptographic authentication, end point security, self-service password recovery, and secure password management. It can withstand both simple and sophisticated attacks protecting critical information for emergency responders. IronKey won the Government Computer News’ Best of FOSE Award for FY 2007 and is now available to the public as a more secure alternative than standard USB drives. In addition to DHS S&T funding, IronKey received over $20M of venture funding and has gone from the 2 founders to over 100 employees since 2005. xSecure64 was funded to develop technology to make the deployment of Domain Name System Security (DNSSEC) easier. Secure64 developed the DNSSEC Signer, which is intended to fully automate the process of implementing and operating Domain Name System Security (DNSSEC). The DNSSEC Signer appliance is a turnkey “DNSSEC-in-a-box”solution that eliminates most, if not all, of the concerns associated with DNSSEC deployment. These concerns include complexity, cost, correct configurations, andsecurity, and they are all addressed by the DNSSECSigner. The DNSSEC Signer also won the Government Computer News’ Best of FOSE Award for FY 2009 in the security software category. Secure64 has soldproducts to the Department of the Interior, Department of Labor, and Department of Commerce.x The Vulnerability Discovery and Remediation Tool was developed by a research team from StanfordUniversity, Coverity, and Symantec. The CoveritySCAN project uses the developed vulnerabilitydiscovery technology to scan open-source code toidentify security holes that can be fixed before software or operating systems are released. The system stores all identified bugs in a database which is accessible bydevelopers, allowing them to help secure open-sourcesoftware, which is a critical part of the Nation’sinfrastructure. The tool is now available as acommercial product and has over 900 customers across government and private industry.x VIAssist, a visualization toolkit for netflow analytics, was developed by Secure Decisions, Inc. and is available commercially. This technology reduces the time needed to discover flow anomalies by highlighting important information in massive amounts of data. Itwill also present multiple perspectives to allow analysts to view data in new ways and understand cyber attacks in context. This technology is currently being pilotedwithin the US-CERT for use as a visual analysisplatform for network flow data.x The award-winning anti-phishing technology, known as Password Hash (PwdHash), was developedby a research team from Stanford University andselected as the winner in Computerworld magazine'sHorizon Awards competition for cutting-edgetechnologies. Password Hash is a browser plug-in thatis available as open source and can be download forfree to protect user’s personal information. It protectsagainst attempts by phishers to divert Internet users tofraudulent Web sites in order to steal their personal and financial information. Password Hash is now available as a plug-in for all Internet Explorer, Mozilla Firefox,Opera, and Chrome browsers.x The University of Michigan has developed a Botnet Detection and Mitigation Tool. Based on thetheory that no single detection approach willsuccessfully mitigate the threat of botnets, theUniversity of Michigan has developed and deployed asystem that detects bot behavior rather than specific bot symptoms. This more comprehensive approach meansquicker detection and adaptability as attackers attemptto evade the system. This technology is being pilotedby United States Computer Emergency Readiness Team (US-CERT) and is involved in a regional deploymentpilot with the City of Seattle. It will be made availablevia open source.As I mentioned previously, I really didn’t take advantage of the SBIR program while at DARPA. Given that DH S S&T was a new environment and had a much smaller budget, I used the SBIR program as an excellent way to get lots of small technology development efforts funded. I have managed over 40 SBIR Phase I and over 20 SBIR Phase II efforts over the past 6 years and can identify 8 commercial products that have been successfully transitioned. A couple of these successes include:x In 2005 Komoku, developed an advanced Root-Kit Detection Tool. Rootkits are malicious programs(malware) that are designed to take control of acomputer’s operating system at the root level, wherethey can often hide from detection by standard anti-malware software. Komoku’s customers have includeda number of high-security government agencies,including DARPA, the U.S. Navy, and the DOD.Komoku was acquired by Microsoft in March 2008.x Endeavor Systems developed a botnet detection and mitigation tool called the Inline Botnet Extractionand Response System. An extension to their Firstlightproduct line, Endeavor focused on the development of a malware analysis engine and malware signaturedistributor. The tool also integrated the inline botnetextraction capability, analysis engine, and the signaturedistributor. Endeavor Systems was bought by McAfeein January 2009.x Solidcore developed new techniques for malware identification, by extending existing Solidcoretechnology to perform real-time analytics anddissemination of analyses, with the ultimate goal ofresponding to attacks as they occur. Solidcore is aleading provider of dynamic whitelisting technologythat helps enterprises control which applications areinstalled on a computer, resulting in improved ITcompliance, security, and availability. Solidcore waspurchased by McAfee in June 2009.V.C ONCLUDING R EMARKSIn order to achieve the necessary deployment of new innovation, technology transition needs to be a key consideration for all R&D investments. This requires the federal government to move past working models where most R&D programs support at most limited operational evaluations or experiments, most R&D program managers consider their job done with final reports, and most research performers consider their job done with publications.Future R&D activities need to focus on the real goal: technology transition. Current R&D Principal Investigators (PIs) and Program Managers (PMs) aren’t rewarded for technology transition. Academic PIs are rewarded for publications, not transition. Future government-funded R&D programs need to reward government program managers and PIs for transition progress.If we are truly going to transform the “innovation landscape”, then R&D processes need to ensure that we spend program funds on technology transition and have an。