A Statistical Assessment of a Process To Evaluate the Commercial Success Of Inventions

Abstract

In over twenty years of operations the Canadian Innovation Centre has evaluated, through its Inventor's Assistance Program (IAP), the commercial potential of over 12,000 early stage inventions. Prior to 1989, the Canadian Innovation Centre (CIC) used a version of the Preliminary Innovation Evaluation System (PIES), developed by Gerald Udell at the Oregon Innovation Center in 1974, to evaluate the commercial potential of early stage inventions. Since 1989, the CIC has used a modified version of PIES in their evaluation process. I first estimate the ability of this program's analysts to forecast the probability that an invention will become commercialized. I also estimate a model to predict the probability that an invention will become commercialized based on the IAP's evaluation of several underlying early stage characteristics of the invention. I find that such a statistical model is based on a limited set of variables and predicts future commercial success almost as well as the IAP's forecast of success. I then use factor analysis to determine if the ratings provided by the CIC evaluation service are representative of the underlying theoretical variable structure of PIES or their modified version. Factor analysis is applied to two distinct periods that are separated by a distinct alteration of the theoretical variable structure in 1989. While I find that the factor analysis provides evidence that the post 1989 theoretical structure does provide interpretation of some of the dimensions in the ranking variables, when a combination of the post 1989 and the pre 1989 structure are examined interpretability of the extracted factors is significantly improved. Finally, I compare the model estimated on the underlying early stage characteristics with a model estimated on the extracted factors. When the predictive accuracy of the two models is compared, I find that both procedures produce models that predict almost equally well. The models and the IAP perform better than R&D managers' predictions of their own R&D projects' successes. The thesis provides recommendations for the assessment and maintenance of evaluation models for inventions, innovations and R&D projects.