Is our software any good?
Is our work on it making it better or worse?
Can we quantify how much it has changed?
Engineering organizations face these questions constantly, and know there are not any easy answers. Luckily, we can draw on well known risk assessment techniques from epidemiologists and actuaries. We will explore the historic development of these ideas from studying the effects of smoking to setting maritime cargo insurance rates in babylon, ancient greece, and victorian england. This talk will focus on how Cloudera measures and compares quality of our software.
A useful as observational methods of risk assessment are, they are also easy to misuse and misinterpret. We will discuss some choice examples of misuse and abuse of analytic methods, with examples from Newton’s Principia to particle physicists, and hopefully avoid our own charlatanry in the future.