Probability management

The discipline of probability management communicates and calculates uncertainties as data structures that obey both the laws of arithmetic and probability, while preserving statistical coherence. The simplest approach is to use vector arrays of simulated or historical realizations and metadata called Stochastic Information Packets (SIPs). A set of SIPs, which preserve statistical relationships between variables, is said to be coherent and is referred to as a Stochastic Library Unit with Relationships Preserved (SLURP). SIPs and SLURPs allow stochastic simulations to communicate with one another. For example, see Analytica (Wikipedia), Analytica (SIP page), Oracle Crystal Ball, Frontline Solvers, and Autobox.

The first large documented application of SIPs involved the exploration portfolio of Royal Dutch Shell in 2005 as reported by Savage, Scholtes, and Zweidler, who formalized the discipline of probability management in 2006. The topic is also explored at length in.

Vectors of simulated realizations of probability distributions have been used to drive stochastic optimization since at least 1991. Andrew Gelman described such arrays of realizations as Random Variable Objects in 2007.

A recent approach does not store the actual realizations, but delivers formulas known as Virtual SIPs that generate identical simulation trials in the host environment regardless of platform. This is accomplished through inverse transform sampling, also known as the F-Inverse method, coupled to a portable pseudo random number generator, which produces the same stream of uniform random numbers across platforms. Quantile parameterized distributions (QPDs) are convenient for inverse transform sampling in this context. In particular, the Metalog distribution is a flexible continuous probability distribution that has simple closed form equations, can be directly parameterized by data, using only a handful of parameters. An ideal pseudo random number generator for driving inverse transforms is the HDR generator developed by Douglas W. Hubbard. It is a counter-based generator with a four-dimensional seed plus an iteration index that runs in virtually all platforms including Microsoft Excel. This allows simulation results derived in R, Python, or other readily available platforms to be delivered identically, trial by trial to a wide audience in terms of a combination of a few parameters for a Metalog distribution accompanied by the five inputs to the HDR generator.

In 2013, ProbabilityManagement.org was incorporated as a 501(c)(3) nonprofit that supports this approach through education, tools, and open standards. Executive Director Sam Savage is the author of The Flaw of Averages: Why we Underestimate Risk in the Face of Uncertainty and is an adjunct professor at Stanford University. Harry Markowitz, Nobel Laureate in Economics, was a co-founding board member. The nonprofit has received financial support from Chevron Corporation, General Electric, Highmark Health, Kaiser Permanente, Lockheed Martin, PG&E, and Wells Fargo Bank. The SIPmath 2.0 Standard supports XLSX, CSV, and XML Formats. The SIPmath 3.0 Standard uses JSON objects to convey virtual SIPs based on the Metalog Distribution and HDR Generator.