Jump to content

User:Jim.Callahan,Orlando/Stan Modeling Language

From Wikipedia, the free encyclopedia
  • Comment: Please note that Stan (software) already exists, so please use this draft to improve the article already on the main encyclopaedia. JMHamo (talk) 00:04, 28 September 2015 (UTC)
Stan
Original author(s)[Stan Development Team]
Developer(s)[Stan Development Team]
Initial release2012 (2012)
Stable release
v1.2.0 / March 6, 2013; 11 years ago (2013-03-06)
Written inStan(C++) and RStan(R)
Operating systemUnix/Linux, Mac OS/X & Windows
PlatformStan is platform independent C++. Stan runs at command line. RStan interface written in R and includes a copy of Stan.
Size24.9 MB compressed source
Available inEnglish
TypeStatistical package
LicenseStan(New BSD) and RStan(GPLv3)
Websitemc-stan.org

Stan (Statistical Software)

[edit]

Stan Modeling Language is open source (new BSD) statistical software for Monte Carlo simulation and Bayesian analysis of statistical models (including mixture models, Bayesian regression models and meta analysis -- see chapters 10,11 and 12 of the Stan Modeling Language User's Guide and Reference Manual)..[1]

Stan was developed by the Stan Development Team in 2012 [2] and is similar to the BUGS (Bayesian inference Using Gibbs Sampling) family of software (WinBUGS, OpenBUGS and JAGS), but with a different sampler (and a slightly different language for expressing models). RStan is the R statistical package interface to Stan. [3]

Stan is named in honor of Stanislaw Ulam[4] co-author with Nicholas Metropolis of the first paper (Metropolis and Ulam, 1949) [5] describing the Monte Carlo method . The Monte Carlo method was developed and first used to mathematically model the physics (neutron diffusion) of a thermonuclear weapon (Hydrogen bomb, H-bomb, also known as the "super" advocated by Edward Teller) to test whether the designs ("classic" and Teller-Ulam) for a fusion weapon were practical enough for further development.

Evolution of Markov Chain Monte Carlo Algorithm (MCMC)

[edit]
Main article: Metropolis-Hastings algorithm

The algorithm was eventually named after Nicholas Metropolis, who (worked with Stan Frankel who organized the human computers in the Theoretical Division at Los Alamos laboratory of the Manhattan Project) was in 1953 also co-author of a second paper, "Equation of State Calculations by Fast Computing Machines" along with a team that included Arianna W. Rosenbluth, Marshall N. Rosenbluth, Augusta H. Teller, and Edward Teller which proposed the algorithm for the specific case of the Boltzmann distribution; [6] and W. K. Hastings who extended the Metropolis algorithm to the more general case in a 1970 Biometrika article. [7] [8]

The Metropolis-Hastings algorithm was rediscovered in 1984, as the Gibbs Sampling algorithm, which later (1991) was the key idea behind David Spiegelhalter's BUGS (Bayesian inference Using Gibbs Sampling), family of statistical software (WinBUGS, OpenBUGS and JAGS). [9]

Main article: Gibbs sampling

Gibbs sampling "is named after the physicist Josiah Willard Gibbs, in reference to an analogy between the sampling algorithm and statistical mechanics (physics). The algorithm was described by brothers Stuart Geman and Donald Geman in 1984, some eight decades after the death of Gibbs. [10] The Gibbs sampling algorithm generates an instance from the distribution of each variable in turn, conditional on the current values of the other variables. It can be shown (see, for example, Andrew Gelman et al. 1995) that the sequence of samples constitutes a Markov chain, and the stationary distribution of that Markov chain is just the sought-after joint probability distribution."

Stan Modeling Language is similar in intent to BUGS (Bayesian inference Using Gibbs Sampling), but with a different sampler; Stan uses the No-U-Turn sampler (NUTS), a variant of Hamiltonian Markov Chain Monte Carlo (MCMC) also known as Hybrid Monte Carlo (HMC).

Main article: Hybrid Monte Carlo

Hamiltonian Monte Carlo (HMC) "differs from the Metropolis–Hastings algorithm by reducing the correlation between successive states sampled by using a Hamiltonian evolution between states and additionally by targeting states with a higher acceptance criteria the observed probability distribution. This causes it to converge more quickly to the absolute probability distribution. It was devised by Simon Duane, A.D. Kennedy, Brian Pendleton and Duncan Roweth in 1987. [11]"

In 2009, Sai Hung Cheung and James L. Beck proposed improving Hybrid Monte Carlo by using " "algorithmic differentiation” Rall 1981; Kagiwada et al. 1986 , in which a program code for sensitivity analysis gradient calculation can be created alongside the original program for an output analysis to form a combined code for both output analysis and sensitivity analysis."[12]

According to the Stan Modeling Language User's Guide and Reference Manual (page 121):

HMC "Sampling is based on simulating the Hamiltonian of a particle with a starting position equal to the current parameter values and an initial momentum (kinetic energy)generated randomly. The potential energy at work on the particle is taken to be the negative log (unnormalized) total probability function defined by the model. In the usual approach to implementing HMC, the Hamiltonian dynamics of the particle is simulated using the leapfrog integrator, which discretizes the smooth path of the particle into a number of small time steps called leapfrog steps. ...If the user specifies the number of leapfrog steps (i.e., chooses to use standard HMC), that number of leapfrog steps are simulated. If the user has not specified the number of leapfrog steps, the No-U-Turn sampler (NUTS) will determine the number of leapfrog steps adaptively (Hoffman and Gelman, 2011)." [13]

One of the motivations for switching to the Hamiltonian Monte Carlo (HMC) was the difficulty encountered by Matthew R Schofield (University of Kentucky) in his multilevel time-series regression of climate on tree-ring measurements[14]

"The models we wanted to fit turned out to be a challenge for current general purpose software to fit. A direct encoding in BUGS or JAGS can grind these tools to a halt. Matt Schofield found his multilevel time-series regression of climate on tree-ring measurements wasn’t converging after hundreds of thousands of iterations. ...Matt Schofield managed to fit the tree-ring data using a hand-coded implementation of HMC, finding it converged in a few hundred iterations. HMC appeared promising but was also problematic in that the Hamiltonian dynamics simulation requires the gradient of the log posterior. ...That’s when we discovered reverse-mode algorithmic differentiation" [15]

Using Stan and RStan

[edit]

Stan runs from the command line on Unix/Linux, Apple Mac OS/X and Windows operating systems, as well as from inside the R statistical package using the RStan interface package.

Stan requires hardware powerful enough to build and execute the models. Ideally, that will be a 64-bit computer with at least 4GB of memory and multiple processor cores. [16]

Although any text editor can be used to create Stan modeling language text files, there is a package for Emacs. "Emacs Stan mode allows syntax highlighting and automatic indentation of Stan models in the Emacs text editor." [17]

The Stan Modeling Language includes the Stanc compiler command, which compiles the Stan user's model (a text file) in two steps, first Stanc translates the user's model to templated C++, which is compiled by a [C++] compiler (such as the ones in GCC or Clang) to a platform-specific executable file.

As part of the compilation to C++, the Stanc command uses its own C++ algorithmic differentiation packages to compute the numeric value of the gradient (vector of partial derivatives representing the slope of a hyperplane) of the log of the probability density function. Gradients are required for the Hamiltonian Monte Carlo and NUTS sampler. [18]

RStan includes a copy of Stan; installing RStan will automatically set up Stan. Additional packages for interfacing to Python and MATLAB are also planned. [19]

Stan, like WinBUGS, OpenBUGS, and JAGS, has an active community, which you can access via the user’s mailing list ; see [1] for information on subscribing and posting and to look at archives. [2]. [20]

Programming language

[edit]

Stan and RStan are open source projects developed and maintained by the Stan Development Team. The source code of Stan is written in the C++ programming language and is dependent on the Eigen (C++ library) and Boost C++ libraries. The libraries, compilers, integrated development environments (IDEs) and other development tools are available as open source. Stan has been developed using two portable, open-source C++ compilers, GNU Compiler Collection and Clang, both of which run under and generate code for Unix/Linux, Apple Mac OS/X and Windows.

The compilation of Stan uses the open source Make build automation software utility on all platforms. [21]

RStan, the R interface package for Stan is written in R, and uses the Stan C++ Application Programming Interface (API). The RTools package, a bundle of tools used to build R, can also be used to build Stan.

Stan has an active developer community, which you can access via the developer’s mailing list (which is read by the Stan Development Team); see [1] for information on subscribing and posting and to look at archives. [2]. [12]

Differences from BUGS

[edit]

In addition to the different software license and greater range of operating systems, Stan is similar to BUGS (Bayesian inference Using Gibbs Sampling), but with a different language for expressing models and a different sampler for sampling from the model's posterior distributions. Another difference is BUGS is interpreted; Stan is compiled in two steps, first a model is is translated to templated C++ and then to a platform-specific executable. Stan uses its own C++ algorithmic differentiation packages to compute the gradient of the log of the probability density function (up to a proportion). Gradients are required during the Hamiltonian dynamics simulations within the leapfrog integration algorithm of the Hamiltonian Monte Carlo and NUTS samplers. BUGS computes the log density but not its gradient. The differences between Stan and BUGS are detailed in Appendix C "Stan for BUGS Users" of the Stan Modeling Language User's Guide and Reference Manual available on the Stan website. [22]

References

[edit]

The Stan specific content is copied (and sometimes paraphrased to conform to Wikipedia usage and links) from the "Stan Modeling Language User’s Guide and Reference Manual Stan version 1.20" which is distributed under the Creative Commons Attribute 3.0 Unported License (CC BY 3.0) which grants the right "to Distribute and Publicly Perform Adaptations.". The CC BY 3.0 LICENSE requires inclusion of "the [Resource Identifier (URI) for, this License] with every copy of the Work You Distribute or Publicly Perform.": [[3]]

The context and references to W. K. Hastings (Metropolis-Hastings algorithm) and David Spiegelhalter (developer of BUGS) were inspired by the popular history of Bayesian statistics, "The Theory that Would Not Die."

  1. ^ Stan Development Team (2013-03-06). "Stan Modeling Language User's Guide and Reference Manual Stan version 1.20" (PDF). Stan website. Retrieved 10 March 2013.
  2. ^ Smith, David. "RStan: Fast, multilevel Bayesian modeling in R". BLOG: Revolutions. Revolution Analytics. Retrieved 3/27/2013. {{cite web}}: Check date values in: |accessdate= (help)
  3. ^ Carpenter, Bob. "A Stan is Born". Blog: Statistical Modeling, Causal Inference, and Social Science. Retrieved 3/11/2013. {{cite web}}: Check date values in: |accessdate= (help)
  4. ^ "Stan Modeling Language User's Guide and Reference Manual". page x
  5. ^ Metropolis, Nicolas; Ulam, Stanislaw (1949). "The Monte Carlo method". Journal of the American Statistical Association 44:335-341. 44 (247): 335–341. doi:10.1080/01621459.1949.10483310. PMID 18139350. {{cite journal}}: Unknown parameter |unused_data= ignored (help)
  6. ^ Metropolis, N.; Rosenbluth, A.W.; Rosenbluth, M.N.; Teller, A.H.; Teller, E. (1953). "Equations of State Calculations by Fast Computing Machines". Journal of Chemical Physics. 21 (6): 1087–1092. Bibcode:1953JChPh..21.1087M. doi:10.1063/1.1699114.
  7. ^ Rosenthal, Jeffrey (March 2004). "W.K. Hastings, Statistician and Developer of the Metropolis-Hastings Algorithm". Retrieved 2009-06-02.{{cite web}}: CS1 maint: date and year (link)
  8. ^ Hastings, W.K. (1970). "Monte Carlo Sampling Methods Using Markov Chains and Their Applications". Biometrika. 57 (1): 97–109. doi:10.1093/biomet/57.1.97. JSTOR 2334940. Zbl 0219.65008.
  9. ^ McGrayne, Sharon Bertsch. (2011). The Theory That Would Not Die: How Bayes' Rule Cracked The Enigma Code, Hunted Down Russian Submarines, & Emerged Triumphant from Two Centuries of Controversy. New Haven: Yale University Press. 13-ISBN 9780300169690/10-ISBN 0300169698; OCLC 670481486 The Theory That Would Not Die, p. 226., p. 226, at Google Books
  10. ^ Geman, S.; Geman, D. (1984). "Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images". IEEE Transactions on Pattern Analysis and Machine Intelligence. 6 (6): 721–741. doi:10.1109/TPAMI.1984.4767596. PMID 22499653.
  11. ^ Duane, Simon; Kennedy, A.D.; Pendleton, Brian J.; Roweth, Duncan (3). "Hybrid Monte Carlo". Physics Letters B. 195 (2): 216–222. doi:10.1016/0370-2693(87)91197-X. {{cite journal}}: Check date values in: |date= and |year= / |date= mismatch (help); Unknown parameter |month= ignored (help)
  12. ^ Sai Hung Cheung (April 2009). "Bayesian Model Updating Using Hybrid Monte Carlo Simulation with Application to Structural Dynamic Models with Many Uncertain Parameters" (PDF). Journal of Engineering Mechanics: 243–255. doi:10.1061/ ASCE 0733-9399 2009 135:4 243) (inactive 2023-08-02). Retrieved 3/27/2913. {{cite journal}}: Check |doi= value (help); Check date values in: |accessdate= (help); Unknown parameter |coauthors= ignored (|author= suggested) (help)CS1 maint: DOI inactive as of August 2023 (link) CS1 maint: date and year (link)
  13. ^ "Stan Modeling Language User's Guide and Reference Manual", page 121
  14. ^ Schofield, Mathew. "Model Sensitivity in Climate Reconstruction Using Tree-Ring Data". ASA JSM 2012 Online Program. Retrieved 3/27/2013. {{cite web}}: Check date values in: |accessdate= (help)
  15. ^ "Stan Modeling Language User's Guide and Reference Manual". pages v-vi
  16. ^ "Stan Modeling Language User's Guide and Reference Manual", "Appendix B: Installation and Compatibility", page 243
  17. ^ "Stan Modeling Language User's Guide and Reference Manual", "Chapter 34: Contributed Modules", page 240
  18. ^ "Stan Modeling Language User's Guide and Reference Manual", "Appendix C: Stan for Users of BUGS"
  19. ^ "Stan Modeling Language User's Guide and Reference Manual", "Appendix C: Stan for Users of BUGS", page 256
  20. ^ "Stan Modeling Language User's Guide and Reference Manual", "Appendix C: Stan for Users of BUGS", page 263
  21. ^ "Stan Modeling Language User's Guide and Reference Manual", "Appendix C: Stan for Users of BUGS", page 254
  22. ^ "Stan Modeling Language User's Guide and Reference Manual", "Appendix C: Stan for Users of BUGS"
[edit]

Category:Free Bayesian statistics software Category:Numerical programming languages Category:Monte Carlo software

Retrieved from "https://en.wikipedia.org/w/index.php?title=User:Jim.Callahan,Orlando/Stan_Modeling_Language&oldid=1168383178"