Seminar on Probabilistic Programming : Topics
Each student will choose one topic from the list below. There are more topics than students and hence not all will be covered. To guarantee sufficient coverage of the different aspects some topics are written in boldface -- all of these should be chosen by someone. The topics written in italics require knowledge of somewhat more specialized (and perhaps difficult) concepts, such as computability or probability theory -- you should only pick such a topic if you feel conformtable reading the papers and understand them well enough.
The topics are split into four categories. The first category is about fundamentals of probabilistic programming, the second covers individual languages and approaches, and the third category lists topics presented more from a statistical perspective instead of computer science. Finally, the fourth category covers applications of probabilistic programming.
Each topic lists one or two papers and is coupled with some keywords for quick screening of interesting topics. The papers linked here are good starting points for covering the topic, but typically the presentation will cover also other material. For some of the topics (for example, Church) there exists massive amount of material so selecting what to cover is important part of the seminar work.
Foundations
1. Semantics of probabilistic programming
Papers: Gretz et al. Conditioning in Probabilistic Programming
Keywords: semantics, theory
2. Program analysis
Papers: Chaganty et al. Efficiently Sampling Probabilistic Programs via Program Analysis
Keywords: program analysis, importance sampling, inference efficiency
3. Automatic differentiation
Papers: Wingate et al. Nonstandard Interpretations of Probabilistic Programs for Efficient Inference ; Baydin et al. Automatic differentiation in machine learning: a survey
Keywords: Automatic differentiation, gradient-based optimization
4. Computability of probabilistic programs
Papers: Ackerman et al. On the computability of conditional probability ; Freer and Roy Posterior distributions are computable from predictive distributions”
Keywords: measure theory, probability theory
5. Probabilistic logic
Papers: de Raedt and Kimmig: Probabilistic Programming Concepts”
Keywords: logic, exact inference, model counting/SAT, MCMC
Languages
6. Probabilistic C
Papers: Paige and Wood. A Compilation Target for Probabilistic Programming Languages
Keywords: C, POSIX, sequential Monte Carlo
7. Church
Papers: Goodman et al. Church: a language for generative models, model repository
Keywords: Scheme/Lisp
8. Venture
Papers: Mansinghka et al. Higher-order probabilistic programming with programmable inference
Keywords: Scheme/Lisp
9. BLOG
Papers: Milch et al. Blog: Probabilistic models with unknown objects
Keywords: first-order logic
10. Anglican
Papers: Wood et al. A New Approach to Probabilistic Programming Inference”
Keywords: Scheme, sequential Monte Carlo, Venture
11. Tabular
Papers: Gordon et al. Schema-driven probabilistic programming”
Keywords: databases, graphical models, factor graphs
12. Picture
Papers: Kulkarni et al. Probabilistic programming for scene perception
Keywords: graphics, rendering, MCMC
13. Figaro
Papers: Pfeffer: Object-oriented probabilistic programming
Keywords: Scala, object-oriented programming, MCMC
14. FACTORIE
Papers: McCallum: Probabilistic Programming via Imperatively Defined Factor Graphs
Keywords: Scala, undirected graphical models, MCMC
15. BayesDB
Papers: Mansinghka et al. BayesDB: A probabilistic programming system for querying the probable implications of data
Keywords: SQL, Monte Carlo, universal models, CRP
16. QUERY
Papers: Freer et al. Towards common-sense resoning via conditional simulation: Legacies of Turing in artificial intelligence
Keywords: Turing machine, inductive reasoning, cognitive science, probability theory
Statistics approaches
17. Statistical software for probabilistic programming
Papers: Carpenter et al. Stan: A Probabilistic Programming Language ; Salvatier et al. Probabilistic programming in Python using PyMC3
Keywords: HMC, generative models, statistics
18. Bayesian optimization
Papers: Snoek et al. Practical Bayesian optimization of machine learning algorithms; Shahriari et al. Taking the Human Out of the Loop: A Review of Bayesian Optimization
Keywords: Gaussian processes
Applications
19. Probabilistic program induction
Papers: Lake et al. Human-level concept learning through probabilistic program induction
Keywords: human learning, cognitive science
20. Gaussian process memoization
Papers: Schaechtle et al. Probabilistic emulators with Gaussian Process Memoization
Keywords: Gaussian processes, Venture
21. Automatic statistician
Papers: Lloyd et al. Automatic Construction and Natural-Language Description of Nonparametric Regression Models; http://automaticstatistician.com/index/
Keywords: Data science, probabilistic modelling
Printer-friendly version