Stochastic Processes. From Applications to Theory

Content: An illustrated guide Motivating examples Lost in the Great Sloan Wall Meeting Alice in Wonderland The lucky MIT Blackjack team The Kruskal's magic trap card The magic fern from Daisetsuzan The Kepler-22b Eve Poisson's typos Exercises Selected topics Stabilizing populations The traps of Reinforcement Casino roulette Surfing Google's wavesPinging hackers Exercises Computational & theoretical aspects From Monte Carlo to Los Alamos Signal processing & Population dynamics The lost equation Towards a general theory The theory of speculation Exercises Stochastic simulation Simulation toolbox Inversion technique Change of variables Rejection techniques Sampling probabilities Bayesian inference Laplace's rule of successions Fragmentation and coagulation Conditional probabilities Bayes' formula The regression formula Gaussian updates Conjugate priors Spatial Poisson point processes Some preliminary results Conditioning principles Poisson-Gaussian clusters Exercises Monte Carlo integration Law of large numbers Importance sampling Twisted distributions Sequential Monte Carlo Tails distributions Exercises Some illustrations Stochastic processes Markov chain models Black-box type models Boltzmann-Gibbs measures The Ising model The Sherrington-Kirkpatrick model The traveling salesman model Filtering & Statistical learning The Bayes formula The Singer's radar model Exercises Discrete time processes Markov chains Description of the models Elementary transitions Markov integral operators Equilibrium measures Stochastic matrices Random dynamical systems Linear Markov chain model Two states Markov models Transition diagrams The tree of outcomes General state space models Nonlinear Markov chains Self-interacting processes Mean field particle models McKean-Vlasov diffusions Interacting jump processes Exercises Analysis toolbox Linear algebra Diagonalisation type techniques The Perron Frobenius theorem Functional analysisSpectral decompositions Total variation norms Contraction inequalities The Poisson equation V-norms Geometric drift conditions V -norm contractions Stochastic analysis Coupling techniques The total variation distance The Wasserstein metric Stopping times and coupling Strong stationary times Some illustrations Minorization condition and coupling Markov chains on complete graphs Kruskal random walk Martingales Some preliminaries Applications to Markov chains Martingales with Fixed terminal values A Doeblin-Ito formula Occupation measures Optional stopping theorems A gambling model Fair games Unfair games Maximal inequalities Limit theoremsTopological aspects Irreducibility and aperiodicity Recurrent and transient states Continuous state spaces Path space models Exercises Computational toolbox A weak ergodic theorem Some illustrations Parameter estimation A Gaussian subset shaker Exploration of the unit disk Markov Chain Monte Carlo methods Introduction Metropolis and Hastings models Gibbs-Glauber dynamics The Propp and Wilson sampler Time inhomogeneous MCMC models Simulated annealing algorithm A perfect sampling algorithm Feynman-Kac path integration Weighted Markov chains Evolution equations Particle absorption models Doob h-processes Quasi-invariant measures Cauchy problems with terminal conditions Dirichlet-Poisson problems Cauchy-Dirichlet-Poisson problems Feynman-Kac particle methodology Mean field genetic type particle models Path space models Backward integration A random particle matrix model A conditional formula for ancestral trees Particle Markov Chain Monte Carlo methods Many-body Feynman-Kac measures A particle Metropolis-Hastings model Duality formulae for many-body models A couple of particle Gibbs samplers Quenched and annealed measures Feynman-Kac models Particle Gibbs models Particle Metropolis-Hastings models Some application domains Interacting MCMC algorithms Nonlinear Filtering models Markov chain restrictions Self-avoiding walks Importance twisted measures Kalman-Bucy Filters Forward Filters Backward Filters Ensemble Kalman Filters Interacting Kalman Filters Exercises Continuous time processes Poisson processes A counting process Memoryless property Uniform random times The Doeblin-Ito formula The Bernoulli process Time inhomogeneous models Description of the models Poisson thinning simulation Geometric random clocks Exercises Markov chain embeddings Homogeneous embeddings Description of the models Semigroup evolution equations Some illustrations A two states Markov process Matrix valued equations Discrete LaplacianSpatially inhomogeneous models Explosion phenomenon Finite state space models Time in homogenous models Description of the models Poisson thinning modelsExponential and geometric clocks Exercises Jump processes A class of pure jump models Semigroup evolution equations Approximation schemes Sum of generators Doob-Meyer decompositionsDiscrete time models Continuous time martingales Optional stopping theorems Doeblin-Ito-Taylor formulae Stability properties Invariant measures Dobrushin contraction properties Exercises Piecewise deterministic processes Dynamical systems basics Semigroup and flow maps Time discretization schemes Piecewise deterministic jump models Excursion valued Markov chains Evolution semigroups Infinitesimal generators The Fokker-Planck equation A time discretization scheme Doeblin-Ito-Taylor formulae Stability properties Switching processes Invariant measures An application to Internet architectures The Transmission Control Protocol Regularity and stability properties The limiting distribution Exercises Diffusion processes Brownian motion Discrete vs continuous time models Evolution semigroups The heat equation A Doeblin-Ito-Taylor formula Stochastic differential equations Diffusion processes The Doeblin-Ito differential calculus Evolution equations The Fokker-Planck equation Weak approximation processes A backward stochastic differential equation Multidimensional diffusions Multidimensional stochastic differential equations An integration by parts formula Laplacian and Orthogonal transformations The Fokker-Planck equation Exercises Jump diffusion processes Piecewise diffusion processes Evolution semigroups The Doeblin-Ito formula The Fokker-Planck equation An abstract class of stochastic processes Generators and carre du champ operators Perturbation formulae Jump-diffusion processes with killing Feynman-Kac semigroups Cauchy problems with terminal conditions Dirichlet-Poisson problems Cauchy-Dirichlet-Poisson problems Some illustrations 1-dimensional Dirichlet-Poisson problems A backward stochastic differential equation Exercises Nonlinear jump diffusion processes Nonlinear Markov processes Pure diffusion models The Burgers equation Feynman-Kac jump type models A jump type Langevin model Mean field particle modelsSome application domains Fouque-Sun systemic risk model Burgers equation A Langevin-McKean-Vlasov model The Dyson equation Exercises Stochastic analysis toolbox Time changes Stability properties Some illustrations Gradient flow processes 1-dimensional diffusions Foster-Lyapunov techniques Contraction inequalities Minorization properties Some applications Ornstein-Uhlenbeck processes Stochastic gradient processes Langevin diffusions Spectral analysis Hilbert spaces and Schauder bases Spectral decompositions Poincare inequality Exercises Path space measures Pure jump models Likelihood functionals Girsanov's transformations Exponential martingales Diffusion models The Wiener measure Path space diffusions Girsanov transformations Exponential change twisted measures Diffusion processesPure jump processes Some illustrations Risk neutral Financial markets Poisson markets Diffusion markets Elliptic diffusions Nonlinear filtering Diffusion observations Duncan-Zakai equation Kushner-Stratonovitch equation Kalman-Bucy Filters Nonlinear diffusion and Ensemble Kalman-Bucy FiltersRobust Filtering equations Poisson observations Exercises Processes on manifolds A review of differential geometry Projection operators Covariant derivatives of vector fields First order derivatives Second order derivatives Divergence and mean curvature Lie brackets and commutation formulae Inner product derivation formulae Second order derivatives and some trace formulae The Laplacian operator Ricci curvature Bochner-Lichnerowicz formula Exercises Stochastic differential calculus on manifolds Embedded manifolds Brownian motion on manifolds A diffusion model in the ambient space The infinitesimal generator Monte Carlo simulation Stratonovitch differential calculus Projected diffusions on manifolds Brownian motion on orbifolds Exercises Parameterizations and charts Differentiable manifolds and charts Orthogonal projection operators Riemannian structures First order covariant derivatives Pushed forward functions Pushed forward vector fields Directional derivatives Second order covariant derivative Tangent basis functions Composition formulae Hessian operators Bochner-Lichnerowicz formula Exercises Stochastic calculus in chart spacesBrownian motion on Riemannian manifolds Diffusions on chart spaces Brownian motion on spheres The unit circle S = S1 _ R2 The unit sphere S = S2 _ R3 Brownian motion on the Torus Diffusions on the simplex Exercises Some analytical aspects Geodesics and the exponential map A Taylor expansion Integration on manifolds The volume measure on the manifold Wedge product and volume forms The divergence theorem Gradient flow models Steepest descent model Euclidian state spaces Drift changes and irreversible Langevin diffusions Langevin diffusions on closed manifolds Riemannian Langevin diffusions Metropolis-adjusted Langevin models Stability and some functional inequalities Exercises Some illustrationsPrototype manifolds The Circle The 2-Sphere The Torus Information theory Nash embedding theorem Distribution manifolds Bayesian statistical manifolds The Cramer-Rao lower bound Some illustrations Boltzmann-Gibbs measures Multivariate normal distributions Some application areas Simple random walks Random walk on lattices Description Dimension 1 Dimension 2 Dimension d >
 3 Random walks on graphs Simple exclusion process Random walks on the circle Markov chain on cycles Markov chain on the circle Spectral decomposition Random walk on hypercubes Description A macroscopic model A lazy random walk Urn processes Ehrenfest model Polya urn model Exercises Iterated random functions Description A motivating example Uniform selection An ancestral type evolution model An absorbed Markov chain Shuffling cards Introduction The top-in-at random shuffle The random transposition shuffleThe riffle shuffleFractal models Exploration of Cantor's discontinuum Some fractal images Exercises Computational & Statistical physics Molecular dynamics simulationNewton's second law of motion Langevin diffusion processes The Schroedinger equation A physical derivation A Feynman-Kac formulation Bra-kets and path integral formalism Spectral decompositions The harmonic oscillator Diffusion Monte Carlo models Interacting particle systems Introduction Contact process Voter process Exclusion process Exercises Dynamic population models Discrete time birth and death models Continuous time models Birth and death generators Logistic processes Epidemic model with immunity Lotka-Volterra predator-prey stochastic model The Moran genetic model Genetic evolution models Branching processes Birth and death models with linear rates Discrete time branching Continuous time branching processes Absorption - death process Birth type branching process Birth and death branching processes Kolmogorov-Petrovskii-Piskunov equations Exercises Gambling, ranking and control The Google page rank Gambling betting systems Martingale systems St. Petersburg martingales Conditional gains and losses Conditional gains Conditional losses Bankroll managements The Grand Martingale The D'Alembert Martingale The Whittacker Martingale Stochastic optimal control Bellman equations Control dependent value functions Continuous time models Optimal stopping Games with Fixed terminal condition Snell envelope Continuous time models Exercises Mathematical finance Stock price models Up and down martingales Cox-Ross-Rubinstein model Black-Scholes-Merton model European option pricing Call and Put options Self-financing portfolios Binomial pricing technique Black-Scholes-Merton pricing model The Black-Scholes partial differential equation Replicating portfolios Option price and hedging computations A numerical illustration Exercises Bibliography Index