Vrbik Jan - Informal Introduction to Stochastic Processes with Maple

Informal Introduction to Stochastic Processes with Maple — read online for free the complete book (whole text) full work

Below is the text of the book, divided by pages. System saving the place of the last page read, allows you to conveniently read the book "Informal Introduction to Stochastic Processes with Maple" online for free, without having to search again every time where you left off. Put a bookmark, and you can go to the page where you finished reading at any time.

Light

Font size:

↓

↑

Georgia Tahoma Arial Verdana

Reset

Interval:

↓

↑

Bookmark:

Make

1234567...73

Jan Vrbik and Paul Vrbik Universitext Informal Introduction to Stochastic Processes with Maple 2013 10.1007/978-1-4614-4057-4_1 Springer Science+Business Media, LLC 2013

1. Introduction

Jan Vrbik 1 and Paul Vrbik 2

(1)

Department of Mathematics, Brock University, St Catharines, Ontario, Canada

(2)

Department of Computer Science, The University of Western Ontario, London, Ontario, Canada

Abstract

A stochastic (a fancy word for random) process is a collection (often infinite, at least in principle) of random variables , labeled by a parameter (say) t , which represents time. The random variables are usually denoted by X ( t ) when t has a continuous scale of real values and X t when t is restricted to integers (e.g., day 1, day 2).

A stochastic (a fancy word for random) process is a collection (often infinite, at least in principle) of random variables , labeled by a parameter (say) t , which represents time. The random variables are usually denoted by X ( t ) when t has a continuous scale of real values and X t when t is restricted to integers (e.g., day 1, day 2).

Example 1.1 (Trivial Stochastic Process). A random independent sample from a specific distribution of infinite size, that is, X 1, X 2, X 3, , is the simplest example of a stochastic process.

A more typical stochastic process will have individual random variables correlated with one another.

Stochastic processes are of four rather distinct categories, depending on whether the values of X t and of t are of a discrete or continuous type. The individual categories are as follows.

1.1

Both X t and t Scales are Discrete

Example 1.2 (Bernoulli Process). Flipping a coin repeatedly (and indefinitely). In this case, X 1, X 2, X 3, are the individual outcomes (the state space consists of1 and 1, to be interpreted as losing or winning a dollar).

Example 1.3 (Cumulative Bernoulli Process). Consider the same Bernoulli process as in Example , where Y 1, Y 2, Y 3, now represent the cumulative sum of money won so far (i.e., Y 1= X 1, , , ). This time the Y values are correlated (the state space consists of all integers).

Example 1.4 (Markov Chains). These will be studied extensively during the first part of the book (the sample space consists of a handful of integers for finite Markov chains and of all integers for infinite Markov chains).

X t Discrete, t Continuous

Example 1.5 (Poisson Process). The number of people who have entered a library from time zero until time t . X ( t ) will have a Poisson distribution with a mean of t ( being the average arrival rate), but the X are not independent (Fig. 6.1 for a graphical representation of one possible realization of such a process the sample space consists of all nonnegative integers).

Example 1.6 (Queuing Process). People not only enter but also leave a library (this is an example of an infinite-server queue; to fully describe the process, we need also the distribution of the time a visitor spends in the library). There are also queues with one server, two servers, etc., with all sorts of interesting variations.

Both X t and t Continuous

Example 1.7 (Brownian Motion). Also called diffusion a tiny particle suspended in a liquid undergoes an irregular motion due to being struck by the liquids molecules. We will study this in one dimension only, investigating issues such as, for example, the probability the particle will (ever) come back to the point from which it started.

X t Continuous, t Discrete

Example 1.8 (Time Series). Monthly fluctuations in the inflation rate, daily fluctuations in the stock market, and yearly fluctuations in the Gross National Product fall into the category of time series. One can investigate trends (systematic and seasonal) and design/test various models for the remaining (purely random) component (e.g., Markov, Yule). An important issue is that of estimating the models parameters.

In this book we investigate at least one type of each of the four categories, namely:

Finite Markov chains, branching processes, and the renewal process (4);

Poisson process, birth and death processes, and the continuous-time Markov chain (Chaps. 58);

Brownian motion ();

Autoregressive models ().

Solving such processes (for any finite selection of times t 1, t 2, , t N ) requires computing the distribution of each individual X ( t ), as well as the bivariate distribution of any X ( t 1), X ( t 2) pair, trivariate distribution of any X ( t 1), X ( t 2), X ( t 3) triplet, and so on. As the multivariate cases are usually simple extensions of the univariate one, the univariate distributions of a single X ( t ) will be the most difficult to compute.

Yet, depending on the type of process being investigated, the mathematical techniques required are surprisingly distinct. We require:

• All aspects of matrix algebra and the basic theory of difference equations to handle finite Markov chains;
• A good understanding of function composition and the concept of a sequence-generating function to deal with branching processes and the renewal theory;
• A basic (at least conceptually) knowledge of partial differential equations (for );
• Familiarity with eigenvalues of a square matrix to learn how to compute a specific function of any such matrix (for ); and, finally
• Calculus ().

In an effort to make the book self-contained,we provide a brief overview of each of these mathematical tools in the chapter appendices.

We conclude this section with two definitions:

Definition 1.1 (Stationary).

A process is stationary when all the X t have the same distribution,and also: for each ,all the ( X t , X t +) pairs have the same bivariate distribution,similarly for triplets,etc.

Example 1.9. Our queueing process can be expected to become stationary (at least in the t limit,i.e., asymptotically),but the cumulative-sum process is nonstationary.

Definition 1.2 (Markovian property) .

A process is Markovian when

or, more generally, to compute the probability of an event in the future, given a knowledge of the past and present,one can discard information about the past without affecting the answer. This does not imply X i +1 is independent of,for example X i 1, X i 2.

Example 1.10. The stock market is most likely non-Markovian (trends), whereas the cumulative-sum process has a Markovian property.

The main objective in solving a specific stochastic-process model is to find the joint distribution of the processs values for any finite selection of the t indices. The most basic and important of these is the univariate distribution of X t , for any value of t , from which the multivariate distribution of several X t (usually) easily follows.

References

M. S. Bartlett. An Introduction to Stochastic Processes, with Special Reference to Methods and Applications . Cambridge University Press, Cambridge/New York, 1980.

Next page

1234567...73

Light

Font size:

↓

↑

Georgia Tahoma Arial Verdana

Reset

Interval:

↓

↑

Bookmark:

Make

Similar books «Informal Introduction to Stochastic Processes with Maple»

Look at similar books to Informal Introduction to Stochastic Processes with Maple. We have selected literature similar in name and meaning in the hope of providing readers with more options to find new, interesting, not yet read works.

Tamoghna Ghosh

Practical Mathematics for AI and Deep Learning: A Concise yet In-Depth Guide on Fundamentals of Computer Vision, NLP, Complex Deep Neural Networks and Machine Learning

Daniel W. Stroock

An Introduction to Markov Processes

Durrett

Essentials of Stochastic Processes

Gerard-Michel Cochard

Introduction to stochastic processes and simulation

Carlos A. Braumann

Introduction to Stochastic Differential Equations with Applications to Modelling in Biology and Finance

William G. Gray

Introduction to Environmental Modeling

Nicolas Lanchier

Stochastic Modeling

Andreas E. Kyprianou

Fluctuations of Lévy Processes with Applications: Introductory Lectures

Pierre Brémaud

Fourier Analysis and Stochastic Processes

Barry L. Nelson

Stochastic Modeling: Analysis and Simulation

Howard G. Tucker

A Graduate Course in Probability

Erhan Cinlar

Introduction to Stochastic Processes

Discussion, reviews of the book Informal Introduction to Stochastic Processes with Maple and just readers' own opinions. Leave your comments, write what you think about the work, its meaning or the main characters. Specify what exactly you liked and what you didn't like, and why you think so.