Stochastic Processes - MA636

Location Term Level Credits (ECTS) Current Convenor 2018-19
Canterbury Autumn
View Timetable
6 15 (7.5) PROF J Zhang


Prerequisite and co-requisite modules
Level 6:
For delivery to students completing Stage 1 before September 2016:
Pre-requisite: MA321 (Calculus and Mathematical Modelling), MA322 (Proofs and Numbers), and either MA323 (Matrices and Probability) and MA306 (Statistics) or MA319 (Probability and Inference for Actuarial Science) and MA326 (Matrices and Computing); MA552 (Analysis), MA553 (Linear Algebra) and either MA629 (Probability and Inference) or MA529 (Probability and Statistics for Actuarial Science 2); or their equivalents.
Co-requisite: None

For delivery to students completing Stage 1 after September 2016:
Pre-requisite: MAST4009 (Probability), MAST4011 (Statistics), MAST4006 (Mathematical Methods 1), MAST4007 (Mathematical Methods 2), either MAST4010 (Real Analysis 1) and MAST4004 (Linear Algebra) or MAST4005 (Linear Mathematics), and MAST5007 Mathematical Statistics; or their equivalents.
Co-requisite: None





Introduction: Principles and examples of stochastic modelling, types of stochastic process, Markov property and Markov processes, short-term and long-run properties. Applications in various research areas.
Random walks: The simple random walk. Walk with two absorbing barriers. First–step decomposition technique. Probabilities of absorption. Duration of walk. Application of results to other simple random walks. General random walks. Applications.
Discrete time Markov chains: n–step transition probabilities. Chapman-Kolmogorov equations. Classification of states. Equilibrium and stationary distribution. Mean recurrence times. Simple estimation of transition probabilities. Time inhomogeneous chains. Elementary renewal theory. Simulations. Applications.
Continuous time Markov chains: Transition probability functions. Generator matrix. Kolmogorov forward and backward equations. Poisson process. Birth and death processes. Time inhomogeneous chains. Renewal processes. Applications.
Queues and branching processes: Properties of queues - arrivals, service time, length of the queue, waiting times, busy periods. The single-server queue and its stationary behaviour. Queues with several servers. Branching processes. Applications.

Marks on this module can count towards exemption from the professional examination CT4 of the Institute and Faculty of Actuaries. Please see for further details.


This module appears in:

Contact hours

48 hours

Method of assessment

80% Examination, 20% Coursework

Indicative reading

Ross, S.M. (1996) Stochastic Processes. New York, Wiley.
Breuer, L. and Baum, D. (2005) An introduction to Queueing Theory and Matrix-Analytic Methods. Springer, Dordrecht.
Jones, P.W. and Smith, P. (2001) Stochastic Processes: An Introduction. London, Arnold.
Karlin, S., Taylor, H.M. (1998) A First Course in Stochastic Processes. 3rd Edition, Academic Press, London.
Ross, S.M. (1970) Applied Probability Models with Optimization Applications. Holden-Day, San Francisco.
Cox, D.R. and Miller, H.D. (1965) The Theory of Stochastic Processes. Chapman & Hall/CRC.

See the library reading list for this module (Canterbury)

See the library reading list for this module (Medway)

Learning outcomes

The intended subject specific learning outcomes.
On successfully completing the level 6 module students will be able to:
1 demonstrate systematic understanding of key aspects of stochastic modelling;
2 demonstrate the capability to deploy established approaches accurately to analyse and solve problems using a reasonable level of skill in calculation and manipulation of the material in the following areas: random walks, discrete and continuous time Markov chains, queues and branching processes;
3 apply key aspects of stochastic modelling in well-defined contexts, showing judgement in the selection and application of tools and techniques.

The intended generic learning outcomes.
On successfully completing the level 6 module students will be able to:
1 manage their own learning and make use of appropriate resources;
2 understand logical arguments, identifying the assumptions made and the conclusions drawn;
3 communicate straightforward arguments and conclusions reasonably accurately and clearly and communicate technical material competently;
4 manage their time and use their organisational skills to plan and implement efficient and effective modes of working;
5 solve problems relating to qualitative and quantitative information;
6 make competent use of information technology skills such as online resources (Moodle);
7 communicate technical material competently;
8 demonstrate an increased level of skill in numeracy and computation;
9 demonstrate the acquisition of the study skills needed for continuing professional development.

University of Kent makes every effort to ensure that module information is accurate for the relevant academic session and to provide educational services as described. However, courses, services and other matters may be subject to change. Please read our full disclaimer.