MATH 34600: Information Theory

This is the web page for the Level 3 course 'Information theory' lectured by Professor Oliver Johnson of the School of Mathematics of Bristol University.
Lectures take place in the second half of teaching block 1 (Weeks 7-12).
Monday 1, Tuesday 12 (both in G.09 Fry Building) Thursday 12 (in 2.41 Fry Building)
Maths Cafe timetabled for Tuesdays 4-5, G.16 Fry Building
Drop-in sessions: 11.30-12.30 Mondays. I will be in my office (G.83 Fry Building) if you would like to ask about the course - or send me an email at other times.
A more detailed description of the course including syllabus, suggested reading and unit aims.

Lecture notes and Problem Sheets

Lecture notes (printed copies available in lectures)
Corrections
Problem Sheets (printed copies available in lectures)
This material is copyright of the University of Bristol unless explicitly stated otherwise. It is provided exclusively for educational purposes at the University and is to be downloaded or copied for your private study only.

Books

Core books:
R B Ash. Information Theory, Dover Publications, 1990
D J C Mackay. Information Theory, Inference, and Learning Algorithms, CUP, 2003
T M Cover & J A Thomas. Elements of Information Theory, Wiley Interscience, 1991
Other useful references:
C E Shannon. The Mathematical Theory of Communication (part 1, part 2), Bell Systems Technical Journal, 1948
I Csiszar & J Koerner.Information Theory: Coding Theorems for Discrete Memoryless Systems (2nd ed.), Cambridge University Press, 2011
A Renyi. A diary on information theory, Akademiai Klado, Budapest, 1984
Probability reference
G R Grimmett & D Welsh. Probability: An Introduction, Oxford University Press, 1986

Diversions and links from lectures

Lecture 1: Shannon's models for English
Lecture 2: Shannon's juggling machine
Lecture 3: Hartley function
Lecture 4: Bell Labs Shannon site
Lecture 5: Storage over time
Lecture 6: Cost of data storage over time
Lecture 7: Fortune's Formula
Lecture 8: Photos of Shannon's papers
Lecture 9: Morse code encoding tree and ASCII
Lecture 10: von Neumann anecdote
Lecture 11: Differential entropy
Lecture 12: Information-theoretic Central Limit Theorem
Lecture 13: Shannon and Turing
Lecture 14: questionnaires
Lecture 15: Shannon and crypto
Lecture 16: A Mind at Play: How Claude Shannon Invented the Information Age