Is it possible to study information theory while studying a first course on probability?

1,166

I wouldn't recommend taking both courses simultaneously. Most in the field would consider probability theory a prerequisite and not a co-requisite for information theory.

When it comes to texts, you might take a look at David Applebaum's Probability and Information: An Integrated Approach. It's one of the very few that treats both subjects together - though still sequentially. (It also has an elegant and more sophisticated approach to measure theory for probability without getting into the excessive rigor of analysis that is typically presented.)

For probability texts, you're likely better off with those from Leon-Garcia (the most popular text in many electrical engineering departments) or older inexpensive Dover Publications like those from Paul Pfeiffer or Alfred Renyi which were specifically written (and include information theoretic examples/related material) for students who expected to move into information theory afterwards.

Share:
1,166

Related videos on Youtube

Fraïssé
Author by

Fraïssé

Updated on August 29, 2020

Comments

  • Fraïssé
    Fraïssé about 3 years

    I'm currently taking a course on intro to probability.

    The course is not mathematically rigorous and does not invoke theorems from real analysis, etc. The course covers all the way from basic notions permutations, combinations, to pdf, cdf, till just before law of large numbers.

    I wonder if I would also learn information theory at the same time. For that, I will need a simple set of notes or (even better) an introductory text that goes from the ground up. I think something just under C.E. Shannon's thesis and above basic notions in probability will help.

    Has anyone taken a similar course or been in a similar situation? Any literature recommendation will be helpful.