The decisive event which established the discipline of information theory, and brought it to immediate worldwide attention, was the publication of Claude E. Shannon’s classic paper “A Mathematical Theory of Communication” in the Bell System Technical Journal in July and October 1948.
In this revolutionary and groundbreaking paper, the work for which Shannon had substantially completed at Bell Labs by the end of 1944, Shannon for the first time introduced the qualitative and quantitative model of communication as a statistical process underlying information theory, opening with the assertion that”The fundamental problem of communication is that of reproducing at one point, either exactly or approximately, a message selected at another point.”
With it came the ideas of
- the information entropy and redundancy of a source, and its relevance through the source coding theorem;
- the mutual information, and the channel capacity of a noisy channel, including the promise of perfect loss-free communication given by the noisy-channel coding theorem;
- the practical result of the Shannon–Hartley law for the channel capacity of a Gaussian channel; and of course
- the bit – a new way of seeing the most fundamental unit of information.
