On the Calculation of Mutual Information for Channels with Gauss-Markov Noise

Abstract

In this thesis, we study channels with additive Gauss-Markov noise. Such noise models are natural in many contexts and the characterization of the mutual information is unknown. We specifically study the relationship between mutual information and the Minimum Mean Square Error (MMSE) associated with the signal estimation in such channels. The early work of Duncan on channels with additive Brownian motion noise showed that the mutual information over an interval [0,T] of a channel with a general (not necessarily Gaussian) signal in the presence of additive Brownian noise is equal to the integral of the causal MMSE error over the period, where by causal we mean the filtered estimate. Our objective in this thesis is to expand upon this result in the presence of Ornstein-Uhlenbeck noise in the channel. It is shown that the same relation between mutual information and MMSE error holds. Additionally, the result is extended to a more general case in which the stochastic signal is a general function of the past. We derive the results using the machinery of stochastic calculus, specifically the Girsanov theorem and this is of independent interest.