Objective: To investigate the relationships between structural damage in the whole brain, the temporal lobes, and the frontal lobes and cognitive decline at old age. The authors hypothesized that widespread brain damage as quantified using magnetization transfer imaging (MTI) is related to global cognitive decline, whereas regional damage to the temporal lobes is related to memory impairment, and regional damage to the frontal lobes is related to executive dysfunctioning. Methods: Cognitive function of 22 patients with probable AD, 13 patients with mild cognitive impairment (MCI), and 28 elderly controls was assessed using an extensive neuropsychological test battery. Structural damage in the whole brain, the temporal lobes, and the frontal lobes was estimated using volumetric MTI analysis. Associations between MTI measures and neuropsychological tests were investigated using Pearson correlation analysis. Results: MTI measures of the whole brain, as well as the temporal and the frontal lobes, were strongly associated with global cognitive deterioration and impairment in memory, orientation, language, praxis, gnosis, and executive functioning. However, there were no specific cognitive correlates of regional brain damage to the temporal and frontal lobes. Conclusions: Using whole brain volumetric magnetization transfer imaging, the authors demonstrated that cognitive decline in patients with mild cognitive impairment and AD is associated with widespread structural brain damage. As there were no specific relationships between regional brain damage and impairment of specific cognitive functions, pathology in AD and mild cognitive impairment is much more generalized than was expected.