Purpose To investigate the discrepancy between patients with multiple sclerosis (MS) without atrophy who have already developed cognitive impairment and patients with MS with atrophy who have preserved cognitive function. Materials and Methods This retrospective imaging study, with imaging acquired between 2008 and 2012, included 332 patients with MS (106 men and 226 women; mean age, 48.1 years; range, 23.0-72.5 years) and 96 healthy control participants. Cognitive impairment was defined as cognitive performance of z less than -1.5 compared with that in control participants in greater than or equal to two cognitive domains. Atrophy was defined as cortical and deep gray matter volumes of z less than -1.5 compared with that in control participants. White matter lesions were assessed with T2-imaging, tract fractional anisotropy (ie, integrity) with diffusion MRI, and regional centrality (ie, importance within network) with functional MRI. Within each atrophy group, patients with cognitive impairment and preserved cognitive function were compared and regression analyses were performed to predict cognitive impairment. Results A total of 132 of 328 patients with MS had no atrophy; of these, 42 of 132 (32%) had cognitive impairment. Cognitive impairment in patients without atrophy was predicted by level of education (Wald test, 11.63; P < .01) and posterior cingulate centrality (Wald test, 6.82; P < .01). A total of 65 of 328 patients with MS had atrophy; of these, 49 of 65 (75%) had cognitive impairment. Cognitive impairment in patients with atrophy was predicted by white matter tract fractional anisotropy (Wald test, 4.89; P = .03) and posterior cingulate centrality (Wald test, 7.19; P < .01). Conclusion Cognitive impairment was related to white matter damage, but only in patients with MS with atrophy. In patients without atrophy, a lower level of education was most important for cognitive impairment. Posterior cingulate cortex showed functional abnormalities in all MS groups with cognitive impairment, regardless of atrophy.