In a multicenter setting, we applied voxel‐based methods to different structural MR imaging modalities to define the relative contributions of focal lesions, normal‐appearing white matter (NAWM), and gray matter (GM) damage and their regional distribution to cognitive deficits as well as impairment of specific cognitive domains in multiple sclerosis (MS) patients. Approval of the institutional review boards was obtained, together with written informed consent from all participants. Standardized neuropsychological assessment and conventional, diffusion tensor and volumetric brain MRI sequences were collected from 61 relapsing‐remitting MS patients and 61 healthy controls (HC) from seven centers. Patients with ≥2 abnormal tests were considered cognitively impaired (CI). The distribution of focal lesions, GM and WM atrophy, and microstructural WM damage were assessed using voxel‐wise approaches. A random forest analysis identified the best imaging predictors of global cognitive impairment and deficits of specific cognitive domains. Twenty‐three (38%) MS patients were CI. Compared with cognitively preserved (CP), CI MS patients had GM atrophy of the left thalamus, right hippocampus and parietal regions. They also showed atrophy of several WM tracts, mainly located in posterior brain regions and widespread WM diffusivity abnormalities. WM diffusivity abnormalities in cognitive‐relevant WM tracts followed by atrophy of cognitive‐relevant GM regions explained global cognitive impairment. Variable patterns of NAWM and GM damage were associated with deficits in selected cognitive domains. Structural, multiparametric, voxel‐wise MRI approaches are feasible in a multicenter setting. The combination of different imaging modalities is needed to assess and monitor cognitive impairment in MS.