Patients with a diffuse glioma may experience cognitive decline or improvement upon resective surgery. To examine the impact of glioma location, cognitive alteration after glioma surgery was quantified and related to voxel-based resection probability maps. A total of 59 consecutive patients (range 18–67 years of age) who had resective surgery between 2006 and 2011 for a supratentorial nonenhancing diffuse glioma (grade I–III, WHO 2007) were included in this observational cohort study. Standardized neuropsychological examination and MRI were obtained before and after surgery. Intraoperative stimulation mapping guided resections towards neurological functions (language, sensorimotor function, and visual fields). Maps of resected regions were constructed in standard space. These resection cavity maps were compared between patients with and without new cognitive deficits (z-score difference >1.5 SD between baseline and one year after resection), using a voxel-wise randomization test and calculation of false discovery rates. Brain regions significantly associated with cognitive decline were classified in standard cortical and subcortical anatomy. Cognitive improvement in any domain occurred in 10 (17%) patients, cognitive decline in any domain in 25 (42%), and decline in more than one domain in 10 (17%). The most frequently affected subdomains were attention in 10 (17%) patients and information processing speed in 9 (15%). Resection regions associated with decline in more than one domain were predominantly located in the right hemisphere. For attention decline, no specific region could be identified. For decline in information speed, several regions were found, including the frontal pole and the corpus callosum. Cognitive decline after resective surgery of diffuse glioma is prevalent, in particular, in patients with a tumor located in the right hemisphere without cognitive function mapping.