Objective: To investigate the relationship between brain volume and disability worsening over ≥3 years in the natural history of primary progressive multiple sclerosis using data from the placebo group of the INFORMS trial (n = 487; clinicaltrials.gov NCT00731692). Methods: Magnetic resonance imaging scans were collected annually. Brain volume loss was determined using SIENA. Patients were stratified by baseline normalized brain volume after adjusting for demographic and disease-burden covariates. Results: Baseline normalized brain volume was predictive of disability worsening: Risk of 3-month confirmed disability progression was reduced by 36% for high versus low baseline normalized brain volume (Cox's model hazard ratio 0.64, P = 0.0339; log-rank test: P = 0.0297). Moreover, on-study brain volume loss was significantly associated with disability worsening (P = 0.012) and was evident in patients with or without new lesions or relapses. Brain volume loss depended significantly on baseline T2 lesion volume (P < 0.0001). Despite low inflammatory activity at baseline (13% of patients had gadolinium-enhancing lesions) and throughout the study (mean 0.5 new/enlarging T2 lesions and 172 mm3 T2 lesion volume increase per year), baseline T2 lesion volume was substantial (mean 10 cm3). Lower normalized brain volume at baseline correlated with higher baseline T2 volume and older age (both P < 0.0001). Interpretation: Baseline brain volume and the rate of ongoing brain atrophy are significantly associated with disability worsening in primary progressive multiple sclerosis. Brain volume loss is significantly related to baseline T2 lesion volume, but partially independent of new lesion activity, which might explain the limited efficacy of anti-inflammatory treatment.