The objective of the study was to investigate the timing of central nervous system tissue atrophy in MS by evaluating longitudinal retinal volume changes in a broadly representative cohort with disease duration across the entire arc of disease. In this longitudinal study, 135 patients with MS and 16 healthy reference subjects underwent spectral-domain optical coherence tomography (OCT) at baseline and 2 years later. Following OCT quality control, automated segmentation of the peripapillary retinal nerve fiber layer (pRNFL), macular ganglion cell–inner plexiform layer (mGCIPL) and macular inner nuclear layer (mINL) was performed. Generalized estimation equations were used to analyze longitudinal changes and associations with disease duration and clinical measures. Participants had a median disease duration at baseline of 16.4 years (range 0.1–45.4). Nearly half (44 %) of the MS patients had previously experienced MS-related optic neuritis (MSON) more than 6 months prior. The MS patients demonstrated a significant decrease over 2 years of the pRNFL (−1.1 µm, 95 % CI 1.4–0.7, p < 0.001) and mGCIPL (−1.1 µm, 95 % CI −1.4 to −0.8, p < 0.001). This thinning was most pronounced early in the course of disease. These findings were irrespective of previous episodes of MSON. No consistent pattern of change was observed for the mINL (−0.03 µm, 95 % CI −0.2 to 0.2, p = 0.795). This longitudinal study demonstrated that injury of the innermost retinal layers is found in MS and that this damage occurs most rapidly during the early stages of disease. The attenuation of atrophy with longer disease duration is suggestive of a plateau effect. These findings emphasize the importance of early intervention to prevent such injury.