Microglia are brain-resident macrophages with trophic and phagocytic functions. Dominant loss-of-function mutations in a key microglia regulator, colony-stimulating factor 1 receptor (CSF1R), cause adult-onset leukoencephalopathy with axonal spheroids and pigmented glia (ALSP), a progressive white matter disorder. Because it remains unclear precisely how CSF1R mutations affect microglia, we generated an allelic series of csf1r mutants in zebrafish to identify csf1r-dependent microglia changes. We found that csf1r mutations led to aberrant microglia density and distribution and regional loss of microglia. The remaining microglia still had a microglia-specific gene expression signature, indicating that they had differentiated normally. Strikingly, we also observed lower microglia numbers and widespread microglia depletion in postmortem brain tissue of ALSP patients. Both in zebrafish and in human disease, local microglia loss also presented in regions without obvious pathology. Together, this implies that CSF1R mainly regulates microglia density and that early loss of microglia may contribute to ALSP pathogenesis. Oosterhof et al. show that colony-stimulating factor 1 receptor (CSF1R) primarily regulates microglia density and not their normal differentiation. In addition, they find widespread depletion of microglia in CSF1R-haploinsufficient zebrafish and leukodystrophy patients, also in the absence of pathology, indicating that microglia depletion may contribute to loss of white matter.