Individual differences in subcortical brain volumes are highly heritable. Previous studies have identified genetic variants that underlie variation in subcortical volumes in adults. We tested whether those previously identified variants also affect subcortical regions during infancy and early childhood. The study was performed within the Generation R study, a prospective birth cohort. We calculated polygenic scores based on reported GWAS for volumes of the accumbens, amygdala, brainstem, caudate nucleus, globus pallidus, putamen, and thalamus. Participants underwent cranial ultrasound around 7 weeks of age (range: 3–20), and we obtained metrics for the gangliothalamic ovoid, a predecessor of the basal ganglia. Furthermore, the children participated in a magnetic resonance imaging (MRI) study around the age of 10 years (range: 9–12). A total of 340 children had complete data at both examinations. Polygenic scores primarily associated with their corresponding volumes at 10 years of age. The scores also moderately related to the diameter of the gangliothalamic ovoid on cranial ultrasound. Mediation analysis showed that the genetic influence on subcortical volumes at 10 years was only mediated for 16.5–17.6% of the total effect through the gangliothalamic ovoid diameter at 7 weeks of age. Combined, these findings suggest that previously identified genetic variants in adults are relevant for subcortical volumes during early life, and that they affect both prenatal and postnatal development of the subcortical regions.