Background: It is unclear to what extent adherence to dietary guidelines may specifically affect visceral fat and liver fat. We aimed to study the association between the Dutch Healthy Diet Index (DHD-index) and total body fat, visceral adipose tissue (VAT) and hepatic triglyceride content (HTGC) in middle-aged men and women. Design: In this cross-sectional study, VAT was assessed by magnetic resonance imaging (MRI) in 2580 participants, and HTGC by proton-MR spectroscopy in 2083 participants. Habitual dietary intake and physical activity were estimated by questionnaire. Adherence to the current Dutch dietary guidelines was estimated by the 2015 DHD-index score based on the thirteen components (vegetables, fruit, wholegrain products, legumes, nuts, dairy, fish, tea, liquid fats, red meat, processed meat, sweetened beverages, and alcohol). The DHD-index ranges between 0 and 130 with a higher score indicating a healthier diet. We used linear regression to examine associations of the DHD-index with VAT and HTGC, adjusted for age, smoking, education, ethnicity, basal metabolic rate, energy restricted diet, menopausal state, physical activity, total energy intake, and total body fat. We additionally excluded the components one by one to examine individual contributions to the associations. Results: Included participants (43% men) had a mean (SD) age of 56 (6) years and DHD-index score of 71 (15). A 10-point higher DHD-index score was associated with 2.3 cm2 less visceral fat (95% CI; −3.5; −1.0 cm2) and less liver fat (0.94 times, 95% CI; 0.90; 0.98). Of all components, exclusion of dairy attenuated the associations with TBF and VAT. Conclusions: Adherence to the dietary guidelines as estimated by the DHD-index was associated with less total body fat, and with less visceral and liver fat after adjustment for total body fat. These findings might contribute to better understanding of the mechanisms underlying associations between dietary habits and cardiometabolic diseases.