Although ductal carcinoma in situ (DCIS) is a non-obligate precursor to ipsilateral invasive breast cancer (iIBC), most DCIS lesions remain indolent. Hence, overdiagnosis and overtreatment of DCIS is a major concern. There is an urgent need for prognostic markers that can distinguish harmless from potentially hazardous DCIS. We hypothesised that features of the breast adipose tissue may be associated with risk of subsequent iIBC. We performed a case–control study nested in a population-based DCIS cohort, consisting of 2658 women diagnosed with primary DCIS between 1989 and 2005, uniformly treated with breast conserving surgery (BCS) alone. We assessed breast adipose features with digital pathology (HALO ®, Indica Labs) and related these to iIBC risk in 108 women that developed subsequent iIBC (cases) and 168 women who did not (controls) by conditional logistic regression, accounting for clinicopathological and immunohistochemistry variables. Large breast adipocyte size was significantly associated with iIBC risk (odds ratio (OR) 2.75, 95% confidence interval (95% CI) = 1.25–6.05). High cyclooxygenase (COX)-2 protein expression in the DCIS cells was also associated with subsequent iIBC (OR 3.70 (95% CI = 1.59–8.64). DCIS with both high COX-2 expression and large breast adipocytes was associated with a 12-fold higher risk (OR 12.0, 95% CI = 3.10–46.3, P < 0.001) for subsequent iIBC compared with women with smaller adipocyte size and low COX-2 expression. Large breast adipocytes combined with high COX-2 expression in DCIS is associated with a high risk of subsequent iIBC. Besides COX-2, adipocyte size has the potential to improve clinical management in patients diagnosed with primary DCIS.