Introduction: Pretreatment enlarged lateral lymph nodes (LLN) in patients with locally advanced low rectal cancer are predictive for local recurrences after neoadjuvant (chemo)radiotherapy (n(C)RT) followed by total mesorectal excision (TME). Not much is known of the impact on oncological outcomes when in addition malignant features are present in enlarged LLN. Patients and Methods: A multicenter retrospective cohort study was conducted at five tertiary referral centers in the Netherlands and Australia. All patients were diagnosed with locally advanced low rectal cancer with LLN on pretreatment magnetic resonance imaging (MRI) and underwent n(C)RT followed by TME. LLN were considered enlarged with a short axis of ≥ 5 mm. Malignant features were defined as nodes with internal heterogeneity and/or border irregularity. Outcomes of interest were local recurrence-free survival (LRFS), distant metastatic-free survival (DMFS), and overall survival (OS). Results: Out of 115 patients, the majority was male (75%) and the median age was 64 years (range 26–85 years). Median pretreatment LLN short axis was 7 mm (range 5–28 mm), and 60 patients (52%) had malignant features. After a median follow-up of 47 months, patients with larger LLN (7 + mm) had a worse LRFS (p = 0.01) but no difference in DMFS (p = 0.37) and OS (p = 0.54) compared with patients with smaller LLN (5–6 mm). LLN patients with malignant features had no difference in LRFS (p = 0.20) but worse DMFS (p = 0.004) and OS (p = 0.006) compared with patients without malignant features in the LLN. Cox regression analysis identified LLN short axis as an independent factor for LR. Malignant features in LLN were an independent factor for DMFS. Conclusion: The current study suggests that pretreatment enlarged LLN that also harbor malignant features are predictive of a worse DMFS. More studies will be required to further explore the role of malignant features in LLN.