Esophageal adenocarcinoma (EAC) is a disease with dismal treatment outcomes. Response to neoadjuvant chemoradiation (CRT) varies greatly. Although the underlying mechanisms of CRT resistance are not identified, accumulating evidence indicates an important role for local antitumor immunity. To explore the immune microenvironment in relation to response to CRT we performed an in-depth analysis using multiplex immunohistochemistry, flow cytometry and mRNA expression analysis (NanoString) to generate a detailed map of the immunological landscape of pretreatment biopsies as well as peripheral blood mononuclear cells (PBMCs) of EAC patients. Response to CRT was assessed by Mandard’s tumor regression grade (TRG), disease-free- and overall survival. Tumors with a complete pathological response (TRG 1) to neoadjuvant CRT had significantly higher tumor-infiltrating T cell levels compared to all other response groups (TRG 2–5). These T cells were also in closer proximity to tumor cells in complete responders compared to other response groups. Notably, immune profiles of near-complete responders (TRG 2) showed more resemblance to non-responders (TRG 3–5) than to complete responders. A high CD8:CD163 ratio in the tumor was associated with an improved disease-free survival. Gene expression analyses revealed that T cells in non-responders were Th2-skewed, while complete responders were enriched in cytotoxic immune cells. Finally, complete responders were enriched in circulating memory T cells. preexisting immune activation enhances the chance for a complete pathological response to neoadjuvant CRT. This information can potentially be used for future patient selection, but also fuels the development of immunomodulatory strategies to enhance CRT efficacy.