Introduction: The presence of tumor-infiltrating lymphocytes (TILs) is correlated with good prognosis and outcome after (immuno)therapy in triple-negative and HER2-positive breast cancer. However, the role of TILs in luminal breast cancer is less clear. Emerging evidence has now demonstrated that genetic aberrations in malignant cells influence the immune landscape of tumors. Phosphatidylinositol 3-kinase (PI3K) is the most common altered pathway in ER-positive breast cancer. It is unknown whether changes in the PI3K pathway result in a different composition of the breast tumor microenvironment. Here we present the retrospective analysis of a prospective randomized trial in ER-positive breast cancer on the prognostic and predictive value of specific tumor-associated lymphocytes in the context of PI3K alterations. Methods: We included 563 ER-positive tumors from a multicenter trial for stage I to III postmenopausal breast cancer patients, who were randomized to tamoxifen or no adjuvant therapy. The amount of CD8-, CD4-, and FOXP3-positive cells was evaluated by immunohistochemistry and quantified by imaging-analysis software. We analyzed the associations between PIK3CA hotspot mutations, PTEN expression, phosphorylated proteins of the PI3K and MAPK pathway (p-AKT, p-ERK1/2, p-4EBP1, p-p70S6K), and recurrence-free interval after adjuvant tamoxifen or no adjuvant treatment. Results: CD8-positive lymphocytes were significantly more abundant in PIK3CA-mutated tumors (OR = 1.65; 95% CI 1.03-2.68). While CD4 and FOXP3 were not significantly associated with prognosis, patients with tumors classified as CD8-high had increased risk of recurrence (HR = 1.98; 95% CI 1.14-3.41; multivariable model including PIK3CA status, treatment arm, and other standard clinicopathological variables). Lymphocytes were more often present in tumors with increased PI3K downstream phosphorylation. This was most pronounced for FOXP3-positive cells. Conclusion: These exploratory analyses of a prospective trial in luminal breast cancer suggest high CD8 infiltration is associated with unfavorable outcome and that PI3K pathway alterations might be associated with the composition of the tumor microenvironment.