Background: Participation in regular physical activity is associated with a multitude of health benefits across the life course. However, many people fail to meet PA recommendations. Despite a plethora of studies, the evidence regarding the environmental (physical) determinants of physical activity remains inconclusive. Objective: To identify the physical environmental determinants that influence PA across the life course. Methods: An online systematic literature search was conducted using MEDLINE, ISI Web of Science, Scopus and SPORTDiscus. The search was limited to studies published in English (January 2004 to April 2016). Only systematic literature reviews (SLRs) and meta-analyses (MAs) of observational studies, that investigated the association between physical determinants and physical activity outcomes, were eligible for inclusion. The extracted data were assessed on the importance of determinants, strength of evidence and methodological quality. Results: The literature search identified 28 SLRs and 3 MAs on 67 physical environmental characteristics potentially related to physical activity that were eligible for inclusion. Among preschool children, a positive association was reported between availability of backyard space and outdoor toys/equipment in the home and overall physical activity. The availability of physical activity programs and equipment within schools, and neighbourhood features such as pedestrian and cyclist safety structure were positively associated with physical activity in children and adolescents. Negative street characteristics, for example, lack of sidewalks and streetlights, were negatively associated with physical activity in adults. Inconsistent associations were reported for the majority of reviewed determinants in adults. Conclusion: This umbrella SLR provided a comprehensive overview of the physical environment determinants of physical activity across the life course and has highlighted, particularly amongst youth, a number of key determinants that may be associated with overall physical activity. Given the limited evidence drawn mostly from cross-sectional studies, longitudinal studies are needed to further explore these associations. Registration: PROSPERO CRD42015010616.