Women’s physical functioning declines with age and the rate of decline increases with age, but substantial disparities exist in trajectories over time. To inform development of interventions to optimise physical functioning across the adult life span, the aim is to explore which lifestyle and socio-economic position (SEP) factors contribute to disparities in physical functioning across the adult life span in women. Younger (born 1973–1978, n = 14,247), middle-aged (born 1946–1951, n = 13,715) and older (born 1921–1926, n = 12,432) participants from the Australian Longitudinal Study on Women’s Health completed six questionnaires between 1996 and 2012 at approximate 3-year intervals. Physical functioning was measured with a 10-item subscale of the Short-Form Health Survey (score 1–100). Relationships between age and physical functioning were modelled using spline regression, stratified by baseline categories of physical activity, alcohol intake, smoking status, level of education, managing on income and index of neighbourhood socio-economic disadvantage for area. Multivariable models excluding one of the six factors were compared with models including all six factors to examine the relative importance of each factor. Women with unhealthy lifestyles (inactive, smokers or risky alcohol intake) and lower SEP had lower levels of physical functioning and more rapid declines across the adult life span. The variables with the greatest relative contribution to the models for physical functioning differed by age cohort: i.e. education and physical activity in younger women, managing on income and physical activity in middle-aged women and physical activity in older women. For optimal physical functioning, socio-economic factors seemed particularly important in younger and middle-aged women, while physical activity seemed important at all ages.