Background: Inappropriate medication use can affect functional independence in older adults. Aims: The aim of the study is to examine associations between potentially inappropriate medication use and Activities of Daily Living (ADL) and Instrumental Activities of Daily Living (IADL) in geriatric rehabilitation inpatients. Methods: A longitudinal, prospective, observational study was undertaken at a teaching hospital. Potentially inappropriate medications (PIMs) and potential prescribing omissions (PPOs) were measured at acute admission, and at admission and discharge from geriatric rehabilitation. Associations between PIM and PPO use and ADL and IADL scores were examined at admission to geriatric rehabilitation, discharge and 3-month post-discharge. Results: A total of 693 inpatients were included. At the 3-month post-discharge, PPOs were associated with lower IADL scores (incident rate ratio = 0.868, 95% CI 0.776–0.972). There were no significant associations between PIMs and PPOs use at admission to geriatric rehabilitation with longitudinal changes of ADLs and IADLs from geriatric rehabilitation admission to 3-month post-discharge Renal PIMs were associated with higher IADL scores at 3-month post-discharge (incidence rate ratio = 1.750, 95% CI 1.238–2.474). At 3-month post-discharge, PPOs involving vaccinations were associated with a lower IADL score (incident risk ratio = 0.844, 95% CI 0.754–0.944). Conclusions: Inappropriate medication use involving PPOs was associated with lower IADL scores at 3-month post-discharge from geriatric rehabilitation but not with ADL scores. Greater attention is needed in reducing PPOs in geriatric rehabilitation inpatients that can potentially impact IADLs. In the community, health professionals need to be vigilant about assessing how older patients’ physical functioning may be affected by inappropriate medication prescribing.