There has not been much study of risk profiles in older people according to different levels of practice in physical activity. For this reason, the aim of this research was to evaluate whether the elements that influence the quality of life and factors such as gender and education can predict the level of physical activity in the physically active elderly population. The Fernández–Ballesteros quality of life questionnaire and the WHO International Physical Activity Questionnaire were applied to a sample of 397 people with a mean age of 69.65 years (SD = 4.71). The results revealed the following predictive factors of practicing a low level of physical activity (p < 0.05): being a woman; having a low educational level; and low scores in activity and leisure and in functional skills. In conclusion, gender, education, functional skills, activity and leisure, and health are elements of quality of life that predict the level of physical activity performed by the elderly, where it is necessary to use leisure activities (visiting friends, playing games, running errands) as an indirect way to increase participation in physical activity.