I've been in a habit for decades now of not really structuring my spare time. If there is an event, I might schedule that, but otherwise, I don't really have an after-work schedule.
Society doesn't expect you to have one, but I think it's the only way of achieving certain goals that no one cares about but you.
I tell myself I have a need to relax, but the fact that I don't immediately fall asleep when I go to bed tells me otherwise. In other words, I'm being lazy.