I suspect that the culprit is the expectations you set to be 100% productive 100% of the time.
You see, productivity loves your work products — any kind of work product. BUT it can’t distinguish between work that contributes to your profit and work that distracts you from achieving your goals. It just likes to see you busy.
It’s no wonder that any work-related “free” time you have gets eaten up quickly.