Yes, that is the problem in a nutshell. Duo is great for continuity but not good for proficiency. In its core, Duo is a game. That is both good and bad. So, sort of you have to take the good and discard the bad.

Also since Japanese is one of the hardest language to learn and it takes so long even for people who is studying seriously, it rather important to be smart about it.