you're not crazy! and congrats on your first developer job.
i quit a decent paying job myself a few years ago because i wanted to make games. everybody called me crazy. now i'm happier than ever.