I've always struggled with pumping. With my kids being looked after by my mom when they were (are) babies and me at work, what I did was to get my mom to feed them only 3oz per session. Sometimes she gave them a bit more and if she ran out of stock I just have to play catch up.

With direct latching, I know I do not always produce a lot. I just try to feed them frequently when they are with me. But it never cross my mind that the fussing were from hunger. Haha... I thought they were just trying to settle to sleep etc.

It's been 6 years now, with two 6 months pregnancy breaks in between.
Perhaps, what you need is just a change of mindset - you will always produce enough for your baby.
All the best!