Whenever demand increases, premiums inevitably go up as well. That makes stacking more challenging, as you can't get as many ounces for the same amount you previously paid. There are ways around this that can allow you to increase your holdings with little to no cost to do it.
Vintage bullion commands a premium of its own, regardless of what spot price and regular bullion premiums are doing. If you're willing to part with some of them, you could easily pick up an extra ounce or two of silver or fractional gold by finding someone willing to trade, or even striking a deal with your local coin store.
Low Mintage Bullion
These low mintage pieces can fetch a very high premium even over vintage bullion, if you're willing to part with it. Extremely low mintage pieces will often appreciate substantially in price, just months after initial release, which means you could easily sell it or trade it in for more weight to add to your stack.
Sometimes governments experience unrest, wars, supply shortages, etc., which can severely limit the available supply of bullion made by that country's respective mint. This causes the demand, and the premiums on them, to increase as a result. Selling them at peak demand can be a fast way to gain extra ounces of generic silver and gold.
Older Sovereign Coins
Using the US as an example, 90% silver stopped being made for circulation after 1964. The same can be said for many other countries. This means this older currency is limited. When it was cheaper, a lot of it typically was melted down and turned into modern bullion. If you have this or older sovereign gold coins, they might be in higher demand and could also be sold or traded for more modern bullion.
The Goal
Regardless of what you choose to sell or trade in, the goal should be to use that extra premium to put it towards the lowest premium gold and silver possible. This maximizes the amount of bullion you can add to your stack. The added benefit is that because the bulk of the transaction is a conversion, spot price will have a minimal impact. If spot goes lower, it means you can average your cost lower in the future. Whatever you choose to do, make sure it fits in with your budget and also your long-term plans for your bullion stack.