This is the way I understand it.

The government doesn't even own the money. The Central Bankers do. They create it from nothing and lend it to the government at interest. At the point it is put into circulation, it has the highest purchasing power. By the time you and I get it, it will be worth less due to inflation.

When you deposit it in the bank, you become a creditor to the bank for whatever amount you have in deposits. If the bank has financial trouble, you will be treated just as a creditor.

Besides, all the $$ are in digital form these days, and there is a big move to do away with cash altogether. There simply is not enough physical cash to cover all deposits.