A common misunderstanding amongst new bitcoin users is that their bitcoins actually reside on their computer. In order to understand where bitcoins actually reside, it is necessary to grasp three concepts: a ledger, private key encryption and bitcoin addresses.
The ledger
A ledger is just a book of transactions. A transaction log. The book has pages, and each row on the page describes the date, source, destination and amount transferred. Auxiliary columns may be present, but aren't the focus of this article.
The bitcoin system relies on a global, public ledger called the blockchain. In a sense, a ledger is a chain of pages, each page appearing sequentially. Likewise, the blockchain is composed of pages, or blocks. Each page in a ledger will generally have the same number of transactions, precisely because the number of rows that fits on a page is fixed. This is not the case with a block. Any number of transactions may appear in a block.
The blockchain represents the entire history of bitcoin transactions. The first block is known as the genesis block. New pages are added to the bitcoin ledger approximately every 10 minutes. The process by which pages are added is a collective problem solving endeavour. The group or individual which solves the problem is then awarded 25 bitcoins (an amount which decreases over time). The details of this process are particularly complicated, let's address that another day.
If you possess any bitcoins now, you could determine the page in which they came into existence by looking at the blockchain.
Private key encryption
Private key encryption is a mouthful to say, especially if you've never heard of it before. There's a good chance you have created an account on some site somewhere. Generally the process is the same: choose a username and a password. Sometimes you can use your email address as your username. If so, then understanding private key encryption really isn't that big of a leap.
Just as you have a username and password, you have a public key and a private key. A difference is that the public key can be computed from the private key. Exactly how this occurs (it's mathematically complex) isn't important, but just worth understanding that the two are related to each other. You cannot however generate a private key from a public key. This is what allows you to share your public key.
What does a private key look like? Here's an example:
818bfca298181abc59001e3f87ff7e81b9310872a21eef4fc683096f4de83c70
If you used this as a password, it would be a fairly strong one, but a bit difficult to remember. If we convert the above private key into a public key, what does that look like?
1LpiMNtNR18UEJNBxCZx3p32tAkWsAGuPJ
This public key representation is actually a bitcoin address.
Bitcoin addresses
When you want to send bitcoins to someone, they must provide you with a bitcoin address. If you want to receive bitcoins, you'll need one as well. Acquiring one is not difficult. The bitcoin client you are using should have a means for generating one. If you are using an online wallet, it will have that functionality built into it.
Suppose you send 5 bitcoins to the following bitcoin address / public key:
1LpiMNtNR18UEJNBxCZx3p32tAkWsAGuPJ
What happens? Once the transaction is verified and a page is generated, the blockchain (the global public ledger) now associates those 5 bitcoins with the above address. The only way to access those bitcoins is with the equivalent private key. Without the private key, there is no way to access the coins.
When the coins are sent to the address above, the transaction is signed using your private key. The bitcoin network will only verify such transactions that are properly signed by the correct private key. The signing process releases the association the coins have with your private key. After being released, they are then assigned to the new address, the address you are sending to.
As a result of the above, bitcoins do not exist on your computer. Rather, what exists are public and private keys. Technically speaking, even they don't need to exist. A private key can be written down on a piece of paper, and the public key generated from it without ever touching a computer or a network. Coins can then be sent to that public key. When you're ready, you can import the private key into the bitcoin client you are using and reassign the coins to another address.