That sounds great! It's important to eat healthy in combination with exercise. Constantly making small steps in the right direction will lead to lasting improvements. Some people try to "crash diet" to lose weight quickly, but they usually give up after a short period of time. That's why it's most beneficial to slowly stop eating unhealthy foods while introducing nutritious foods to your diet and making them a part of your lifestyle.
Without getting into matters such as GMOs, pesticides, and gluten, you can absolutely find cost-conscious fruits and vegetables that won't break the bank.
Frozen vegetables (like peas, broccoli, and green beans) are inexpensive and sometimes cheaper than fresh (depending on the season). Otherwise in the produce section of the grocery store you'll find carrots, onions, potatoes (healthier than pasta), bananas, apples, oranges, and sweet potatoes, among numerous others.
It's almost always better to eat things that are minimally processed.
Otherwise, if you ever find yourself sitting around (watching TV or on the computer) for long periods of time, take breaks to walk around the house (maybe clean up any mess while doing so). In turn, you'll become a healthy, happier, more productive person (with maybe a cleaner house).
RE: Why i want to start eating healthy - Why fruits and veggies are good for you!