Explore First, Exploit Later