The answer to the riddle is "the letter H" It is inside the word "house" and also starts the word "outside."