Rules and Exceptions: A Delicate Balance