Focusing on Following Directions