When NOT to Learn Scales