I mostly agree, however setting your phone number includes the verification process. With that, Google knows for near certain that this is indeed your number.