You don’t have to delete your Gmail account right away. Keep it around to search the history if you need to. You can set up an auto-forward rule in Gmail to forward all email to your new address, then you can reply from your new address. You can take it a step further and include a please note my email address has changed in your signature. Just like moving away from Google, moving away from Gmail can be done in baby steps.