But when Youtube shares the key with me/my client the first time, is that also encrypted?

Here’s an explanation of what happens during the initial TLS handshake.

…if ISP automated the process of gathering keys and decrypting web traffic for a certain site with them for all users, would that work for them?

Not sure this is exactly what you’re asking, but there’s the concept of forward secrecy for defending recorded encrypted traffic from future key compromises.