Yeah, it’s to protect them from disease. In almost all circumstances a place with tap water from a municipal source is fine.
Premature infants might be advised to only get sterile water for a bit as an extra precaution, and people might also hold off a little longer on well water.
The US has a water system effectively comparable to the ones across Europe, FYI. That includes lead levels, since it wasn’t just the US that used lead pipes.
In most circumstances lead pipes are safe to replace with different materials as part of routine maintenance. It’s only very notable incidents where things go wrong that have driven a push for greater haste, since it highlighted the consequences of things going wrong.