The overflow need to be filled with water?

Yes, it is important to include the overflow tank or bottle as integral with the radiator and coolant system as a whole. They all intermix. Actually, the level in the overflow tank is a true measure of the correct amount to keep your coolant at. There are two marks on the overflow: full hot and full cold. The coolant system on all modern cars is a sealed system and therefore should never require adding, save a very small amount through evaporation within the overflow tank- the only part that is open to atmosphere.

So once you get all the air out of the system that is possible, both in the engine and in the radiator, you have a full level everywhere, including the overflow tank where it should always remain somewhere between the full hot and full cold marks.

You are right that the overflow collects the heated radiator coolant. But it is due to the expansive properties of coolant-or water- that it expands when heated, (but not boiling-that is very bad news of a different sort). Your radiator cap is designed to open at a specific pressure, just enough to allow coolant to flow into the overflow. As the engine and coolant cools, it contracts, the level drops in the radiator, and a natural vacuum there will siphon the overflow tank back into the radiator until it is full again. That is why you have the full hot and full cold marks.
I will shut off the lecture now. I hope you have liked my answer. If not, I give you permission to kill me now- if I may so jest.