Yes, the planned speakers should all have at least been informed and invited. They should also be confirmed and added to the session by the time the Call for Participation (CfP) deadline expires (please note: each speaker needs their own speaker account, see FAQ "How do I add co-speakers?").
If you organize a panel it is your responsibility that all speakers add information to their profile and manage these themselves – see also: person responsible for session. Only propose a panel that can actually be realised in that constellation of participants.