The three main constituent groups involved in organizing the Games are the International Olympic Committee (IOC), the National Olympic Committees (NOCs), and the International Sports Federations (IFs), not the government of the host country.
While the government of the host country has a role to play in supporting the bid of their city in hosting the Games, it does not take part in the Olympic Host Contract and does not issue the invitations to NOCs to participate. Invitations to the NOCs are issued only by the IOC.