Elements of a Zimun
When at least three people eat a bread meal together, they are obligated in the mitzvah of making a zimun (“invitation”; i.e., a prayer assemblage of three or more). To be eligible to form a zimun it does not suffice for the participants to happen be together at the table at some point during the meal; they must eat with a kvius(established routine) by either beginning or ending the meal together.
The Beginning and the End
Beginning the meal together entails all members eating at least some of the first kezayis (like an olive, i.e., the minimum amount required for bentching—reciting the after-blessing) together—if one or two participants sit down after the first one completed his kezayis, it is not considered beginning together.* (Some poskim are stricter and require the three to actually begin eating in unison.)
Alternatively, they can end the meal together, meaning that even if only two people are together, and the third one joins them later, he can still participate in the zimun by eating at least a kezayis before they begin to bentch. Even if the two are not eating anymore, the third forms a zimun as long that they can still theoretically continue the meal—they are not too full (even with just room for dessert); they have not announced a desire to bentch; they have not decided to conclude their meal (see Halachah #181 on hesach hada’as); and they have yet to wash mayim acharonim (the handwashing ritual before bentching ). Nonetheless, even when they are not required to make a zimun since the third participant arrived after they’ve technically (and halachically) ended their meal, it's a mitzvah to make a new brachah and eat another kezayis of bread to create a zimun, if possible.
If two participants are ready to end the meal and make a zimun while the third person is still eating, he is obligated to stop eating and join the zimun. In case the third person is not willing to participate in joining the zimun, the other two should recite it, and the third person may be counted in the zimun as long as he is in the room and can hear them bentching.
All in it Together
Another condition for a zimun is eating at the same table. If the three did not sit at the same table, but sat within view of each other, and began the meal with the intent of joining in the bentching, it is considered as one table, and suffices for a zimun. In this case of multiple table seating, there is no obligation to make a zimun, but it is still considered a mitzvah. Some, however, interpret the mitzvah as the need to create the right conditions for a proper zimun: making a kvius by eating a kezayis together at one table to meet all the requirements.
Once the requirements for a zimun are met, the participants are obligated to participate, and may not begin bentching prior to the recital of the zimun. This applies to anyone eating together, even those not in the core group of three and to women who are participating in the meal, as well. If there are more than six men, they can create more than one zimun. However, if they are more than ten, but less than twenty, they may not split into separate groups for zimun, so that each participant remains part of the minyan (quorum of at least ten) for bentching.
* For the measurement of kezayis, see Halachah #235 and Halachah #537. For zimun at large gatherings, see Halachah #651.