A few notes on writing multiplayer games
Dan Kegel
Nov '98
I'm a programmer who has had the pleasure of working on
a few popular multiplayer games written by Activision;
the first two I worked on were Mechwarrior 2: Netmech DOS,
and its followon Mechwarrior 2: Mercnet DOS.
I was mostly responsible for getting the networking up
and running after somebody else designed it.
I ended up writing a DirectPlay clone for DOS, MacOS, Win32,
and Linux. I also learned that there's a lot more to game
networking than DirectPlay functionality.
Mercnet
Mercnet is a 3d giant robot combat simulation, where each player
pilots a giant robot which strides around the world, possibly
with teammates, accomplishing the various objectives of the mission
(inspect an enemy building, blow up an enemy vehicle, etc.)
in the face of often overwhelming opposition.
The game is derived from a role-playing game from FASA, and a lot of the
richness of the game comes from the original.
The successful player learns quickly how to customize their
'Mech to have a good blend of weapons, armor, and other equipment,
and learns the tactics specific to online play (e.g. leading the target
to compensate for network lag, and the manoever called the Circle of Death).
Observations based on experience, in no particular order
- Chat is very important- but it won't be used if it isn't well-designed.
Get onto a few of the successful systems, like Kali, and see how
people use the chat there. Don't fall into the trap of making chat
an afterthought.
- Bullies who enjoy bothering others are far more common that I imagined.
These people's behavior can quickly render a chat area useless.
Chat rooms need a way for people to ignore bullies.
- More subtle forms of harassment in chat rooms can also be a problem,
e.g. impersonation. This bothers the serious league player more than the
casual player. The only way to solve this may be to have a password
server, and make people log in before playing.
- Users love to be able to edit their player names and game names
to indicate their clan or league affiliation, their current kill count,
or their mood. Don't take that away from them.
- Many companies will be tempted to rush multiplayer games out the door
without fully understanding the key problems unique to online gaming.
Games which don't solve these problems well - or which crash - will not
be blockbusters, so it's in game companies' interest to get it right.
Anyone who really knows how to do it right is probably very valuable.
- Players get upset if they fire at somebody, see it hit, but no damage.
This happens in Mercnet because visual effects like explosions are
computed locally, but damage is computed on the target's computer.
This kind of situation happens far more often in the field than in the lab;
it's caused by two different effects: packet loss, which causes the
weapons data to not reach the target computer, and packet latency,
which may cause the weapons data to be misinterpreted on the target
computer because it arrived too late.
- Don't be sensitive to single packet loss. Figure out a way
to compensate for lost weapons packets - either make it clear to the
user the packet didn't get through (the client/server approach),
or use redundancy (you are sending that weapons info out to more
than one computer, so...)
- Manage use of bandwidth carefully. Using more bandwidth
than is really available overloads the network, and
causes increased latency and packet loss. For action games which do
dead reckoning, and which broadcast a fixed size packet at a fixed
interval, the interval should be a function of the average latency;
that way, as the latency (i.e. modem backlog) goes up, the game
adapts, and sends less data; this reduces the network overload, and
thus reduces latency. Budget about a man-month to optimize this.
Better yet, assign a programmer full time to making sure the game
handles packet loss and latency without degrading the user's experience.
- Test with simulated packet loss and latency! It's quite easy to
throw in three lines of code in your game to simulate 10% packet loss on
reception. If your game can't deal well with that level of packet loss,
you are going to have problems in the real world.
- Test with a mix of fast (e.g. cable modem) and slow (e.g. 20kbps modem)
connections in the same game. This is a common situation in the real world.
If your game is not good at adapting to available bandwidth, the slow
interfaces will get more laggy (have increased latency) as the game goes on.
If your game is sensitive to single packet loss, the slow machines will
be the first to start dropping packets, which can cause them to seem
invulnerable if the packets being dropped are weapons packets.
-
If you have a game design that can cover for lost packets,
don't use an ordered reliable packet delivery service like TCP
for anything time-critical.
A single packet retransmission can really bog you
down. Some third-party gaming networks use TCP by default- beware.
- Test! If possible, write a regression test for each interesting
module, and each interesting core behavior of the multiplayer engine.
For instance, come up with a regression test that proves that
you are insensitive to packet loss or latency. If it can be automated,
so much the better.
- The developer should test in the target environment. If the game
is supposed to support eight dialup Internet users on Pentium 133's with
16 megabytes of RAM, the developer should have free access to a bay
of eight of those machines, and should verify himself that the game
can launch and play successfully with the max number of players before
sending the game down to the testers. If the programmer doesn't
do this, failure is almost assured.
- Beware lowered expectations! If the people testing your game
expect a certain problem (e.g. long load time, poor multiplayer accuracy),
they won't report it. Ideally, there should be an experienced programmer
leading the testing effort, so he or she can tell when the emperor has
no clothes.
- Don't send out anything with known crash bugs to beta testers. You
will get no useful information back; all the testers will report about are
the known crashes, even if you ask them not to.
- Get bigger disks. The developer's machine should have as much disk
space as the developer can possibly imagine using, and then some.
Eight gigabytes is about right these days. Same goes for the test machines.
This will cost you peanuts, and will save you juggling time later.
Links