Recommended Viewing: Halo Reach Network Design GDC Talk

Interesting network talk about Halo Reach’s network design, Bungie’s most recent game before Destiny, by David Aldridge. You can view it here, and I break down the bullet points in the blog post below.

  • They use a decimal priority system about what information goes into packets, how often they need to go, ect. 3 factors that prioritized information calculated from player distance & frustrum
  • They talk about situations like throwing a grenade, kind of basic, but little things like player response being faster when the player throws grenades.
  •  What happened when their armor lock mechanic didn’t work due to latency so they introduced a measured round trip time to the activation of their shields to align the activation of the mechanic as opposed to some arbitrary animation delay.
  • Assassinations – how they used the camera transition to hide the lag. They had issues bringing entities close to each other and then teleporting them into the direction they needed to fix.
  • Removing game mechanics entirely improved player perception sometimes, despite pissing off designers.
  • Extensive testing, even in a cabin in the woods where a tester was getting into crazy game states due to absurd latency
  • Wanting a feature so badly to work that “optimizing” only gained 1% efficiency while introducing tons of bugs.
  • Tracking tools (Profiling, Videos, Playtests) – their profiler could record player’s session with graphs that show replication based on type (event, state, ect), inspect specific event or state replications, inspect robjects even in gameplay, ect. (Per object bandwidth use as well, as well as how much bandwidth an object WANTS to use but doesn’t) After every frame, they injected debug data about packets into their video playback (Films) analyzing network performance per player even offline.
  • Playtests for network performance once a month with traffic shaping tools – force groups of players into laggy situations with laggy networking, ask for feedback without them knowing of the conditions, with a controller button. The button would inject a time stamp on the Films so playback can jump directly to player tested issues. (this was huge for them apparently as it also helped fix game mechanics for gameplay progs. Subtle issues with the game get reported, such as grenades doing too much splash damage)
  • During Films, point the camera at objects, press a button, inspect all network properties and packets coming to or from that object. Rewind, retest, different perspectives, with Films
  • Reduced 80% > bandwidth use from Halo 3 to Halo Reach, all from a tool made in 6 weeks~
  • Halo 3 had 50% positions/velocities/orientations & 20% player control & 20% weapon related & 10% misc data. But the issue wasn’t optimizing the heavy hitters, it was optimizing the higher level stuff FIRSTE.G host->client control removed data that was duplicated in object state, removed data for stuff client didn’t need to know, optimized encoding (60% reduction bandwidth) 2 weeks effort
  • Identify issues before fixing, bandwidth not always obvious: Idle grenades had high priority values due to a Halo 3 bug fix, grenades were classed as “Equipment” even if they were not being used (idle grenade equipment on the floor waiting to be picked up)
  •  Fake friction on hills were made so that interpolation for players going down hills would make sense. Designers specify the slope for objects. Saved 10% bandwidth on loose objects
  • Ragdoll networking sync the initial state of the ragdoll  – ragdolls need to be networked because bullets get ~blocked~ by ragdolls. Can’t have desync ragdolls blocking bullets differently for clients. Humping is a signature move, without synced dolls, you have broken humping.
  • Damage with Machine Gun (15x a second bullets) were not optimized for the calculations. The fix: allow client prediction of some damage
  • Periodic update of game stats taking priority over gameplay traffic (limit stats data to less than 10% of each packet fix)
  • Lag artefacts from objects that entered “panic” mode from far far away because they were disturbed (limit the panic)
  • As long as you’re not memcopying vec 3 into packets, you’re probably fine optimization – just look at other issues (ie the host->client removing of data fix)
  • Migrating hosts can migrate in the middle of the game but needs good detection Didn’t go into how….
  • Regular internal playtests with traffic shaping, consider full time network testers even early. Only job to stress the networking of the game. Slam the game. Keeps the game play staff in check too from doing crazy things.
  • Whats next (Destiny) – reduce boilerplate code, extensions of replication protocols to support low use cases like state machine sync

 

Leave a Reply

Your email address will not be published. Required fields are marked *