DEVELOPER'S BLOG

Killing Bugs

Some "behind the scenes information" about the bug fixing process



Bug fixing is an important part of game development. It's a part which is not very attractive to do, because it's most often not very creative, but it is necessary - without it, no product can be released.


This blog entry aims to tell you a little about how the bug fixing process is organized within Bohemia Interactive. Some of the practices described here are a common industry practice, some may be specific to how our company is doing things.


There are several different phases in a product's life, and in each of them bug fixing is handled in a slightly different way. One flexible formalization you can use at bug selection is to view it as an optimization problem. When a bug is fixed, you get some bonus from this fix (or you can view it on the other hand, that an unfixed bug causes a penalty). Fixing a bug costs you something - time, money, work ... and sometimes the cost may be not obvious: sometimes a fix which is simple on a programmers side has a significant gameplay impact and needs to be tested extensively.


Fixing a Bug


First let us take a look at fixing an individual bug. This is usually done in several steps



  • Read the bug report

  • Read it again to understand it :)

  • Finding what is causing the bug

  • Fixing the bug

  • Deploying the fixed version


Good bug reports are of critical importance here. Good bug reports contain repro steps, so that the person fixing the issue can see it with his own eyes (or debugger, or whatever sense he can use). The best bug reports contain short repro steps - if the repro steps are complicated and hard to follow, the total time to fix the bug is increased.


Create - Building the Game


project-plan.jpg


Product being developed from design to beta, new features implemented.


Bug sources:



  1. People in the team working on the game

  2. Publisher feedback


Bug penalties:



  1. Workflow impact - does the bug prevent someone else from doing his work?


Bug fixing cost:



  1. Time to fix

  2. Time to test not so important, as at this stage the big game testing is still ahead of us anyway


Planned bug fixing capacity:



  1. Each person in a team should dedicate one day per week to bug fixing




Polish - Bug Fixing Phase


bug-shirt-theme.png


Between Beta and Gold there are generally no new features implemented, only bugs are fixed and sometimes existing features refined.


Bug sources:





  1. Publisher requirements - the publisher can reject the product because of some issues considered "critical" by them. Such issues need to be fixed, or in some cases we need to communicate with the publisher and perhaps convince them that the issue is not so important.

  2. Internal testing - which bugs do we view as major blocks for the product to be released.

  3. Closed external beta.


Bug fixing cost:



  1. Time to fix - this is the obvious cost: someone needs to fix the issue, be it a programmer, artist, designer.

  2. Time to test dependent content: AI bugs are infamous for this. Some AI fixes may drastically change the playability of the game. When an AI bug is identified close to the release date, but is not critical enough to justify delaying the release, it may happen that its fixing is postponed into a patch.


Bug penalties:



  1. Risk of bad product reputation, which may also result in bad reviews and lesser sales

  2. Workflow impact


Planned bug fixing capacity:



  1. Bug fixing is the primary activity for everyone in the team




Support - After Release Maintenance


arma_cover.jpgBugs are fixed which we were either unable to fix before release, or we did not know about them before release.


Bug sources:



  1. Customer feedback - this is you, complaining about bugs using various channels (technical support,community bug tracker, forum, etc etc).


Bug penalties:



  1. Lost sales because of bad reputation.


Bug fixing cost:



  1. Time to test dependent content - this is very problematic, as at this stage there is usually not any regular QA process funded by the publisher any more.

  2. Time to fix. Not obvious here may be the fact that another product is already in development, therefore fixing bugs from older products may cause schedule issues for the new product, especially when the fix is not related to it.

  3. An interesting situation here is the fixing cost may be significantly reduced when a fix was developed and tested in a product which is already in development. An example of this is that ArmA patches sometimes contain fixes which were implemented for ArmA 2. Sometimes the issues fixed this way are not very important, but as the cost of transferring the fix is sometimes relatively low, this fix is included for ArmA as well.


Planned bug fixing capacity:



  1. If there are many known bugs at the gold date, there is still usually a substantial amount of time reserved to fixing them so that customer impact can be minimized or reduced. Later on, there is no permanently reserved capacity for this at all, what is done is that sometimes there are short periods of time dedicated to bug fixing either by individuals, or by small teams.

Published on