Game Quality Assurance

…also known as, game testing

• Game Quality Assurance (QA):
  • QA Best Practices Report from the 2003 International Game Developers Association Conference
  • "Testers -- The Unsung Heroes of Games"
  • "Inside the 'dream job' of a video game tester"
  • https://www.reddit.com/r/playtesters/

• To the general population, being a game tester sounds like a dream job.
• Testing is not easy.
• Many of the best programmers and engineers started their career as testers.
  • Testing trains you in what to look out for and to not make embarrassing errors.

• Quality Assurance (QA) - the professional term for testing
• Bug - A software problem that exists at the implementation-level (i.e., only exists in source code)
• Flaw - A software problem at a much deeper level, particularly in the design, which likely bleeds into the code level as well
• System testing - Test the individual game components; primarily done internally, within development team
• Play testing - Primarily done among development team, product and marketing managers
• Alpha - Refers to the fact that everything has been implemented, but there are many bugs and gameplay/functionality issues; testing audience widens to senior-level personnel in company
• Beta - Refers to the fact that everything has been implemented, and many critical gameplay/functionality issues and bugs have been resolved (but there are small bugs still lingering); even wider audience, nearly public

• No software is perfect.
• Software is getting more difficult to make and manage due to increasing…
  • Complexity
  • Demand for extensibility
  • Demand for connectivity

• Testing often done very late in the integration process, right before roll-out.

• Testing can be a thankless and under-appreciated job.
  • You will likely tick off engineers and programmers when you find bugs, and tell him/her to fix them.
  • Because testing is done so late in the integration process, a lot of known and found bugs will not be resolved in time because the product needs to be shipped.
  • Most “agree” that testing should be tightly incorporated into the development cycle, but testing is often pushed to the very end.

• Excellent communication - You will interact with developers. You need to explain things clearly. Developers must to able to comprehend and recreate the problems you describe.
• Tech and game-savvy - You must be able to play the game, and navigate through the product, largely on your own with minimal instructions.
• Endurance - You must be able to work on a project for a long time, for many hours at a time, especially during crunch time.
  • There will likely be much repetitive and mundane work consisting of playing an unfinished game over and over again, installing software, adding/removing hardware.
• A good attitude

• Reporting bugs
  • Done through defect tracking systems.
  • Your report must be descriptive and concise for the developers to recreate and fix the problem.

• The types of bugs that you report (by severity):
  • Showstopper - These bugs must be fixed immediately. (e.g., glaring problems including audio glitches, game crashes, missing features, etc.)
    • Consequences could include: bad press, poor sales, people getting fired, and a black mark on the company's name.
  • Serious - Problems that gamers will likely notice, but does not take away from the gameplay experience (thus, can hold off on fixing immediately)
  • Low - Not so glaring problems that can be fixed in the next revision or patch
  • Suggestion - Changes or feature additions that would be nice to have

• Other possible fields in a bug report:
  • Reported on date
  • Report by (your name)
  • Assigned to (developer)
  • Feature affected
  • Description
  • Build/version of game affected

• Response to bug report:
  • The assigned-to developer will look into the problem and will add to the report:
    • A status: {Open, Closed, Resolved, Not a bug, Cannot replicate}
    • Resolved on date
    • Resolution method
    • Assigned to (another developer)

• After bug has been resolved, tester must verify the fix.

• If you have a plan to test a set of features, and changes and bugfixes are made, you have to go back and retest everything (the set of features) from the start.
• Important if you have a patch to be released for the game.
• Bugfixes and changes can introduce new bugs.

• Very different from QA testing
• Not necessarily trying to find bugs

• What you are testing in usability testing:
  • Ease of use
  • Interface design
  • Can you perform a certain task in a game?
  • Is the game actually fun?

• Evaluation
  • Usability testing should be conducted by someone who is not a developer (no knowledge of the game, thus no influence on testing).
  • Subjects should also have no prior knowledge of the game.
  • Subjects must be carefully monitored, and all actions documented by evaluator/interviewer (e.g., note where the subject was confused, monitor repeated mouse button and keyboard presses).
  • Subjects are encouraged to think aloud.

• Integrate testing into the development process.
• Instead of testing everything at the end, test at certain milestones.

• A lot of QA work is mundane.
• Write scripts, where possible, to do repeated and mundane tasks.
  • Scripted testing can be very efficient and save a lot of person-hours.

• Define what needs to be tested.
• Provide a list of expectations for testers.
• Prioritize:
  • You do not necessarily want to test everything, but you do want to test the critical game features.
• Identify the classes of problems to be tested for a feature (e.g. kicking a soccer ball), and test the classes a number of times in different game settings:
  • Kick the ball out-of-bounds.
  • Kick the ball to another player.
  • Kick the ball into the goal.
  • Run and kick the ball.
  • Kick the ball while it is stationary (free-kick or corner kick).