cs498gd:game_quality_assurance
Game Quality Assurance
Reading
- Game Quality Assurance (QA):
Game Testing Common Perceptions: Myths and Facts
- To the general population, being a game tester sounds like a dream job.
- Testing is not easy.
- Many of the best programmers and engineers started their career as testers.
- Testing trains you in what to look out for and to not make embarrassing errors.
Game Testing Glossary:
- Quality Assurance (QA) - the professional term for testing
- Bug - A software problem that exists at the implementation-level (i.e., only exists in source code)
- Flaw - A software problem at a much deeper level, particularly in the design, which likely bleeds into the code level as well
- System testing - Test the individual game components; primarily done internally, within development team
- Play testing - Primarily done among development team, product and marketing managers
- Alpha - Refers to the fact that everything has been implemented, but there are many bugs and gameplay/functionality issues; testing audience widens to senior-level personnel in company
- Beta - Refers to the fact that everything has been implemented, and many critical gameplay/functionality issues and bugs have been resolved (but there are small bugs still lingering); even wider audience, nearly public
The Real World
- No software is perfect.
- Software is getting more difficult to make and manage due to increasing…
- Complexity
- Demand for extensibility
- Demand for connectivity
- Testing often done very late in the integration process, right before roll-out.
- Testing can be a thankless and under-appreciated job.
- You will likely tick off engineers and programmers when you find bugs, and tell him/her to fix them.
- Because testing is done so late in the integration process, a lot of known and found bugs will not be resolved in time because the product needs to be shipped.
- Most “agree” that testing should be tightly incorporated into the development cycle, but testing is often pushed to the very end.
Skills Required
- Excellent communication - You will interact with developers. You need to explain things clearly. Developers must to able to comprehend and recreate the problems you describe.
- Tech and game-savvy - You must be able to play the game, and navigate through the product, largely on your own with minimal instructions.
- Endurance - You must be able to work on a project for a long time, for many hours at a time, especially during crunch time.
- There will likely be much repetitive and mundane work consisting of playing an unfinished game over and over again, installing software, adding/removing hardware.
- A good attitude
General Defect Tracking
- Reporting bugs
- Done through defect tracking systems.
- Your report must be descriptive and concise for the developers to recreate and fix the problem.
- The types of bugs that you report (by severity):
- Showstopper - These bugs must be fixed immediately. (e.g., glaring problems including audio glitches, game crashes, missing features, etc.)
- Consequences could include: bad press, poor sales, people getting fired, and a black mark on the company's name.
- Serious - Problems that gamers will likely notice, but does not take away from the gameplay experience (thus, can hold off on fixing immediately)
- Low - Not so glaring problems that can be fixed in the next revision or patch
- Suggestion - Changes or feature additions that would be nice to have
- Other possible fields in a bug report:
- Reported on date
- Report by (your name)
- Assigned to (developer)
- Feature affected
- Description
- Build/version of game affected
- Response to bug report:
- The assigned-to developer will look into the problem and will add to the report:
- A status: {Open, Closed, Resolved, Not a bug, Cannot replicate}
- Resolved on date
- Resolution method
- Assigned to (another developer)
- After bug has been resolved, tester must verify the fix.
Types of Testing
Regression Testing
- If you have a plan to test a set of features, and changes and bugfixes are made, you have to go back and retest everything (the set of features) from the start.
- Important if you have a patch to be released for the game.
- Bugfixes and changes can introduce new bugs.
Usability Testing
- Very different from QA testing
- Not necessarily trying to find bugs
- What you are testing in usability testing:
- Ease of use
- Interface design
- Can you perform a certain task in a game?
- Is the game actually fun?
- Evaluation
- Usability testing should be conducted by someone who is not a developer (no knowledge of the game, thus no influence on testing).
- Subjects should also have no prior knowledge of the game.
- Subjects must be carefully monitored, and all actions documented by evaluator/interviewer (e.g., note where the subject was confused, monitor repeated mouse button and keyboard presses).
- Subjects are encouraged to think aloud.
Game QA Best Practices
Integrate QA
- Integrate testing into the development process.
- Instead of testing everything at the end, test at certain milestones.
QA Automation
- A lot of QA work is mundane.
- Write scripts, where possible, to do repeated and mundane tasks.
- Scripted testing can be very efficient and save a lot of person-hours.
Have a test plan
- Define what needs to be tested.
- Provide a list of expectations for testers.
- Prioritize:
- You do not necessarily want to test everything, but you do want to test the critical game features.
- Identify the classes of problems to be tested for a feature (e.g. kicking a soccer ball), and test the classes a number of times in different game settings:
- Kick the ball out-of-bounds.
- Kick the ball to another player.
- Kick the ball into the goal.
- Run and kick the ball.
- Kick the ball while it is stationary (free-kick or corner kick).
cs498gd/game_quality_assurance.txt · Last modified: 2018/11/15 13:50 by jchung