TDD Failures
Difficulty of TDD in game development.
May, 2006
David Weller (2004) published an article about the difficulty of implementing TDD as a design methodology in game development using DirectX.
While our project is not a video game, some of the issues raised in the article are applicable and were confimed by the development team during the early stages of this project.
The main problem experienced using TDD is that certain things are extremely difficult to prove, although we find it very appropriate to use when the code was designed to be called in a specific order or whose result is expected.
About this, the article says that to make the test cases automatic and repeatable it should not require the user interaction during their execution. This was not the case of the application developed by the team where a difference of one frame back or forth, the stages of the swing, angles or paths lead to different code execution.
One solution posed by the article is the use of "mock objects", by which we can simulate the environment and the conditions of implementation to test certain features.
The team began developing a "mock object" to test the video playback capabilities trying to simulate the DirectX runtime environment.
We quickly reached the same conclusion as the author of the article. The "mock objects" never behave exactly like the original, regardless of the effort made.
Our problem was that DirectX set a serie of environment variables defined by the device, which are not defined when running a unit test by Nunit.
We conducted a brief research into that matter, seeing some solutions that add more complexity in the development of each test case than the team was ready to face.
It fact, the team's idea about TDD is that it is only useful if it make the design tasks more simple and clear. The fact of having to implement major design solutions to develop test cases was rejected by the team.
Instead it was decided to make checklists with test items to be followed at the end of each iteration, so we will always perform the same tests to ensure proper operation of each functionality.
WELLER, David 2004. The Random Musings of David Weller [online]
