if !blogClogged

Software development and other stuff.

Why too much gaming development is infested with bugs

leave a comment »

Video Game Development

photo credit: brummiedave85 via photopin cc

Though I have been working with a large focus on SDLC and ALM practices as a consultant and as a practitioner I have spent no time that I can recall formally working with SDLC for gaming development. Despite being a computer geek I was never into the gaming side of computers as either a casual game player or real gamer. That changed suddenly over the past couple of years. At over 40 years old I began to play computer games. First on a console and then adopted gaming on the PC. It’s all because of my triplet children who are about to turn 8. What I have come to find when I look at the gaming world while wearing my professional hat is alarming and distressing. But I also see opportunity.

First, the alarming the and distressing part. In short, what I have seen is a what appears to be a massive lack of proper quality controls. As I have begun to delve into gaming development I have found that the obvious quality shortcomings are almost certainly coming in part from poor, limited existence or non-use of testing tools, mechanisms or processes. I have seen in the gaming development landscape a world in which many of the testing tools seem to function and integrate at a level that would be more consistent with an enterprise application development landscape of 1999 than 2014. Yes, I said 1999 – the previous century. Compared to testing tools used by enterprise developers I have thus far found that the gaming industry is in many places what I gauge to be 10 – 15 years behind. The caveat is that I am just starting to survey the wider landscape and there are some third party (not from the gaming engine developers) testing tools I have yet to crack. But if I take the Unity game engine as an example you will now find Unit Test support using Nunit and support for creating some automated Integration tests – this is a new capability that has just become available to the public in December of 2013. This means the penetration is zero for any game on the market or about to hit the market that used Unity unless it was being done entirely separately.

Without doubt testing game code is very tough. It’s an intentionally rich environment with mind boggling permutations. Testing of any sort in that environment is tough and automated testing is much tougher.  I have great sympathy for those trying to apply to gaming testing tools similar to enterprise testing tools.  The concepts of quality control don’t change between the two environments but that doesn’t mean that the same type of tools working in the same type of way will translate well. The gaming development engine creators have created marvels of technology. The game development tools have become so powerful and so rich that developers can create massive, rich, open playing worlds that can be hard to comprehend how it can be done. Because the developers can create these worlds and the gamers and business interest demand the creation of these spectacular games then the developers go and create to the very edge.  The problem…

The testing tools for gaming development have not remotely kept pace with the sophistication and complexity of the product it should be testing. The developers ability to create larger and more wondrous things also generates increasing numbers of bugs. With a cutting edge bug engine a.k.a gaming development engine but testing tools and methodologies that may be a decade behind, the number of escaped bugs is only going to rise.

Here is another example from the gaming world that as a player in the game I can determine likely reasons I see certain failing behaviors repeatedly. I’m not picking on the MMORPG Neverwinter but I will provide one of what could be numerous examples of a recurring bug. Like most MMORPGs Neverwinter has an auction house for offering goods for sale between players. It has a defective search component. Defective to this day. The search is a rather important part of finding an item when the volume of listings are very large as would be expected in an MMORPG. Currently the search is still usable for most tasks. Add to this that the search results are supposed to be sortable by  some of the columns in the results. The classic clicking the column header should re-sort the search (single column header sort only).  After one of the weekly updates the column sort no longer properly sorted. It effectively makes the ability to find the items desired impossible in many circumstance as the result set will only return 400 records no matter if there are 5000. So you can’t find the next item up for bid perhaps or the lowest buyout price, or the maximum asking price etc. Typically after the Thursday update there is a Friday emergency update to patch the things they broke in the weekly Thursday patch. Those things happen we all know how that works- especially in a large complex game. I give them a pass on patches to patches. We all have to do it from time to time. The broken sort was fixed in the patch to the update. Until the next weeks update. Then it reverted back to the broken state of the week before. The broken->fix, broken-> fix pattern went on for a few weeks, as I recall, and then a hiatus for some weeks and the it started the cycle over for a few weeks again. I cannot say for certaint looking only at the resulting screens as a player what was happening in their lifecycle but I can make a few educated guesses. First, there obviously is no regression test for that functionality despite it breaking multiple times in updates. Or there is a regression test and there is poor traceability so it is not recognized that that set of functions should be tested because it has been impacted. Or there is a regression test and it was considered too low priority and not run- even after breaking it multiple times. I can also make an educated guess that there is some poor source code control going on. Thus the defect continues to get reintroduced and not tested because of the poor traceability between the code churn and the regression test that may or may not exist.  This is another area in which it seems that often the connectivity between the game developers engine IDE workspace and version control is often not well integrated. Many times it doesn’t have to be well integrated but in defense of game development proper version control handling is often not as easily done as  in other industries. That though, is a problem and not a symptom. It’s a process and ALM issue.

I’m going to wrap this up and have more later. The point as I hope you have gathered now is that there are multiple reasons you may have seen so many big games hit the market and be loaded with bugs and fixes and updates with feature enhancements introduce even more. From what I am learning in my delve into the tooling for game development is that one of the primary reasons is just what I placed in bold text above – the testing tools and testing process maturity for gaming development are horrendously behind the curve in comparison to the technology they need to test.  The second reason I’d propose for the common scenario of released buggy games is not at all technical and is process only in that it is a business decisions on “ready for release” – it seems many organizations are not concerned with their release quality because the monetary impact is still favorable for them in delivering “released” code of high complexity and huge potential but of low quality in terms of reliability, bugs and user experience.  When they can release higher quality product and with reduced overall cost to reach that quality through better tooling and practices and we as gamers refuse to buy the product and stick it out until Update 5+, 10+ whatever, then I predict quality will rise. For the moment the development tools far outpace the testing tools and thus cost effective quality controls achieved in proper testing  will not happen. As I see it.

Written by Michael Ruminer

February 28, 2014 at 11:29 pm

Posted in Uncategorized

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: