In summer, we talked about the work of our QA teams, which focus each team has and the importance of these teams for the overall production process of the game – doing a lot more than just looking for bugs or playtesting Anno 1800. If you haven’t checked it out yet, we heavily recommend you do so before you continue reading to first learn about the tasks and areas of responsibility of our three teams: our internal QA team, the Live QA team and the Quality Control teams.

And as we mentioned back then, we now want to give you some insights into which steps are taken from finding an issue to a fix being released

In order for you to get a quick first overview, have a look at the graphic below. We will then go into more detail in the blog itself.

Finding or Reporting an Issue

Usually, there are two ways for an issue to pop up on our radar: either it’s being reported by you via one of our channels (or forwarded to us by one of the Anno Companions) or we notice it during internal testing – or playing 😉

In both cases the issue is being put into JIRA, which is a tool that can be used for various purposes, in this case, we use it to report and track bugs. During this process, each issue is categorized on several factors and tagged if required, most importantly: severity (impact of bug if encountered) and probability (how likely will players encounter this issue)

These two factors then make up the priority (e.g., a high probability but low severity issue will result in low or medium priority).

Reproduction

The next step is super important: We attempt to reproduce the issue, meaning we try to make it appear on purpose on our end. This helps us locate potential causes for the issue which in turn makes it a lot easier for us later to address and fix the problem later.

This step can sometimes be quite work-intensive. For example, to reproduce issues from the live version, the QC team sometimes needs a very specific hardware combination for tests, which might be extremely hard to come by.   Sometimes this also means having to work with hardware manufacturers like AMD or Nvidia directly to get or test specific hardware combinations to reproduce and investigate specific issues.

It’s therefore extremely helpful for us if you, when reporting issues to us, add as many details as possible: Can you reproduce it yourself? What did you do before the issue appeared? Which hardware are you using? Etc.

For Anno 1800, due to the variety of options to set up a game and each player’s unique playstyle, it often happens that the Live QA team might need to ask for save files, screenshots, videos or system information after receiving the initial report. This is when the Customer Support and the Community Management teams come into action: They will reach out to the player and provide more information if possible.

These reproduction steps are therefore also detailed in each JIRA ticket so that everyone working on this issue can quickly check it for themselves. Also added is a reproduction rate: Can the issue be recreated every time when following certain steps or does it only happen occasionally? Or are we even unable to reproduce an issue that has been reported to us?

Fixing the Issue

Fixing the issue means further investigating the cause of the problem based on the information provided.

This happens based on the prioritization we’ve done in step 1. Additional factors are available resources inside the team, which can mean that even if an issue has a low priority, the fix will still end up in the update if we have free resources in the responsible department. Keep in mind that the team is simultaneously also working on the next content update or DLC.

We also want to highlight here that of course the team responsible for the fix greatly depends on the kind of issue. A wrong quest text? A missing texture? An item effect not working? Misaligned icons? Each might need a certain specialist to address it.

While in some cases it can be, for example, possible for one of our coders to figure out the cause and location of an issue without reproduction steps, this is not the norm and always greatly extends the time required to create a fix.

That said, any kind of fix can take time because usually finding the exact cause is not a matter of minutes: We must figure out what exactly is leading to the behavior we have observed and reproduced. Additionally, there are possible risks, especially since video games like Anno 1800 are super complex: Fixing one issue can lead to a different system not working as intended anymore, new bugs can appear, etc. Some systems of the game are far more tricky to work on than others.

Testing the Fix

Therefore, each fix needs to be tested properly. Not only isolated just for itself (is the bug solved?) but also in combination with the rest of the Game Update – i.e., the new version of the game. This is to make sure that a fix is not breaking something else, as mentioned above.

Detailed reproduction steps are therefore also important for the testing since we need to know how one could initially encounter the bug: Does the issue now NOT appear anymore after following the outlined steps?

As you might have guessed, similarly to the previous steps, testing isn’t done in a day. We usually enter a “validation period” before we deploy a Game Update. This period usually is about 2 weeks long and consists of two phases:

1. The first one happens internally here in Mainz and it’s a “Feature Freeze” which we call the “Golden Ticket Phase” – because only QA-approved, “golden” JIRA tickets are still allowed to be worked on and included in the update. Everything else will have to wait for a future update.
2. In the second week, our colleagues in the QC teams are doing the validation of the update. We are not working on the update anymore at this point.

Of course, it can happen that a specific fix or even the Game Update itself turns out to be not working or to have issues. In that case, we need to go back to the previous steps (including another round of validations) which in the worst case can mean having to delay the release of the Game Update.

Game Update Release

When the new version has been tested and validated, we’re almost ready for release. During this process, we’re also in contact with colleagues in other teams so everyone is aware we’re planning to release the update that day.

We also usually bundle multiple fixes together in such a Game Update: This might mean that some issues are already fixed and waiting to be released to the live version of the game, but due to the amount of work involved, there is a balance we have to strike between addressing a problem in time while still not overburdening our teams with constant releases.

The “Hotfix” is an exception, and we talked about it in our previous DevBlog.

Our major Game Updates which release alongside a DLC usually go live at 6 PM CET while the smaller ones usually go live at 2 PM CET. And if you haven’t noticed yet: Our favorite release day is Tuesday. ?

Since we regularly see misunderstandings about our internal processes and the time it takes to address and fix bugs and other technical issues, it was a priority for us to shed some more light on this topic specifically.

As we have hopefully made clearer in this blog, the process behind bug reporting and bug fixing is not an easy one: It takes time and, most importantly, a lot of patience and resources. If you have any questions about it or would like to know more about other specific processes, don’t hesitate to reach out in the comments below!