Am I correct to choose quality assurance as the field to look upon, or should I go after something else?
Yes, I believe so.
Do you think the tools you've posted could help me achieve what I have in mind?
The tools alone are not enough, obviously. It's like asking: "Do you think C# could help me achieve what I have in mind?". It could, but simply installing and configuring the tools is not going to make everything work magically.
In my opinion, the real challenge of the QA lies in
changing the mentality of the team, not just the processes and workflow. Changing the processes/workflow is easy, straightforward and the tools to do this exist and are widely available. Changing the mentality of people is an incredibly difficult problem.
How can you convince your boss that your devs need more time before shipping because they need to write more tests? Or that you need to spend 10000$ on new machines so you can run a proper CI server? How can you teach the programmer that even though her patch does the job, it's not on par with the QA standard and needs to be refactored?
QA can be as much of a political problem than a technical one
Where to start?
I'm not going to lie to you. Adding proper QA to an existing project as an afterthought may be a very difficult task and you will likely fail. But there are lessons to be taken so that you start on a stronger basis on your next project. I will expose some of the lessons I've learned while working on OpenStack, maybe it'll help you.
Testing
Automated tests are the cornerstone of modern QA. If you don't have tests covering all your functions, then everything else is useless. A quick web search for "benefits of automated testing" should yield millions of web pages explaining why this is important; I'm not going to repeat the same arguments here.
Now here's the very important part: testing never works if it's added as an afterthought. It's wrong to think "I'll write my code first then merge it. The QA team will write the tests later". In OpenStack, we expect every new patch of code to contain the appropriate tests. If you send me the most amazing code for the most urgent feature but you don't include the tests, I'm going to refuse your patch and ask you (nicely) to add the tests so I can validate it.
Some people even argue that you should write your tests first, before even writing the code.
It's very important that you have a good test coverage of your code, so the other tools can start making sense.
Peer Review
This is the second big cultural change that needs to be introduced. In OpenStack, peer review is at the heart of the QA system.
- Everyone can review any patch.
- Every patch needs to be peer reviewed before being merged.
This second one is very important. Nobody is too important for peer review. It's not the "senior" reviewing the "junior"'s patch. It's literally throwing as many eyeballs as possible at the code before accepting it. This is a big mental shift. No senior dev can push code on her own. Newbies can start getting familiar with the code by doing code reviews. Gerrit will help you a lot for this, but the most important part is the
cultural shift.
Continuous integration
It's a natural follow-up to the fact that you're having good test coverage. A CI will be a machine that is constantly running your test. Every time there's a new patch, the CI runs all the tests, every time there's a backport, run all the tests, every time you feel like it, run all the tests.
Having automated tests means you're running these tests all the time.
You should check these out
Take the time to go through them, particularly the 2nd one:
I don't think this will give you the answer to what you should do on your project, but it will show you how complicated and fascinating this problem can be. Hopefully it'll give you some ideas you can apply quickly.
Do some open source work
I repeat the advice I gave you earlier. Go knock on the door of your favorite open source project and get involved in their QA team. You'll learn so much!
