Are you thinking of how to cut your software testing and QA costs without compromising on product quality? That’s one of the most burning questions as far as software development goes. Companies who’ve already had issues as a result of poor software testing will agree that when done the wrong way, QA can result in negative and even detrimental consequences for business – from having to do a major overhaul of your entire application at the post-release stage to a tarnished reputation, damaged track record and/or decreased/lost user loyalty.
If you allocate too many QA resources for your software project, you’ll have to pay a huge bill at the end of each month and there’s no guarantee you’ll ever be able to justify this spending by gaining appropriate ROI for your testing efforts. On the other hand, if you fail to put a sufficient number of resources to test your digital product, you’ll be at risk of facing overheads as a result of later bug-fixing. So you need to find a golden middle and make sure you hire enough people not to inflate the budget and eliminate future overheads.
In this article, I’ll explore how to find a good balance between the cost of your software testing and the quality of your software product. I’m going to present my 8 ways of reducing QA costs without decreasing your software quality standards.
1. Start testing your product as early as possible
One of the most common mistakes is to start testing the product at a later stage when it is almost ready for release. The earlier your QA team syncs up with the development team, the lower the chance of missing an error in production. In addition, an error identified at an early stage of development will always cost cheaper than one found after the final release. Way cheaper. Our experience shows that the cost of bug-fixing at the later stages can be 30 times more expensive than bug-fixing at the prototype stage.
Source: habr.com
A real-life 8allocate’s client case
One of our client Dedicated QA Teams had to test a server application that collected and processed a large array of user data. The client’s in-house developers, however, refused to submit it for testing to our hosted QA team, saying the app has a lot of known issues and suggesting that testing could be done at the final stage. The Client who was fully accountable for project management and delivery agreed with developers and let them build the product without interim testing. As a result, they spent 4 months working on the new app’s version without having tested the previous version.
When the outsourced QA team started testing the application with real user data, it turned out that the selected database architecture could not cope with such data loads. A development team of five people had to re-plan everything, rewrite the design and architecture of the application, and the QA team had to test it from scratch. As a result, the app was released with a 3-month delay, which cost the client twice as much as they had planned initially.
The bottom line: the urge to save 150 man-hours on software testing and push it to the release stage led to the loss of 1,100 man-hours and doubled the cost of testing for the Client.
2. Use API virtualization
API virtualization mimics system behavior and implies that no real software development environment is required to simulate realistic behavior of application. By using API virtualization tools such as Smartbear, Wiremock, Microfocus Data Simulation, and so on, QA engineers, front-end developers and system integrators can start doing their part of work as soon as developers produce an initial specification (Swagger, RAML, WSDL, etc.) early on at the software design stage.
Using API virtualization can help QA managers get ahead of the game by creating functional and performance tests against virtual APIs, and, thus, reduce the overall cost of using 3rd party APIs. Testing against the real 3rd party APIs can involve fees beyond the established budget. When you virtualize external APIs, you don’t have to pay for anything and, as a result, you won’t likely get an unexpected explosion in cost at the end of the billing cycle with your service provider.
3. Never save on analytics
Let’s assume you’ve listened to my advice and decided to implement the first method. At what stage of your product development is it most reasonable to lash out on testing and hire a QA team? The answer is – at the application architecture planning stage, or at least at the stage of discovery and collection of business and technical requirements.
Without proper analytics, you risk launching a product that nobody needs. This happens when you do not consider the peculiarities of each of your target audiences.
Do you remember a “horror” case about a running app that was tailored specifically to Asian users? However, none of the users was ever able to install the app on their smartphone! Why? Because of an error made at the stage of requirements gathering – the discovery and all feasibility hypotheses were made using the iPhone and Samsung phones which aren’t as popular in Asia as Huawei and OPPO.
The bottom line: no matter if you involve an external QA service provider or choose to test your software product internally, user testing and finding bugs at the support stage costs ten times more. And your reputation risks will always outweigh financial losses in this situation.
4. Define testing goals and expectations
Do you know how to tell a good test manager form a bad one? The good ones immediately start bombarding you with questions in order to understand exactly what product quality you’re looking to receive. We call it the collection of expectations. The quality of your software depends on certain building blocks. An experienced test manager will want to know what kind of building blocks to put in the foundation of your product’s quality.
When the client comes to 8allocate and says: “I want it to work perfectly well,” we begin by asking what they mean by “perfectly well”. Once we understand it, we know how to make it work to client expectations. So, asking the client/product owner as many product related questions as possible can help you get a better picture and visualize a clear roadmap of what needs to be tested, when, and how.
Having collected client expectations, we set up SMART goals, decompose them, define tasks and KPIs based on them. As a result, we clearly define:
- Types of testing;
- Testing milestones and due dates;
- Line-up;
- Possible risks.
All of this is needed to deliver the expected level of product quality and eliminate any guesswork.
Sometimes, however, incidents occur. We once worked with the client who needed a robust InsureTech solution. As usual, we held a comprehensive discovery session, conducted a thorough analysis and made a proposal. Due to the complexity of functional requirements and large size of modules planned, we suggested that regression testing be fully automated, as regression testing was the most laborious and time-consuming QA activity. Despite our recommendations, the client chose not to automate regression testing, because, as they believed they had a strong internal team of manual testers and wanted to make use of them. Unfortunately, all our further arguments and estimations were ignored, too. We could not get the client’s buy-in and for three months we’d conducted regression tests manually, which, in fact, proved highly unprofitable for the customer.
At the end of the day, the client put the whole project on hold, spent 2 months analyzing it, and returned to us with a proposal to use the regression test automation plan that we’d presented 5 months ago.
The bottom line: besides understanding the project mission, goals and the client’s vision, you need to be able to argumentatively prove your point of view, especially if you’re 100% sure you’re right. The client’s failure to follow your suggestions can result in financial losses down the road, so if you can’t get their buy-in from the very beginning, it sometimes makes sense to reject such a client.
5. Automate testing wisely
To be able to save money with automation, the product must be stable. Even if it changes dynamically, it doesn’t mean you don’t need automation. The use of tools and special apps is already sort of automation. For example, you can help automate routine operations that eat up most of your testers’ time. Say, create a utility tool that collects the necessary configurations.
A real-life 8allocate’s client case
One of our clients wanted to release code every day instead of twice a week. At the same time, the full manual regression testing took two to three days to complete. And since 80% of the product’s functionality was pretty stable, it was decided to automate it. For this purpose, we developed four automation scenarios and conducted a comparative analysis of their effectiveness. We used 8 indicators for this task:
- The number of test cases per scenario;
- Automation (man-days) – the number of resources per one automated scenario (excluding test data for each test case);
- Single Test Case automation (man-hrs) – the cost of automating a single test case;
- Manual testing (man-days) – the cost of manual testing of the scenario as a whole;
- Results investigation (man-hrs) – the cost of researching the results of the AutoTest run;
- Execution times – the number of test runs required for the whole project development. This number reflects the expected number of runs, taking into account the stability of the functionality, the required schedule of test runs (regularity of obtaining information), etc.;
- Automation efficacy (%) – the effectiveness of test automation as a percentage of time saved. Given the computation errors, test automation can be considered effective if its efficacy exceeds 150%;
- Saved time (man-days) – the number of man-days saved during the entire project.
Two automation scenarios passed the final selection. Their use allowed us to reduce the regression from two days to a couple of hours for running autotests, and save two hours for the remaining manual checks, which were unprofitable to automate.
The bottom line: test automation can either reduce QA labor costs tenfold, or it can drain the budget completely. Without the ability to analyze and calculate ROI, you always risk giving up on automation.
6. Use junior resources where possible
Still struggling to automate your software testing? Or can’t afford to hire professional QA engineers due to their high cost on the local jobs market? Hiring junior resources for a small yet reasonable salary can be a good idea! Yet, there’re nuances you need to understand: a child can find a bug in his favorite online game and break the app with monkey testing, but will he still be able to find all of the bugs? That’s pretty unlikely. While newbies can be useful for some manual test tasks, you need to have a real QA master on your team to give tasks in a language rookies will understand and mentor them as the project goes.
At 8allocate, we run one of Ukraine’s leading Coding Academies that has just been recognized as Ukraine’s Top 10 extracurricular school. It graduates 4,000-6,000 IT specialists every year. This allows us to source the best junior talent to assist your senior staff on projects (with non business critical tasks, of course). As the practice shows, it takes us up to a month to onboard a rookie and “plug” them into your project as testers. They normally cost very little compared to mid-level resources and can be nurtured internally with your senior’s help.
Check out when it makes sense to hire junior developers for your software project.
The bottom line: using newbies labor for QA proves highly cost-effective. The junior talent is able to bring real benefits to your software project starting from the week two, and help reduce the team adaptation time by 4 times (from 160 to 40 hours, on average).
An experienced mentor will help you grow and develop a stellar QA team within a very short timeframe, which will help you save costs on using several mid-level and senior talent on your project.
7. Always choose quality over quantity
You do not want to pay through the nose for hiring many experienced QA guys who command high salaries but can quickly get tired and bored at working on a mainstream testing project. On the other hand, you don’t want your QA team to be composed of 50 low-cost and inexperienced juniors.
While your gut feeling is important, still try to use good BAs to help you assess the amount of QA tasks, the number of people needed to get the job done, and identify their skills and qualifications. Be pragmatic and use less expensive offshore talent to increase bottom line savings.
A real-life 8allocate’s client case
Not so long ago, we were approached by a client who had 12 testers in their in-house team and wanted to extend it offshore to increase the bandwidth and do more work for the same budget. They had the following roles in their team: a Test Manager, one senior, one middle and 9 junior testers. As a first step, we conducted an audit of the client’s internal testing processes and identified around 1,000 (!!) unnecessary tasks such as regression of functionality that wasn’t upgraded, etc. It helped us figure out how to optimize the tests.
Our analysis showed that the project would only need a Test Manager, three junior automation testers (whom we could supply from our Coding Academy best-achieving graduates’ pool), one senior QA engineer and one middle tester. So the client retained just a Test Manager and a senior QA guy in-house, and hired a mid-level tester and replaced 9 junior testers with 3 same-level offshore employees in Ukraine. The team was curtailed by half, but only gained from it.
The bottom line: our client’s in-house QA team audit allowed us to reduce the cost of pre-release testing by 50%, achieve a 5-fold increase in the client’s ROI for QA (as a result of automation), and reduce pre-release testing time by 23 man-hours.
8. Outsource your QA function to an expert provider
According to a recently published CEE Software Development Report 2019, 17% of all software projects outsourced to Ukraine relate to QA and testing. This number increases from year to year due to some obvious reasons:
1) Lack of QA talent in North America, Europe, and globally
As reported by Forbes, 84% of enterprise executives say the lack of digital QA talent is a huge issue! If large companies face this challenge, it’s obvious that startups and SMEs feel these shortcomings even more.
2) The high cost of senior and mid-level talent
According to Indeed.com, in NYC, the average mid-level manual QA engineer costs $85,000 per annum; in London – $88,000; in Ukraine, you can get a QA tech lead for as low as $33,600 a year. This salary discrepancy explains a lot!
Also, time to hire a QA resource in Ukraine is less than 30 days, while in North America it can take up to 60 days and more. In this highly dynamic world speed is crucial, so accelerating your time to hire will help speed up your time to market accordingly.
3) Lack of internal QA expertise
If you’re a small non-IT company from the USA or UK, the chance is slim you can develop your software product onshore/in-house without inflating your budget. You simply won’t be able to launch an in-house team without making substantial investments in the office space and infrastructure, i.e. increasing your operating expenses.
By outsourcing QA, you can access untapped pools and, thus, source good yet more cost-effective QA specialists to save on cost arbitrage and to benefit from the speed of testing and development in general.
Conclusion
Quality is a multifaceted concept, and, therefore, its price may vary depending on the depth of this term’s understanding by the product owner. In software development, there are many ways to save money at the expense of quality, but there are not many options for improving quality without overpaying. Compliance with the above methods helped us earn the reputation of a reliable outsourcing partner (which can be verified on Clutch) who cares about each client’s budget and always seeks ways to optimize and save it without compromise on quality.
