The bedrock of every successful Core Banking implementation is in having robust QA. What does it take to provide that quality assurance in large, complex transformation programmes?
The boardroom of every bank that has a core banking change underway invariably, at some stage, gets to make that very difficult call – of having to choose between adequate QA and timely cutover. Having executed over 25+ large core banking transformations personally, one can relate to the degree of discomfort, knowing fully well what the opportunity cost of the alternative is!
This is where a well-planned and tightly integrated Quality Assurance approach can make a real difference, especially when one seeks the assurance that the system will indeed work, when the cutover curtains are raised. At the end of the day, this is about changing wheels at 30,000 feet, and the margin for error is wafer-thin if it is there at all!
These are days of Dev-ops where the software development and IT operations are expected to be inter-woven and delivered in an agile manner, allowing for continuous delivery and assured quality. That being said, for most part it, core banking yet follows a waterfall approach. Which therefore begets 4 key questions that need to be answered:
What | Context, Coverage and Scope
An end-to-end QA for core banking is more than just an end-user acceptance testing. A full-fledged core banking transformation would typically involve Factory level unit-testing before the product gets shipped, a System Integration testing that confirms that it is technically fit to proceed further, UAT or the user acceptance testing that confirms that the business functionality is well addressed, Regression testing that allows for revalidation of the solution, performance testing that confirms the system stands the test of a stressed loading and, in addition to the above, penetration testing to confirm system vulnerabilities to third party interventions.
We are seeking to focus this discussion on the user acceptance validation, which is related more to the functionality and the features of a core banking system. We need to recognise that every bank has a core banking platform aligned to its own needs and the actual application of the platform comes with an orientation that is aligned to the products and services rendered by the bank. In other words, unless the testing coverage is truly reflective of the end-user’s actual experience, the investment of time, effort and resources on a QA process will render it to be sub-optimal.
When | Before, during and after
The timing of a QA process can be everything in a core banking transformation programme. Having to bolt the proverbial stable door after the horses are stolen is exactly what it means when banks seek to test the product after declaring a go-live date. The point here is not about having things totally open-ended but ensuring that there is a structure and sequence to the process, which is in line with industry best practices.
For instance, planning at least 3 to 4 rounds of user acceptance testing is a pre-requisite for drawing adequate reassurance on quality. The key here is to ensure that the testing is done after the parameters are defined, and the system is loaded with migrated real data and is reflective of a post go-live experience. Attempting to have end-users trained on a product before it is UAT verified can be suicidal. As they say, there is only one opportunity to make a good impression, and the users at the branch are the ambassadors who get to speak with customers of the transformation experience. It pays to ensure their experience is good, and their views of the product are positive, in their first encounter. And hence, there is a strategic perspective to both the timing and sequencing.
To summarise, here are 4 simple rules that help articulate the sequence, albeit in a reverse manner:
- Do not take the system live unless all users are well oriented.
- Do not orient the users unless all features of the product are well tested.
- Do not test the product unless the test of System Integration goes well.
- Do not accept the product for SIT unless it has the confirmation of a factory test.
Who | Internal, external or both?
Ownership is key. That may sound quite an old adage, but nothing wiser can said than that. The value of any QA process is about how much it is in eyes of its beholder, and intrinsically also implies that ‘who’ that proverbial beholder is, matters a lot. For instance, banks that are obsessively quality conscious may be less willing to engage external partners, and on the other hand, there are banks that are more comfortable in outsourcing the process end-to-end to a third party.
The secret sauce perhaps is in striking a balance. While the strategic outlook of what needs to be tested, the benchmark performance expected and the exit criteria that qualify the test are to be defined by the bank at a strategic level, the heavy-lifting of test execution may be carried out with some external assistance – especially where having trained hands improves efficiency and, as a corollary, significantly reduces the timelines. That said, holding the business functions from within the bank accountable for the outcome and responsible for the process is a mantra banks can ill-afford to ignore.
How | How long, how deep and how far?
The topic may not only sound like that of the next James Bond flick, but also has the ingredients to give it a run for its money! When it comes to quality, there are no definitive rules of what is good or bad, just principles of what is acceptable and what is not. From our experience, there are some rules of thumb that may help determine the approach:
- How long? The number of rounds to be tested is a function of the degree of customisation and complexity of system interfaces. Having 3 to 4 rounds of UAT including a regression test is generally advisable, although of course, the more the merrier.
- How deep? The number of test cases is a function of the depth that is sought through the QA, and this is where a lot of automation tools have also been effective – in expanding the permutations of a testbed. The thumb-rule for a typical core banking programme is to have a about 75k to 100k+ test cases. This may vary based on the size and coverage of the bank.
- How far? The degree of reassurance is inversely proportional to the exit criteria. In other words, the more stringent the criteria, the more assured is the quality, and the longer the duration of the testing cycle in meeting them. It is therefore paramount to ensure that the exit is not only measured by the outstanding defects, but also by the defect density which is a closer reflection of the system quality.
“If I only had an hour to chop down a tree, I would spend the first 45 minutes sharpening my axe,” Abraham Lincoln once famously said. That is so very relevant when it comes to QA in a core banking programme. The deeper the QA process, the smoother the cutover and experience thereafter. As we all know, the proof of the pudding, ultimately, is in the eating!