In the current age of rapid innovation and automation, legacy core banking systems are often marked by outdated technology and architecture. These systems encounter numerous challenges, including rigidity, integration complexity, inefficient processes, vendor dependencies, and difficulty meeting evolving customer expectations. Consequently, core banking transformation and modernization have become imperative for banks today, a necessity they cannot overlook.

Thought Machine has developed the foundations of modern banking with its innovative core banking solution, Vault Core, which enables banks to build any product through smart contracts written in Python. Vault Core allows banks to define their product logic through its Universal Product Engine, either prebuilt and customized smart contracts or those written entirely from scratch. The extensive customization that Vault Core offers enables users to efficiently create, manage, and innovate new products, delivering a high-performance banking experience. With this capability, a critical need arises for testing and validating these contracts to ensure their proper functioning throughout their lifecycle. In this article, we will delve deeper into various testing methodologies, their limitations, and the optimal approach to testing that can make the process faster and more efficient.

What are smart contract testing approaches?

Smart contracts are the code descriptions of financial products that must be tested to ensure they perform as required, preventing any negative impact on the various complex workflows involved in a banking system. Testing these contracts helps banks ensure that products are correctly coded, safeguarded against major failures and disruptions, and avoid placing unnecessary loading on the bank environment. These tests are to be performed during the Vault Core development or implementation phase as part of the CI/CD pipeline each time there is a change to the repository used in developing the contracts. Virtusa’s Thought Machine-certified engineers, smart contract developers, and quality assurance engineers employ a combination of different testing approaches, including unit testing, simulation testing, end to end testing, and time cursor testing, to validate the functioning of these smart contracts.

 Unit testing 

Our Thought Machine-certified engineers at Virtusa perform unit testing to examine the functions and hooks of each smart contract, including their respective boundary cases while excluding the parameters and metadata. This testing is conducted in isolation from other components, resulting in cost-effectiveness, rapid execution, and simplified debugging. However, unit testing has constraints, such as limited coverage, the inability to test against diverse environments, and API calls.  We perform a second level of testing, called simulation testing, to achieve better results.

Simulation testing 

This methodology simulates the financial product's behavior and validates functionalities over a timeframe using an API endpoint. We test existing and customized product features across various inputs and events while simulating an account lifecycle. As a simulation, it is the second fastest testing method, allowing scenario testing for past and future events. Simulation testing is inherently constrained by inadequate API coverage, the incapacity to generate tangible entities, the inability to test RESTful and streaming APIs, and dependency on live environment connections. These limitations lead us to use a third level of testing, known as end to end testing.

End to end testing

During end to end testing, the smart contract undergoes a comprehensive evaluation, encompassing all associated services and interfaces. At Virtusa, we utilize our domain expertise and technical SME capabilities to test both existing and customized products, employing a variety of permutations and combinations of features to address complex scenarios. Despite its comprehensive coverage, this approach presents several challenges. It requires a high level of configuration  to engage all services and APIs, strains resources throughout the end-to-end lifecycle, results in slower performance, and leaves artifact products in the production environment.

Manual testing

Manual testing is the final resort for identifying and validating suspected edge or corner cases. This method involves substantial configuration, especially when interacting with RESTful and streaming APIs. It's imperative to outline a business case for examining product behavior in these edge cases. Manual testing is the most costly and time-consuming testing methodology due to its resource-intensive nature.  For these reasons, end to end testing is typically favored over manual testing.

At Virtusa, we have extensive experience with these four testing methodologies for existing and customized products. Our domain and technical SMEs cover everything from lending to deposit products to creating test cases that address specific features and complex scenarios, including edge cases. Customizations require comprehensive retesting to ensure other functionalities remain unchanged, which can delay product release. To address this, our Thought Machine COE developed a test automation tool. This regression testing tool uses an end to end approach combined with the time cursor technique to build test scripts for the CI/CD pipeline, generating 360-degree reports with step-by-step results and pass/fail outcomes that are easy to read for both business and technical users, making it scalable and customizable as needed.