Core Conversion Misstep #4: Your Testing Didn’t Go Far Enough
This blog is the fourth in a series about things that are frequently overlooked or forgotten when a Credit Union is considering a Core Conversion. Subsequent blogs will be posted weekly and highlight each item in-depth. Make sure to follow along with the series on Olenick.com.
A core conversion has many, many components and a small amount of time to account for all of them. You have already read about a test and defect management repository, the importance of performance testing, and the risk of not testing your core data.
The next blog post in this series of often-neglected components of a core conversion is about downstream testing. Many organizations believe that they have gone far enough when they’ve tested the core application, but what about the data that moves in and out of the core system? With a project as complex as a conversion, there is more to test than just the core.
DOWNSTREAM TESTING ITEMS
We call this “downstream” testing because many of the applications you will need to test are receiving data from your core.
There are two primary things to test:
- That the data made it to the correct field in the downstream application
- That the data is formatted correctly
It is easier than one would think for fields to be misaligned following a conversion. Your testing will identify the fields that need to be re-mapped.
A formatting issue can result in a number of different errors, most of which are non-critical to your data integrity. While it is rare for formatting to cause a true data integrity issue, a money field that now reads $100.0000000000, as an example, is an embarrassing issue if it is visible to members in online banking or printed on a statement.
Generally, you want to look in two places for items to test. The first is data that is sent to third parties and members. Consider account statements, mortgage statements, invoices for loan payments, and teller and ATM receipts. You will need to ensure that the name is for the correct member and that the address is correct. Review the balances, interest rate, and amount owed to ensure each field is correctly mapped, as well as transaction information.
Verify that the name and address line up with the envelope window for mailed statements. Confirm that the headers, footers, and logo are in the right place, and that the page breaks meet your formatting specifications. It would be unusual for all of these to have shifted, but it is possible. If the new core is sending every character in the field, including all of the invisible spaces, data will end up being pushed into margins or a new row.
The second place to look for items to test is your reporting database(s). More than likely, you have at least one database housing critically important data. Think about where your Board report and call report data is housed. Much of that information is likely to come from your core.
Much like statements, you will want to review the values for accuracy, as well as headers, footers, and formatting. You may find that account numbers are displayed exponentially – 2.54×1011 instead of 254397684343 – an obvious and confusing mistake for any report. Double check that report titles and totals are correct.
RISKS OF FAILING TO PLAN FOR DOWNSTREAM TESTING
Many organizations neglect to plan for downstream testing when designing their overall test plan, but uncover defects after testing has kicked off. The result is a scramble to quickly plan, write test cases, and execute.
There are three risks to mitigate if you fail to plan ahead of time:
1. UNPLANNED EXPENSE.
There will be a cost for the overtime the existing test staff or the addition of last-minute resources will need to execute the unexpected work. If overtime and additional staff are not options, you may need to reprioritize your test workload. This could mean foregoing other, equally important test tasks. When there is a scramble for testers, you will also find yourself scrambling for developer time to address the defects the testers find.
2. REPUTATION RISK.
Some of your downstream testing will involve third-party organizations. Just as you may be scrambling for resources to execute testing, your partners may have to scramble to refresh test data, coordinate time with you to review results, and have their staff available for questions.
3. PUSHING YOUR GO-LIVE DATE.
In some organizations, this is the worst-case scenario, particularly when the date has been communicated to the Board of Directors and partners, and activities have been arranged for the day or week of the go-live.
PLANNING FOR TESTING
If you are fortunate enough to have time to plan for downstream testing, you should start with a report clean up. More than likely, your requirements state to “replicate existing reports,” but many of them are probably unused. Meet with the business units who own or use those reports to identify what can be eliminated, and then prioritize what is left for testing. This will save you an enormous amount of test effort (and time!) later in your project.
Find out from each department if they have any home grown applications that are importing important data from your core. Often, these are used to create daily, weekly, or monthly status reports for management. They may be using Excel or Access to manipulate the data. If possible, use this planning time to move their status report into your primary reporting database. If that is not possible, make sure you plan for those departments to test their applications using data from the new core well before the production go-live.
Work with your IT department to identify all of the instances where data is being sent outside of your organization. While we already discussed statement and invoice vendors and ATMs, there are many more examples that may be applicable to your organization.
Are you sending to or receiving information from one or more credit agencies? What about a credit card processor? Have you thought about your marketing agency? Those organizations likely have a very specific format and set of core fields that need to be sent or received.
You will need to reach out to your contacts at each organization to notify them of your upcoming conversion and coordinate testing.
Online and mobile banking testing often have their own project plan embedded within the core conversion project plan. The data your members access will be pointing to an entirely new database, tables, and fields, leaving lots of room for incorrect data and incorrect formatting to appear.
If your core conversion is changing your members’ view from individual accounts to an aggregated view of every account associated with their SSN, your test effort will increase sizably.
For some credit unions, it is overwhelming to build a test strategy for all of the downstream integrations, as well as write and execute the test cases. There are third-party organizations with the experience and tools to do this efficiently.
HOW TO EXECUTE TESTING
Whenever possible, your downstream testing should be baked into the test case for a related process. When that is the case, you execute these final test steps as part of your project’s functional test phase, and update your repository with the results.
As with all other test phases, you are likely to find defects. Follow your test manager’s guidance for selecting the defect severity, and they will be assigned to a developer or the new core provider for review.
If your organization has accounted for downstream testing in your test plan, you are ahead of many other organizations, most of which make the risky assumption that there will be few issues. Your core is the most critical system in your organization, and a conversion project is the highest risk project you can undertake. The importance of thorough test planning and execution cannot be understated enough. Plan, execute, and log your results.
In addition to a test repository , performance testing , data integrity , and downstream testing, many organizations neglect to convert their test cases to a regression test suite. Why is that important? Our next blog post puts the cherry on this complicated core conversion sundae.
Olenick is a global software testing firm with headquarters in Chicago, IL.