I would like to discuss an issue one can likely experience with agile development processes and systems which data needs to be maintened during upgrades:
A customer care application for a PC retailer was developed so far and the software is running pretty well. Because warranty fraud is increasing, management decides to put in a check with the hardware supplier to check the validity of the serial number. But when checking the numbers against the webservice of the supplier almost all existing data gets rejected, because the serial numbers were not at all or incorrectlly entered so far. The supplier provides a validation algorithm which is added to the application. But after the next release of the software many employees complain about the system rejecting changes or updates of existing customer data. Also a third party financial system no longer is able to update the payment status.
This scenario describes that existing data is very likely to cause trouble when adding further validation rules. One could limit the data validation to the user interface, which would help the financial system, but still call center agents would no longer be able to update the invalid data.
Automatic data upgrade/conversion/fixing procedures are no promising candidates, because often the information is just missing and need to be obtained manually (e.g. from the customer).
Here some ideas to limit the effect of this problem:
- Before adding new rules, generate a list of existing data which will be affected and let them be fixed on beforehand
- Show rules for a given time only as warning, not as error
- Use additional information to limit validation rules to new data only
- When adding new data fields, think about at least a basic validation
Have you made similar experience? Do you have any better ideas to fight this issues? Or even a procedure to avoid them?