Using the right tools for the job
Therese Coffe,Pensions Secretary, conceded that UK government figures relating to COVID-19 infections may well be inaccurate after what is referred to as a computer glitch. The ensuing blame game has started with Baroness Dido Harding, who runs the Test & Trace operation saying "we report the data when they [PHE] send it. We didn't get it". We now know that the computer glitch was infact coursed by data been processed using Microsoft Excel. The data had exceed the maximum size and has now been resolved by splitting the Excel files into batches.
Microsoft Excel is an excellent tool for advanced data organisation and has one of the most complete data organisation and analysis systems for the common consumer, but has limitation. Accountants and software engineers understand these limitation, not least the fact that complex spreadsheets relying on formulae can be corrupted very easily, when data is input over a formula, or exceeds the expected range.
In situations where organisations have to come up with solutions on under pressure, many would use software solutions like Excel - or its open source equivalent OpenOffice Calc - to process large amounts of data, in order to get information. But, the decision making process is floored, if the people making the decisions, do not have the checks and balances in place to ensure the information is handled correctly. This needs to include consideration for data protection, data errors, user input, validation etc., It is also important to ensure you monitor the use of any solution, to make sure it works as expected.
It appears that these basic checks and balances were not in place and whilst the issue has now been picked up and rectified, it highlights the fact that using the right tool for the right job applies to digital transformation and technology, just as much as in any other application. This is why we use a formal process to identify the issues and outcomes, before defining a solution. It is also critical to have a procedures that ensures the solution is tested in terms of its performance and if it meets the previously identified outcomes.
Whilst it is understandable that a spreadsheet was used in this situation and it is easy to critise, it does highlight the need to learn from our mistakes. Government agencies are renowned for some pretty epic failures, when it comes to technology and to some degree that is understandable, as they are usually pretty huge applications, but that is not an excuse. If a system is critical, it should be treated as such and tested to ensure it meets its intended outcomes within the working environment.