I’ve been following the current fascination (obsession?) with Big Data with great interest. In the mid and late 1990s I worked on a strategic development programme in which a big UK insurer developed management information data warehouses. This piece discusses some of the issues and problems we had to deal with.
I also spent a few years as a computer auditor. This entailed frequent fraud investigations, crawling over huge datasets, trying to spot patterns and make sense of suspected frauds. I talk about this in my follow up article.