The Dead Data Problem

Technological progress has proceeded at a breakneck pace ever since the dot-com bubble at the turn of the millennium. Just a scarce 25 years later, the economy has cycled through the rise and fall of thousands of companies. Each of these companies needs a website, a company email or some form of intranet, and sometimes an app for its products. On top of this, larger companies use productivity tools or CRMs. All these technologies produce and hold on to large amounts of data on a daily basis.

So where does all this data go when it’s not useful anymore?

Data Storyteller Dr. Selena Fisk (Ed.D), describes her experience seeing large numbers of cameras at an airport in Shanghai. Every angle of the airport is captured in video footage which produces some of the largest file sizes imaginable. In an article written for RIMPA Global, she writes, “It got me thinking about just HOW much data these cameras generate, where it is stored, how it is used, and how much ‘dead data’ exists.”

Even the smallest company generates data. Once it becomes too much for a single drive to contain, a business owner may choose cloud storage for flexibility and ease of use. Larger corporations may even lease or build physical storage centres, customising to their exact data storage needs. But what exactly is filling up those terabytes?

“Holding onto dead data isn’t good for our workplaces, our employees, clients or our communities,” concludes Fisk. “Not only is it risky if there happens to be a breach, it also makes finding and using the most useful data more difficult – as there is more data to wade through and make sense of.”

For more concrete numbers, Guillaume Roy, co-founder of employee engagement platform Workleap says, “We have 300 million terabytes of data every day that is getting generated around people. It can become very difficult for HR leaders and business leaders to get insights around this data.”

But Roy has different ideas on how to use a surplus of data that no one knows how to handle. He believes that such data isn’t truly dead, and that the vast information-processing powers of AI can turn data that’s impossibly tedious or even painful to go through—such as employee performance reviews—into useful and actionable insights.

This could potentially not just streamline timeframes, but even expand on conclusions by cross-referencing several information avenues. AI’s ability to recognise patterns could generate insights into employee contributions not just through quarterly metrics or manager observations, but by including data on employee collaboration and their impact across teams.

On the other hand, the more data you store, the more is at risk. The arms race between bad actors and security companies has been ongoing since before the dot-com bubble. 2FA and passwords with multiple special characters in them can protect the individual to a certain point, but what about the vast amounts of information stored by companies worldwide?

At the end of the day, ignoring dead data can put the health of your company at risk. Fisk points out that the Optus and Medibank data breaches would have been much less catastrophic had old data been purged—even past clients from over a decade ago were affected, and Australians are still feeling the impact from those leaks even today.

Fisk is a proponent of a more ‘minimalist approach’ to data collection. Perhaps only the most important documents require multiple copies, while one sighted copy is good enough for everything else. “This approach can help mitigate future dead data problems,” she writes in a LinkedIn Pulse, “by minimising the collection of extraneous data right from the very beginning.”

Our infrastructure is dated, but the increasing need for something new has spurred the progression of innovative ways to track and store data. Brilliant new companies, small and flexible, will be able to implement and maintain these changes more quickly, while slow-moving established corporations may find themselves having to make drastic changes.

Regardless, it’s clear that it’s time for a change in the fundamentals. Like with all new technologies, all other specialities revolving around the needs of AI will grow alongside it, and are worthy avenues to pursue. We’ll watch this space.