Let’s suppose that right after you finish this fine blog post you want to sit down and binge-watch everything that was uploaded to YouTube today.
Well, I’d advise against that since it’ll take, oh I don’t know, the rest of your life?
Every minute over 500 hours of video are being uploaded to the YouTube platform and if you do the math you would need 82 straight years to watch all the content pushed onto YouTube servers over just one day and YouTube is just one site, so how big is the rest of the Internet?
It’s estimated by Statista that current worldwide data centre capacity is around 2000 exabytes (2 billion terabytes), and that’s a lot of storage space. But these days it’s also estimated that the world produces about two-and-a-half exabytes or 2.5 million terabytes of data every single day.
Now, not all of that ends up on the internet but it’s still clear that it’s not enough to keep up with our needs indefinitely especially as the trend these days is to just throw our everything into the cloud instead of actually bothering to delete anything.
This growth and data is not expected to slow down any time soon a recent study by IDC projects that in 2025, just five years from now, the data sphere which includes everything from amazon’s cloud servers to your cell phone will be up to over 160,000 exabytes of data with about 40% of that needing to be saved somewhere.
That works out to about 64 billion terabytes worth of medical records memes and tech videos that will need a server to call home. It’s not surprising then that big tech is trying to get more data centres built as quickly as they can and new technologies like Heat Assisted Magnetic Recording or HAMR are being developed to increase the capacity of traditional hard drives.
In fact, Seagate is aiming to use HAMR to push out a 100 terabyte hard drive sometime around the middle of this decade.
But if we can’t turn out high capacity hard drives and build enough shiny new data centres to accommodate our thirst for data, not to mention find energy sources to power them all, what are the alternatives other than just purging some of the information.
One option is to use algorithms to compress the data but short of some kind of big breakthrough in middle-out compression we’re not anticipating huge advancements there so perhaps more interesting is the research that’s being conducted on entirely new forms of storage
Instead of using magnetic particles, for example, scientists are looking at storage solutions that take up far less physical space, such as DNA based storage, where each base pair represents a bit or nanomaterials like a glass disk that can hold 360 terabytes yet is no bigger than a coin.
However, even if we find a way to encode data using subatomic particles, in other words, we find the most space-efficient way to store data that the laws of physics will allow we would still run out of space in a few centuries if we keep generating digital data at the rate that we are right now
So the bottom line is this I wouldn’t worry too much for the moment about the Internet running out of space, but at some point, we might want to collectively think about just how many variants of the jealous girlfriend meme we really need to keep on our hard drives.
