Distance no Object for Big Data in the Cloud
Added Tuesday 05 September 2017
Steve Jones of IBM Cloud – Aspera explains why trends in big data make high-speed transport to the cloud a must.
Innovative organisations are seizing the promise of cloud. A media company wanting to use the cloud to distribute massive amounts of entertainment content… a national meteorological agency aiming to model, understand and predict weather patterns…a life sciences institute, with researchers across the globe, needing to share huge data sets or medical images in the cloud…
Whatever their sector, these organisations often deal with data sets measuring in tens of terabytes or even in petabytes. And therein lies the problem: how do they quickly and securely migrate files to the cloud to get big data in and out of the cloud service provider’s datacentre?
Contending with latency
Traditional file-sharing tools such as FTP and HTTP were never designed for this kind of task and performance seriously degrades over distance, as network latency kicks in. A few numbers show the problem isn’t going to go away:
- Cisco’s annual Visual Network Index predicts that video will account for a whopping 82% of all Internet Protocol (IP) traffic by 2021
- 85% of managers with responsibility for data storage in healthcare and life sciences plan to increase cloud compute resources, but 72% think IT infrastructure will create research bottlenecks
- 95% of healthcare and life sciences organisations say their existing file collaboration tools don’t meet their needs
So, what’s the answer?
It may come as a surprise in this day and age, but some big-data-in-the-cloud users are still ‘hand-carrying' data by shipping hard disk drives to their cloud provider(s). In taking this route, they run all the risks of introducing time-lags into projects and suffering loss from theft or mishandling. What’s more, there still remains the problem of moving the data within the cloud, between remote storage and across compute nodes.
To realise the promise of cloud, these organisations need the means to transport large volumes of data securely, at high speed to, from and across cloud infrastructures.
Some, such as the hugely successful Netflix (approaching 104 million subscribers worldwide!), have already gained the capability to shorten file transfer times from many hours to mere minutes.
To find out more, take a look at our short (2½-minute) video or click to read a whitepaper on ‘Taking Big Data to the Cloud’.