Farewell to Data Lakes

Billy Chan, Country Manager, Hong Kong & Macau, Pure Storage

Billy Chan, Country Manager, Hong Kong & Macau, Pure Storage

Businesses care less and less about data lake in AI era – and that’s a good thing. The power of data today comes in setting it free.Storage has a unique opportunity to become much more than a siloed repository for the deluge of data constantly generated in today’s hyper-connected world, but rather a platform that shares and delivers data to create value. The contrast between a data lake and a data hub is profound. A data lake is designed to store data. A data hub is designed to share and deliver data. The former is engineered with legacy DAS-based technologies. The latter is a modern system based on an all-flash, cloud-like architecture. Data is today’s challenge; a data hub is today’s solution.

Spending on data-intensive AI systems will reach $19.1 billion this year and rise to $52.2 billion in 2021, according to IDC. Projects run the gamut from automated customer service agents, shopping and product recommendations to health and safety use cases like automated cyber threat detection and AI-powered medical research, diagnosis and treatment.

Some have referred to this opportunity as the “Fourth Industrial Revolution.” That’s a massive undersell. The last industrial revolution was driven by the assembly line – a feat of strategic engineering that helped build a car, faster. Today, we’re talking about feats of engineering that allow cars to drive themselves. It’s apples to atoms. A new generation of tools, fueled by an ability to ingest, store and deliver unprecedented amounts of data, are driving a tidal wave of innovation previously relegated to the realms of science fiction.

A new architecture means one that can share and deliver data anywhere, anytime – a data hub that can be the way forward for modern business

Data’s role in the future of business cannot be overstated. According to a survey conducted by MIT Technology Review and commissioned by Pure Storage, an overwhelming 86 percent of leaders in Hong Kong say data is the foundation for making business decisions, and also leaders agree that it is critical that they analyze data as much as possible to gain competitive advantage given they have so much data now.

Now, it's time for the collective storage industry to rise to meet that challenge and send into the scrap heap outdated architectures that are fundamentally incompatible with one another, don’t communicate and don’t do the one thing required to realize data’s full value – share. A new architecture means one that can share and deliver data anywhere, anytime – a data hub that can be the way forward for modern business.

A true data hub must include four foundational elements:

● High throughput for file (mainstream) and object (future, cloud-native) storage: Backup and data warehouse apps need high throughput to speed queries and analytics;

● True seamless scale-out: Data lake redefined as scale-out infrastructure, enabling applications to run at any scale;

● Multi-dimensional performance, built to respond to any data type with any access pattern: Streaming analytics requires storage to drive any data, small or large, with any I/O pattern;

● Massively parallel architecture: AI clusters, with 1000s of cores and GPUs, need storage to be massively parallel as well.

These four features are essential to unifying data. Too much data today remains stuck in a complex sprawl of silos. Each is useful for its original task. But in a data-first world, silos are counter-productive. Silos mean organizational data can’t do work for the business while it’s not being actively managed.

For organizations that want to keep data stored, a data hub does not replace data warehouses or data lakes. For those looking to unify and share their data across teams and applications, a data hub identifies the key strengths of each silo, integrates their unique features and provides a single unified platform for business.

Think of storage like a bank, or an investment. We put our money in banks, or the stock market or our neighbor’s startup because we want it to do something other than sit there. So, we put our money into systems that make it work for us. Modern organizations need to do the same with data.