and usage . Data lakes and data warehouses are both used for storing data , but they are not the same . A data lake is a large pool of raw data set for future extraction and analysis – it needs to be searchable , but that may be the extent of tooling provided .
“ The oil and gas industry was an early adopter of data lakes to land data for use cases such as minimising unplanned downtime and improving safety . A data warehouse is a repository for structured data supported by a combination of processes and tools to prepare data for a specific purpose . For example , warehousing is essential for the healthcare industry as it utilises it to strategise and predict outcomes , generate patients ’ treatments and share data with medical aid services .”
Lamb has worked for Dell for almost a decade , watching the global explosion of data and working to support the expansion of cloud infrastructure from cutting edge , niche technology to the foundation of modern digital society .
“ There has been a shift towards the use of cloud for data warehouse architecture in recent years as the services and capabilities have matured ,” he continues . “ There are three primary drivers for organisations looking at cloud for data warehousing :
• The inability to handle the speed and volume of multi-source data , especially IoT data ;
• The inability to find a single technological solution to collect , store , and organise data from disparate sources ;
• The inability to handle Big Data projects with a single database ;
“ The challenge is managing these data sources and only integrating the valuable data into the data warehouse .”
35
“ THERE HAS BEEN A SHIFT TOWARDS THE USE OF CLOUD FOR DATA WAREHOUSE ARCHITECTURE IN RECENT YEARS AS THE SERVICES AND CAPABILITIES HAVE MATURED ”
— Rob Lamb , Dell Technology
www . gigabitmagazine . com