In today’s episode of IT in Plain English, Tammy Ven Dange answers, “What’s the difference between a data warehouse and a data lake?”
Both of these terms have to do with the way you might store data for reporting and dashboards across multiple systems
- A data warehouse is organised and stored in tables – kind of like what you see in a huge Excel spreadsheet.
- In a data lake, the information is stored in an unorganised way until you need it.
To help you understand the difference, picture a bookshelf full of books. They are already sorted in multiple ways, such as alphabetically by title, author or subject. When you want to find a book on the bookshelf, you just run one of the already defined searches like “all books written by J.K. Rowling”, and it will come back with the results.
This is a really fast way to find information, but someone had to first organise the bookshelf in that way – which takes time. And, if you decided you wanted just find the books written before 2000, it would be impossible to find them if the publication date wasn’t visible.
Now suppose instead that all the books were stored randomly in a large box. And you once again want to know all the books written by J.K. Rowling. This time, someone must go through the box to find them.
This will obviously take longer, the bigger the box. However, no one had to sort the information to begin with, for searches that may never be needed. And if you wanted to just find those books from J.K. Rowling with the colour red on the cover, you could do that.
That makes data lakes more flexible than a data warehouse, but again slower to run reports.
The storage method you choose will largely depend on a few criteria, such as how often the data is updated and used.
So, there you have it in plain English.
If you have an IT question you want answered, feel free to connect with me here on my website or on LinkedIn and send me a message. I just might answer it on this show.
Sign-up here to subscribe to the “Executive with a Cause” newsletter.
Tammy Ven Dange is a former charity CEO, Not for Profit Board Member and IT Executive. Today she helps NFPs with strategic IT and data decisions.