Continuing on our fictitious financial company, Starlight, series of posts, here is how to set up a data lake on AWS with security as the primary thought.
Introduction
In the fast-moving financial industry, data is a core asset. Starlight Financial needs to use vast amounts of data for decision-making, improving customer experience, and keeping ahead of its rivals. Consider a data lake: it’s a vital part of modern data architectures, letting enterprises store both structured and unstructured data in large quantities of any kind whatsoever. Tony Hoare famously observed that with great data comes great responsibility — and so it is. Eventually, it will be some comfort to know that one of the most important steps for consultancy in validating big data architectures using AWS services has been elucidated. That is to say: test them just like any other system you might use. This is a guide to establishing a highly secure data lake using AWS services, specifically focused on the needs of financial institutions, written by us using a blog structure.
