GlobalMart, a rapidly growing e-commerce startup, faces several data management challenges that impact its ability to generate reliable insights and make timely business decisions. Key issues include:
- Data Quality Issues: Inconsistencies in data formats, missing values, and duplication create inefficiencies in processing.
- Slow Data Transformations – As data volume increases, sluggish transformation processes delay critical insights.
- Lack of Streamlined Workflows – Inefficient data processing and handling of invalid data disrupt operations and reduce overall efficiency.
- Security Risks – Improper handling of sensitive credentials and access keys poses potential security threats.
- Unreliable Deployment & Testing – The absence of a structured framework makes changes to data pipelines error-prone, increasing operational overhead.
These issues led to lack of trust in data systems rendering them useless. In this project you will be spending time to implement the following architecture that addresses all the problems that Globalmart is currently facing in their data systems
TM
