Design and implement scalable, reliable, and efficient data architectures to support business needs.
Define data models, schemas, and storage solutions (e.g., data lakes, data warehouses, NoSQL databases).
Evaluate and select appropriate technologies, tools, and frameworks for data processing, storage, and analytics.
Ensure the architecture supports both batch and real-time data processing based on client needs.
Knowledge on data security, and compliance with regulations (e.g., GDPR, CCPA).
Build and maintain robust ETL/ELT pipelines to ingest, transform, and load data from various sources.
Optimize data pipelines for performance, scalability, and cost-efficiency.
Implement data quality checks and monitoring to ensure data accuracy and reliability.
Automate data workflows and integrate with CI/CD pipelines for seamless deployment.
Design and manage cloud-based data infrastructure (e.g., AWS, Azure, GCP).
Implement infrastructure-as-code (IaC) for provisioning and managing resources.
Collaborate with business stakeholders, product managers, and analytics teams to translate business requirements into technical solutions.