Job Description
Job DescriptionTitle: Azure Data Engineer
Location: Boston, MA :: Hybrid
Rate: $50/hr
Duration: 12 Months
Job Summary
We are seeking an experienced Azure Data Engineer with 7 years of experience in designing, developing, and maintaining scalable data engineering solutions using Microsoft Azure. The ideal candidate will have strong expertise in Azure data services, SQL, Python, ETL/ELT pipelines, and data warehousing, with a proven ability to transform complex data into reliable, high-quality datasets for analytics and business intelligence.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using Azure Data Factory (ADF), Azure Databricks, and Azure Synapse Analytics.
- Build and Eligible to workimize ETL/ELT workflows to ingest, transform, and process structured and unstructured data from multiple sources.
- Develop complex SQL queries, stored procedures, and performance-Eligible to workimized data models.
- Use Python and PySpark to perform large-scale data processing and transformation.
- Design and manage data lakes and lakehouse architectures using Azure Data Lake Storage Gen2.
- Implement data warehousing solutions and dimensional data models to support reporting and analytics.
- Develop data integration solutions for batch and, where required, real-time data processing.
- Implement data quality checks, data validation, error handling, and monitoring mechanisms.
- Eligible to workimize pipeline performance, query execution, storage utilization, and overall processing costs.
- Apply security best practices, including role-based access control (RBAC), managed identities, and secure data access.
- Collaborate with data architects, data scientists, BI developers, business analysts, and other stakeholders.
- Maintain technical documentation, troubleshoot production issues, and support continuous improvement initiatives.
- Use Git and CI/CD practices to manage source code, deployments, and version control.