Location: Bangalore
Experience: 6 – 9 Years
Notice Period: Immediate to 15 Days
Overview
The Azure Data Engineering Lead will play a critical role in shaping data-driven strategies and solutions within our organization. Working alongside a talented team, you will be responsible for designing and implementing robust data solutions on the Databricks platform using Spark. This role requires a strong technical background along with a deep understanding of business needs to ensure data solutions align with organizational objectives.
As part of a collaborative team culture, you will lead projects that drive innovation and efficiency. Your contributions will directly impact our data architecture and analytics capabilities, enabling informed decision-making through data insights.
Responsibilities
- Develop and maintain data pipelines for seamless data flow across systems.
- Create and implement business logic using Spark within the Databricks environment.
- Write and optimize complex SQL queries to enable meaningful data analysis.
- Design and implement ETL workflows using Azure Data Factory (ADF).
- Collaborate with cross-functional teams to identify data needs and requirements.
- Ensure data integrity and optimal data storage solutions using Delta Lake.
- Translate business needs into technical specifications and solutions.
- Contribute to architectural design and best practices for data management.
- Stay updated with industry trends and emerging tools to enhance data engineering capabilities.
- Provide support and mentorship to junior data engineers when necessary.
Key Skills & Requirements
- Bachelor’s degree in Computer Science, Information Technology, or a related field.
- 6-9 years of experience in data engineering, with a strong background in ETL process development.
- Proficient in PySpark and Python programming languages.
- Strong experience in writing advanced SQL queries for data retrieval and analysis.
- Hands-on experience with Azure components, including Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Azure Active Directory (AAD), and Azure Key Vault.
- Strong understanding of Delta tables and data lake architectures.
- Experience in data architecture and ETL design.
- Ability to transform complex business requirements into effective technical solutions.
- Experience in the automotive domain is a plus.
- Knowledge of Data Mesh concepts is advantageous.
If you are passionate about data engineering and eager to work in a dynamic environment, we encourage you to apply!
