Azure Data Engineer
JOB DESCRIPTION
Job Summary:
We are looking for a skilled Data Engineer to join our team. The ideal candidate will have a strong background in Python, SQL, and PySpark and Databricks with experience in developing and managing data pipelines and databases.
Experience Required- Minimum 4 years (Must), Please do not apply if you do not have relevant experience.
Responsibilities:
- Design, build, and maintain efficient and reliable data pipelines and databases.
- Develop and implement ETL processes to integrate data from various sources into a unified view.
- Work closely with data scientists and analysts to ensure the quality and accuracy of data.
- Troubleshoot data-related issues and provide solutions in a timely manner.
- Manage data security, privacy, and compliance requirements.
- Monitor performance and capacity of data pipelines and databases.
- Continuously improve data engineering processes and practices.
- Experience in Azure Log analytics, Data Shares, Data factory and Azure synapse is must.
Requirements:
Excellent problem-solving skills and attention to detail.
Bachelor’s or Master’s degree in Computer Science or a related field.
4+ years of experience in data engineering.
Strong programming skills in Python and SQL.
Experience with Spark and Databricks and related tools such as PySpark or Spark.
Experience in Azure Log analytics, Data Shares, Data factory and Azure synapse is must.
Familiarity with data modeling, database design, and data warehousing concepts.
Proficiency in Linux and command-line tools.
