Bayt.com
https://bayt.page.link/jgSK4379orZ5uxTA7
Back to the job results
Other Business Support Services
Create a job alert for similar positions

Job Description

We are seeking a skilled and experienced Data Engineer with expertise in PySpark and Azure Data Factory to join our team. The ideal candidate will have a minimum of 5 years of professional experience in data engineering and a strong understanding of data processing, transformation, and integration. Knowledge of Power BI or any other data visualization tool is highly preferred.


Key Responsibilities:


  1. Design, develop, and maintain scalable data pipelines using PySpark and Azure Data Factory.
  2. Transform and integrate large datasets from multiple sources to support analytical and business intelligence requirements.
  3. Collaborate with data analysts, scientists, and other stakeholders to define data requirements and deliver solutions.
  4. Optimize data workflows for performance, scalability, and cost efficiency.
  5. Monitor and troubleshoot data pipelines to ensure high availability and reliability.
  6. Implement best practices for data governance, security, and compliance.
  7. Utilize Power BI or other visualization tools to support the creation of dashboards and reports (preferred).

Requirements

Required Skills and Qualifications:


  1. Experience: Minimum 5 years of hands-on experience in data engineering.
  2. Technical Expertise:
    • Strong knowledge of PySpark for data processing and transformation.
    • Proficiency in Azure Data Factory for building and managing ETL/ELT workflows.
    • Familiarity with data storage solutions (e.g., Azure Data Lake, SQL databases).
    • Experience with version control systems like Git.
  3. Visualization Skills: Knowledge of Power BI or other visualization tools is highly desirable.
  4. Problem-Solving: Excellent analytical and problem-solving skills to handle complex data challenges.
  5. Communication: Strong verbal and written communication skills to collaborate effectively with cross-functional teams.
  6. Preferred Skills:
    • Understanding of cloud services in Azure (e.g., Azure Synapse, Logic Apps).
    • Familiarity with Agile methodologies and CI/CD pipelines.
Education:

Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field.


Apply on company site Email to Friend Complete Questionnaire
You have reached your limit of 15 Job Alerts. To create a new Job Alert, delete one of your existing Job Alerts first.
MANAGE
Similar jobs alert created successfully. You can manage alerts in settings.
MANAGE
Similar jobs alert disabled successfully. You can manage alerts in settings.
MANAGE