BIG Data Engineer - for Digital Engineering Company

Apt Resources is searching for a talented BIG Data Engineer to join a leading Digital Engineering Company based in Abu Dhabi. The ideal candidate will have a strong background in batch data processing, the development and maintenance of scalable ETL/ELT pipelines, and expertise with cloud platforms like Azure. This role focuses on designing and building high-performance data solutions to meet business needs, ensuring efficient data processing, and upholding high data quality standards.

Key Responsibilities:

• Design, develop, and maintain scalable batch ETL/ELT pipelines using Azure Data Factory and other Azure services (such as Azure Table Storage, Azure Data Lake, etc.).
• Collaborate closely with data architects and cross-functional teams to understand business requirements and translate them into technical data solutions.
• Optimize data processing pipelines for performance, reliability, and cost-effectiveness.
• Perform data integration, cleansing, and validation to ensure high data quality and consistency across systems.
• Monitor, troubleshoot, and resolve issues related to batch processing jobs to ensure timely and accurate data delivery.
• Ensure compliance with data governance and security standards, and implement best practices in data management.
• Contribute to the continuous improvement of data infrastructure, implementing new tools and technologies to support evolving business needs.
• Work on integrating data from various sources, ensuring seamless data flows into storage and processing systems.