Senior Data Automation Engineer
This job is available in 2 locations Job Type Regular Job Id R230089 Posted Date: 01/29/2025 JOB DESCRIPTION
Position:
Senior Data Automation Engineer
Job Description:
As a Data Automation Engineer, you will play a crucial role in developing and maintaining automated data processing systems. You will work closely with data analysts, and other stakeholders to design, implement, and optimize data pipelines, ensuring the efficient and accurate flow of data throughout the organization.
What You'll Do:
Develop and implement automated data pipelines for ingesting, processing, and storing structured and unstructured data. Good knowledge of AI solutions to analyze, extract, compare, and summarize structured and unstructured data. Good knowledge about open-source AI tools and how they will work in data automation stages, Good knowledge about AI models and using them through (Hugging Face, FIREWORKS) Good experience in data collection methods. Good experience in Rules engines. Integrate data from various sources in real-time and batch mechanisms and perform transformations to ensure consistency, accuracy, and reliability. Implement data quality checks and monitoring mechanisms to identify and address issues proactively. Write code and scripts for automation of data processes using languages such as Python, Java, or Scala. Leverage ETL (Extract, Transform, Load) tools and frameworks [Apache spark, AWS Glue] for efficient data processing. Collaborate with other intelligent data teams. Ensure seamless data flow for machine learning and analytics applications. Implement monitoring systems to track data pipeline performance and identify areas for optimization. Continuously optimize data processes for efficiency and scalability. Document data processes, system architecture, and codebase for knowledge sharing and future reference. Keep documentation up-to-date with any changes or enhancements to the data infrastructure. Implement and adhere to data security and compliance standards. Collaborate with security teams to ensure data protection and privacy.
You're a Perfect Fit If You Have:
3+ years of experience in data engineering, and automation. Strong understanding of data pipelines and data lifecycle management. Proficiency in scripting languages (Python). Good knowledge of AI solutions for understanding documents. Good knowledge about AI models and how to use them. Experience with data automation tools (Airflow). Excellent analytical and problem-solving skills. Strong communication and collaboration skills to work effectively across teams. Knowledge of cloud platforms (AWS, Azure) for data processing. Knowledge of data warehousing and data quality methodologies. Experience with data visualization tools (Tableau, Power BI)
Location:
EG-Cairo, Egypt (Al Emdad & Al Tamween)
Time Type:
Full time*
Job Category:
Information Technology
Recommended jobs based on your profile
No recommendations found
Location City Stars - Al Emdad & Al Tamween, Tower F2, 3rd floor, Nasr City, Cairo, 11599, Egypt Job Type Regular Location City Stars - Al Emdad & Al Tamween, Tower F2, 3rd floor, Nasr City, Cairo, 11599, Egypt Job Type Regular Location City Stars - Al Emdad & Al Tamween, Tower F2, 3rd floor, Nasr City, Cairo, 11599, Egypt Job Type Regular Location SKAV Seethalakshmi, No. 21 Kasturba Rd, near Chinna Swamy Stadium, Bangalore- 560001, Karnataka, India Job Type Regular Job Id R223266 Posted Date 07/19/2024 Location City Stars - Al Emdad & Al Tamween, Tower F2, 3rd floor, Nasr City, Cairo, 11599, Egypt Job Type Regular Job Id R227259 Posted Date 02/03/2025 Location Lorven Tiara, 3rd Floor, Survey No. 34,, Kothaguda Junction, Kondapur, RR District, Hyderabad- 500084, Andhra Pradesh, India Job Type Regular Job Id R230248 Posted Date 02/03/2025 Location Final Plot No 394 & 395, Village Ognaj, Ghatlodiya, Ahmedabad- 380060, Gujarat, India Job Type Regular Job Id R229585 Posted Date 02/06/2025 The information you provide to the chatbot will be recorded to improve your experience. Please read our privacy policy to see how we are storing and protecting your data Today 02:28 pm Bot message
