Job Description
Contexte du recrutement et définition de poste
We are looking for an experienced DataOps Engineer to join our team. This role focuses on building and maintaining robust on-premise cloud infrastructures and data pipelines. You will work with a diverse range of tools and technologies, ensuring the reliability, scalability, and efficiency of our data and cloud ecosystems.
Key Responsibilities:
- Data Stack Infrastructure Management: Deploy and maintain on-premise cloud environments using Hadoop, Kubernetes, Helm, and Minio.
- CI/CD Implementation: Build and maintain infrastructure as code (IaC) with Terraform, automate configuration management with Ansible, and streamline CI/CD workflows using Jenkins and GitLab CI.
- Monitoring and Observability: Oversee system performance using monitoring tools such as Grafana, Kibana, and ELK Stack to proactively address issues and ensure system health.
- Data Operations Management: Oversee the performance of data pipelines and implement proactive measures to address issues.
- Collaboration: Work closely with data engineers, data scientists, and stakeholders to align data delivery processes with business goals.
- Machine Learning Workflow Optimization: Integrate tools like MLflow to manage and streamline machine learning workflows.
Profil recherché
- Bachelor’s degree in Computer Science, Data Engineering, or a related field.
- Proficiency in managing on-premise cloud environments with Kubernetes, Helm, and Minio.
- Strong experience with Hadoop ecosystems for big data processing.
- Experience with CI/CD tools, specifically Jenkins and GitLab Actions.
- Familiarity with task scheduling and workflow orchestration tools such as Apache Airflow and Celery.
- Hands-on expertise in Terraform and Ansible for infrastructure automation.
- Strong problem-solving and communication skills.
Bonus:
- Experience with monitoring and observability tools like Grafana, Kibana, and ELK Stack.
- Experience with MLflow for machine learning lifecycle management is a plus.
- Knowledge of Kafka for real-time data streaming and messaging.
