Job Description
About the job DATA ENGINEER (MANUFACTURING)
Job Description:
We are seeking multiple Data Engineers with expertise in Python, Databricks, SQL, and Azure to develop and manage data pipelines in a manufacturing environment. The ideal candidate will have a strong background in integrating data from various sources, including relational databases, PLCs, and IoT modules, into cloud-based platforms like Azure. This role will focus on building robust data architectures to support advanced analytics, real-time monitoring, and alarm generation.
Key Responsibilities:
- Develop Data Pipelines from FactoryTalk to Azure Databricks: Design and implement data pipelines to transfer data from relational databases, such as FactoryTalk, into Azure Databricks for advanced analytics.
- Collect Data from PLCs and IoT Modules: Develop and implement pipelines to collect real-time data from PLCs and IoT devices using MQTT protocols and transfer it to Azure for processing and alarm generation in Databricks.
- Create Data Pipelines Between Relational Databases: Design, develop, and maintain data pipelines between multiple relational databases, ensuring data synchronization, consistency, and quality for real-time reporting and analysis.
- Optimize Data Integration Workflows: Collaborate with cross-functional teams to ensure seamless data integration and transformation across various systems, including IoT devices, PLCs, and relational databases.
- Monitor and Troubleshoot Pipelines: Ensure data pipeline reliability, troubleshoot issues, and optimize performance for scalability and efficiency.
- Document and Maintain Data Architecture: Create and maintain technical documentation for data pipelines, ensuring easy scalability and troubleshooting for future projects.
- Stay Updated on Emerging Technologies: Keep up with the latest developments in data engineering tools, cloud platforms, and industrial IoT systems relevant to the manufacturing sector.
Requirements:
- 6+ years of experience in data engineering, with strong expertise in Python, Databricks, SQL, and Azure.
- Proven experience in developing data pipelines from relational databases (e.g., FactoryTalk) to Azure cloud environments.
- Strong knowledge of PLCs, IoT modules, and using MQTT protocols for real-time data collection and transfer.
- Hands-on experience building and optimizing data pipelines between relational databases in a manufacturing context.
- Strong understanding of ETL processes, data transformation, and pipeline optimization for scalability and reliability.
- Experience with industrial data systems, control systems, and cloud-based platforms in a manufacturing environment.
Vertical
Technology
Job Details
-
Job Location
-
India
-
Company Industry
-
Other Business Support Services
-
Company Type
-
Unspecified
-
Employment Type
-
Unspecified
-
Monthly Salary Range
-
Unspecified
-
Number of Vacancies
-
Unspecified