DATA ENGINEER (MANUFACTURING)

Job Description:

We are seeking multiple Data Engineers with expertise in Python, Databricks, SQL, and Azure to develop and manage data pipelines in a manufacturing environment. The ideal candidate will have a strong background in integrating data from various sources, including relational databases, PLCs, and IoT modules, into cloud-based platforms like Azure. This role will focus on building robust data architectures to support advanced analytics, real-time monitoring, and alarm generation.

Key Responsibilities:

• Develop Data Pipelines from FactoryTalk to Azure Databricks: Design and implement data pipelines to transfer data from relational databases, such as FactoryTalk, into Azure Databricks for advanced analytics.
• Collect Data from PLCs and IoT Modules: Develop and implement pipelines to collect real-time data from PLCs and IoT devices using MQTT protocols and transfer it to Azure for processing and alarm generation in Databricks.
• Create Data Pipelines Between Relational Databases: Design, develop, and maintain data pipelines between multiple relational databases, ensuring data synchronization, consistency, and quality for real-time reporting and analysis.
• Optimize Data Integration Workflows: Collaborate with cross-functional teams to ensure seamless data integration and transformation across various systems, including IoT devices, PLCs, and relational databases.
• Monitor and Troubleshoot Pipelines: Ensure data pipeline reliability, troubleshoot issues, and optimize performance for scalability and efficiency.
• Document and Maintain Data Architecture: Create and maintain technical documentation for data pipelines, ensuring easy scalability and troubleshooting for future projects.
• Stay Updated on Emerging Technologies: Keep up with the latest developments in data engineering tools, cloud platforms, and industrial IoT systems relevant to the manufacturing sector.

Requirements:

• 6+ years of experience in data engineering, with strong expertise in Python, Databricks, SQL, and Azure.
• Proven experience in developing data pipelines from relational databases (e.g., FactoryTalk) to Azure cloud environments.
• Strong knowledge of PLCs, IoT modules, and using MQTT protocols for real-time data collection and transfer.
• Hands-on experience building and optimizing data pipelines between relational databases in a manufacturing context.
• Strong understanding of ETL processes, data transformation, and pipeline optimization for scalability and reliability.
• Experience with industrial data systems, control systems, and cloud-based platforms in a manufacturing environment.

Vertical

Technology