Rockwell Automation is a global technology leader focused on helping the world’s manufacturers be more productive, sustainable, and agile. With more than 28,000 employees who make the world better every day, we know we have something special. Behind our customers - amazing companies that help feed the world, provide life-saving medicine on a global scale, and focus on clean water and green mobility - our people are energized problem solvers that take pride in how the work we do changes the world for the better.
We welcome all makers, forward thinkers, and problem solvers who are looking for a place to do their best work. And if that’s you we would love to have you join us!
Job Description
Role Name:
Senior Data Engineer (Quality Assurance)
Reports to (role):
· Technical Leader
Key Stakeholder Relationships:
· Product owner, Technical Leader, Solution Architect, Business SME, Business Analyst
Key Responsibilities:
- Lead the design, and continuous improvement of automated test frameworks and testing processes for data pipelines on Databricks, ensuring high data quality and reliability.
- Maintain DevOps pipelines to fully automate testing processes, including unit, integration, and end-to-end testing, with minimal manual intervention.
- Focus on building scalable and reusable test frameworks that allow other data engineers to integrate tests as they build data pipelines.
- Automate test execution and reporting, ensuring test results are readily accessible to developers and team members through tools such as Power BI for streamlined decision-making and transparency.
- Collaborate with teams to ensure end-to-end data quality, performance, and security across the entire pipeline lifecycle.
- Act as a subject matter expert, leading defect management, triaging, and root cause analysis, and providing best practices for automated testing within data engineering.
- Monitor, and report on quality metrics, identifying trends and opportunities for improvement and ensuring compliance with quality standards.
- Conduct training, workshops, and documentation for data engineering teams to promote best practices in automated testing and framework use.
- Stay current with advancements in QA tools, and methodologies to enhance the data engineering QA landscape and processes.
Skills, Knowledge, Experience and Education
- Bachelor’s degree in Computer Science, Data Science, Engineering, or a related field (Master’s preferred).
- 5+ years of experience in data engineering, with a strong focus on quality assurance within data pipelines.
- 3+ years of experience with Databricks, Python, and PySpark for data manipulation and engineering tasks.
- 2+ years of experience with automated testing frameworks, e.g. Pytest, Flake8, for data quality and validation.
- 3+ years of experience in building data validation, data reconciliation, and DataOps frameworks within cloud-based data pipelines.
- 1+ year of experience using Azure DevOps pipelines to automate testing execution.
- Strong experience in building API integrations for source system data reconciliation to ensure data consistency and integrity. Prior experience working with SAP ECC and BODS is a plus.
- Extensive experience working within the Azure environment, with knowledge of cloud data infrastructure and services on Azure.
- Experience with DevOps practices and CI/CD processes, particularly as they relate to automated data pipeline testing.
- Knowledge of SQL, data modeling, and best practices for data quality testing and governance.
- Optional: Experience building and delivering Power BI reports to present test results and quality metrics to stakeholders.
- Excellent communication skills for collaborating across technical and non-technical teams, providing insights and guidance on quality assurance practices.
- Strong analytical skills, attention to detail, and to improving data quality and testing frameworks.
IPC - Information Processing Capability (Factors of Complexity)
- Ability to work on issues of moderate scope where analysis of situations or data requires a review of relevant factors.
- Exercise judgment within defined procedures and practices to determine appropriate action.
· Apply process improvements to facilitate improved outcomes.
· Implement and execute processes across business/function to achieve assigned goals.
· Seek out and embrace relevant perspectives when assessing a situation or making a decision; demonstrate clear understanding of multiple viewpoints.
· Leverage business insights in proposing solutions and facilitating change.
· Scope of work: Receive assignments in the form of objectives with goals and the process by which to meet the goals.
· Ability to manage multiple priorities and projects simultaneously, ensuring stakeholder expectations are managed appropriately.
· Ability to manage competing demands, accept criticism and constructive feedback, while being extremely adaptable and flexible
· Strong analytical skills; ability to distill information from disparate data sources and the capability to tell the “story” behind it, as well as recommendations for next steps
Benefits:
- The ability to collaborate with, learn from colleagues in a highly complex, global organisation.
- We provide a working environment with a creative company, paired with a great compensation package, great benefits, and a supportive atmosphere where you can sharpen with new challenges and development opportunities.
- Hybrid work-from-home and at a determined Rockwell Automation facility.
- Corporate Social Responsibility opportunities,
- Support from our 24/7 employee assistance program.
- Reporting will to Technical Lead.
Primary work location: Noida/Pune/Bangalore, India.
We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender, gender identity or expression, or veteran status. We are proud to be an equal opportunity workplace.
At Rockwell Automation we are dedicated to building a diverse, inclusive and authentic workplace, so if you're excited about this role but your experience doesn't align perfectly with every qualification in the job description, we encourage you to apply anyway. You may be just the right person for this or other roles.
#LI-PS2
#LI-Hybrid
Rockwell Automation’s hybrid policy aligns that employees are expected to work at a Rockwell location at least Mondays, Tuesdays, and Thursdays unless they have a business obligation out of the office.
