ESSENTIAL DUTIES AND RESPONSIBILITIES
· Participate and collaborate with team members to review the use cases for data pipelines.
· Design and develop highly optimized and reliable enterprise level data pipelines to move data to data warehouse in real-time.
· Review and analyze existing business process, identify any gaps, and recommend improvements.
· Design and develop modern real-time solutions with Scala, Spark, Azure Databricks.
· Partner with business users and data scientists to build and deploy machine learning models.
· Improve data governance and operational excellence.
· Develop data expertise and take ownership of data quality for the newly developed pipelines.
· Adhere to company coding standards and best practices.
· Conduct code reviews and create technical solution document.
MINIMUM QUALIFICATIONS
· 4 years of building data pipelines for structured and un-structured data.
· 1+ years of experience of azure data pipeline development.
· 2 + years of experience with Hadoop, Azure Databricks, Stream Analytics, Eventhub, Kafka.
· 2+ years of experience of relational and dimensional data modeling with SQL Server and data warehouse.
· 2+ years of experience with SQL Server, T-SQL, stored procedure development and database optimization.
· 2+ years of experience with MS BI stack (SSIS/SSRS/SSAS, Power BI).
· 1 + years of experience with Azure data platform (Azure data factory, Azure data lake, Azure SQL DB, Azure DW/Synapse).
· 1 + years of experience with Azure ML, Python, Scala
· 2+ years of experience with big data technologies.
· 2 + years of experience with Power BI, Azure Analysis Services
· Experience with working with business stakeholders, requirements & use case analysis.
· Strong communication and collaboration skills with creative problem-solving skills.
PREFERRED QUALIFICATIONS
· Bachelor's degree in Computer Science or equivalent work experience.
· Experience with Agile/Scrum methodology.
· Experience with tax and accounting domain a plus.
· Azure Data Engineer certification a plus.
has context menu