As a Data Platform Tech Lead, you will oversee the architecture, design, and implementation of a self-serve data platform that caters to the needs of data professionals across the organization. You will lead a team of engineers, providing guidance and mentorship while ensuring the platform meets scalability, reliability, and performance requirements. Your role will involve collaborating with cross-functional teams to integrate diverse data sources, developing tools and services that streamline data operations, and fostering a culture of best practices in software engineering and data management.
Roles & Responsibilities:
Lead the design and development of the data platform, ensuring it addresses current and future needs.
Architect scalable solutions for data ingestion, integration, storage, transformation, and pipeline orchestration.
Oversee the deployment of data platform components and ensure seamless integration with existing systems.
Mentor and guide a team of data platform engineers, fostering a collaborative and high-performance work environment.
Provide technical direction and support for team members, ensuring adherence to best practices and industry standards.
Conduct regular code reviews and provide constructive feedback to maintain code quality and team productivity.
Work closely with data scientists, data engineers, and other stakeholders to understand their needs and translate them into technical requirements.
Onboard new team members and other data professionals on using the platform, providing necessary training and documentation.
Promote a culture of continuous learning and improvement within the team.
Monitor the performance and health of the data platform, proactively identifying and resolving issues.
Develop and implement data quality and governance initiatives to enhance the reliability and visibility of data.
Ensure the platform adheres to security and compliance standards.
Drive the development of internal tools and services that enhance data capabilities and streamline workflows.
Stay updated with emerging technologies and trends in data engineering and incorporate relevant advancements into the platform.
Advocate for and implement software engineering best practices within the data team.
REQUIRED EDUCATION, KNOWLEDGE, AND SKILLS:
5+ years of experience in software engineering, DevOps, data architecture, or a related field.
Proven experience leading and managing a team of engineers.
Strong expertise in designing and building scalable data platforms.
Proficiency in Python and SQL; experience with Java/Scala is a plus.
Deep understanding of ETL/ELT concepts and data pipeline orchestration.
Strong experience with cloud platforms, with a preference for Google Cloud Platform (GCP).
Expertise in DevOps technologies, including CI/CD, Terraform, Kubernetes, and GitHub.
Experience with data orchestration tools such as Airflow or Dagster.
In-depth knowledge of database internals, including indexing, partitioning, and the differences between row-based and columnar databases.
Experience with real-time data processing and building near real-time data pipelines.
Familiarity with document-based databases (e.g., MongoDB) for analytics.
Strong organizational and structured mindset.
Excellent communication and collaboration skills.
Ability to advocate for best practices and drive technical excellence within the team.