IntroductionAs a Senior Software Engineer, you will have a significant impact on the development and optimization of our SQL query engines. You will take ownership of complex projects, lead technical discussions, and drive the continuous improvement of our systems. Your expertise will be crucial in ensuring that our data processing infrastructure remains at the forefront of performance and scalability, supporting the analytical needs of our users.
As a Senior Software Engineer working on the Prestissimo and Presto query engines, you will be responsible for the design, development, and optimization of distributed SQL query engines that form the core of our data lakehouse infrastructure. Your role will involve both hands-on coding and architectural oversight, ensuring high performance, scalability, and reliability of the engines
Your Role and ResponsibilitiesDesign and Development: - Architect and implement new features and enhancements in Prestissimo and Presto engines.
- Write high-quality, scalable, and maintainable code primarily in C++ and Java.
- Collaborate with cross-functional teams to understand requirements and deliver solutions that meet business needs.
Optimization and Performance Tuning:
- Analyze and optimize the performance of distributed SQL queries.
- Utilize profiling tools to identify bottlenecks and implement performance improvements.
- Work on vectorization, SIMD (Single Instruction, Multiple Data), and adaptive runtime optimizations to enhance query performance.
Collaboration and Mentorship:
- Collaborate with team members, including software engineers, data scientists, and product managers, to deliver robust solutions.
- Mentor junior engineers, providing guidance on best practices, code reviews, and technical design.
4. Integration and Testing:
- Integrate Prestissimo with various data sources and platforms.
- Develop and maintain automated tests to ensure code quality and reliability.
- Conduct code reviews to maintain code quality and adherence to best practices.
5. Documentation and Knowledge Sharing:
- Create comprehensive documentation for new features and system changes.
- Participate in knowledge-sharing sessions to promote a culture of continuous learning and improvement.
6. Problem Solving and Debugging:
- Investigate and resolve complex technical issues reported by users or identified through monitoring.
- Develop tools and scripts to automate recurring tasks and improve operational efficiency.
7. Community Engagement:
- Contribute to open-source projects related to Prestissimo, Presto, and Velox.
- Engage with the community through forums, blogs, and conferences to share knowledge and gather feedback.
- Prestissimo: A C++ drop-in replacement for Presto workers, leveraging the Velox library for enhanced performance.
- Presto: An open-source distributed SQL query engine for running interactive analytic queries against data sources of all sizes.
- Velox: A core library that provides optimized data processing primitives to enhance the execution efficiency of query engines.
- Vectorization: A method of processing multiple elements of data in parallel using a single instruction, improving performance.
- SIMD (Single Instruction, Multiple Data): A type of parallel processing in CPUs that allows a single instruction to process multiple data points simultaneously.
- Adaptive Runtime Optimizations: Techniques that adjust the execution of queries in real-time based on the current workload and system conditions.
Required Technical and Professional Expertise
- Bachelor’s degree in computer science or a related field
- Minimum 5 years of experience in software development, focusing on distributed systems or database engines.
- Proficiency in C++ and Java; strong understanding of distributed systems and database internals.
- Proven experience with performance profiling, optimization techniques, and the Velox library.
- Excellent problem-solving, collaboration, and communication skills; experience in leading technical discussions and projects.
Preferred Technical and Professional Expertise
- Experience with other programming languages like Python or Go
- Strong written and verbal communication skills
- Master’s degree in computer science
