Job Description
As a Senior Data Engineer/Architect, you will play a pivotal role in building and optimizing the data infrastructure. You will architect and develop robust data pipelines and scalable solutions, ensuring our data systems projects can handle the complexity and scale of our growing business. This role is hands-on, requiring deep technical expertise in ETL processes, query optimization, and data migration, as well as strong analytical and problem-solving abilities. You'll collaborate closely with cross-functional teams to build high-performance data layers, streamline data ingestion and classification, and ensure the integrity and accessibility of our data ecosystem.
At Blackstone, we are looking for a visionary leader who can translate complex data challenges into scalable, high-performing solutions.
Key Responsibilities:
- Data Architecture Design: Architect and design large-scale, end-to-end data solutions that meet the needs of business and technical stakeholders, focusing on scalability, security, and performance.
- ETL/ELT Pipeline Development: Lead the design, development, and optimization of efficient ETL/ELT pipelines that extract, transform, and load data from various sources into structured formats, ready for business intelligence and advanced analytics.
- Data Source Analysis: Perform detailed analysis of structured and unstructured data sources, providing strategic insights on how best to ingest, process, and classify data to create a high-quality data layer.
- Data Layer Development: Design and build a robust, scalable data layer that integrates seamlessly with business analytics platforms and supports both batch and real-time data processing.
- Data Ingestion, Cleansing, and Classification: Develop and implement data ingestion strategies for real-time and batch processes, ensuring data is thoroughly cleansed, validated, and classified in alignment with business goals and governance policies.
- Query Optimization: Take ownership of optimizing complex SQL queries and enhancing database performance, with a focus on reducing query execution times and improving resource efficiency across large datasets.
- Data Migration: Lead and manage large-scale data migration efforts from legacy systems to modern cloud-based platforms, ensuring data accuracy, integrity, and minimal downtime.
- Performance Monitoring & Troubleshooting: Proactively monitor the performance of data systems, troubleshoot bottlenecks, and implement solutions that maximize the speed and efficiency of data processing workflows.
- Collaboration: Work closely with data analysts, data scientists, and business teams to understand data requirements, build data models, and ensure the data architecture is aligned with both technical and business objectives.
- Data Governance & Security: Ensure that all data solutions comply with data governance and security standards, implementing best practices around data classification, data quality, and regulatory compliance (e.g., GDPR, HIPAA).
- Mentorship: Provide technical leadership and mentorship to junior data engineers, fostering a culture of innovation, best practices, and continuous improvement.
