Senior Data Engineer

Responsibilities: 

• Design and Build Scalable Data Pipelines: Architect, develop, and maintain efficient, scalable, and secure data pipelines to handle large datasets across multiple data sources, ensuring reliability and performance.
• Cloud Platform Expertise: Utilize AWS and GCP services (e.g., Amazon S3, Redshift, BigQuery, Dataflow, Cloud Storage, Dataproc) to implement and optimize cloud-based data infrastructure.
• Data Integration: Integrate various data sources, both structured and unstructured, from internal systems and third-party providers, to enable cross-functional teams to access actionable insights.
• Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and provide solutions that drive insights for business intelligence, reporting, and analytics.
• Data Quality & Governance: Implement best practices for data quality, data lineage, and governance to ensure the accuracy and compliance of data across pipelines and systems.
• Optimization and Automation: Continuously optimize data workflows and automate processes to improve efficiency and reduce latency in data operations.
• Performance Tuning: Optimize data storage, retrieval, and processing performance on cloud platforms to ensure optimal cost and time efficiency.
• Security & Compliance: Ensure data privacy and security standards are maintained in alignment with company policies and industry regulations.

Requirements: 

• Experience: 5+ years of hands-on experience as a data engineer, with a focus on data engineering in cloud environments (AWS and GCP).
• Cloud Technologies: Deep expertise in using AWS (e.g., S3, Redshift, Lambda, Glue) and GCP (e.g., BigQuery, Dataflow, Cloud Storage) to build and manage data pipelines and infrastructure.
• Data Engineering Skills: Strong knowledge of ETL/ELT processes, data warehousing concepts, and distributed computing frameworks (e.g., Apache Spark, Hadoop, Airflow).
• Programming: Proficiency in Python, SQL, and other relevant programming languages for data engineering.
• Database Knowledge: Experience working with both relational and NoSQL databases (e.g., PostgreSQL, MySQL, DynamoDB, MongoDB).
• Version Control & CI/CD: Familiarity with version control systems (e.g., Git) and CI/CD pipelines for automating deployments and testing.
• Data Processing Frameworks: Experience with data processing and orchestration frameworks such as Apache Airflow, Apache Kafka, or similar technologies.
• Problem Solving: Strong analytical and troubleshooting skills with the ability to resolve complex data and system issues in a timely manner.
• Media Industry Knowledge (Preferred): Familiarity with data needs and challenges within the media industry (e.g., content analytics, user behavior analysis, and media streaming data).

Preferred Qualifications:

• Certifications: AWS Certified Solutions Architect, Google Professional Data Engineer, or similar certifications in cloud technologies.
• Big Data Technologies: Experience with big data tools, Spark, or similar distributed data processing technologies.
• Machine Learning (Optional): Exposure to machine learning platforms or working with data science teams to enable ML models is a plus. 

Unfortunately, due to the high number of responses we receive we are unable to provide feedback to all applicants. If you have not been contacted within 5-7 days, please assume that at this stage your application has been unsuccessful.