Data Engineer

Job Brief
As a Data Engineer , you will play a critical role in designing, building, and maintaining data infrastructure that powers our machine learning initiatives. You will collaborate closely with Machine Learning Engineers, Data Scientists, and other stakeholders to ensure that data flows seamlessly from various sources into our ML models. Your focus will be on AWS-based solutions, ensuring scalability, security, and performance across all data processes.
VentureDive Overview 
Founded in 2012 by veteran technology entrepreneurs from MIT and Stanford, VentureDive is the fastest growing technology company in the region that develops and invests in products and solutions that simplify and improve lives of people world-wide. We aspire to create a technology organization and an entrepreneurial ecosystem in the region that are recognized as second to none in the world.
Key Responsibilities
Design and Implement Data Pipelines:

• Build robust, scalable, and automated data pipelines to ingest, transform, and process large datasets.
• Integrate structured and unstructured data sources using AWS services like S3 , Kinesis , Glue , Athena , Redshift , and Lambda .
• Ensure data quality, consistency, and reliability throughout the pipeline.

Support Machine Learning Workflows:

• Collaborate with Machine Learning Engineers to provide clean, labeled, and preprocessed data for model training and inference.
• Optimize data storage, retrieval, and processing to enhance ML model performance.
• Assist in deploying and monitoring ML models using AWS services such as SageMaker , Step Functions , and Batch.

Optimize Data Infrastructure:

• Monitor, troubleshoot, and optimize data pipelines and ETL processes for speed and efficiency.
• Implement cost-effective solutions by leveraging AWS cost management tools and best practices.
• Continuously improve data architecture to meet evolving business needs and scale with growth.
• Ensure Security and Compliance:
• Implement security best practices across all data workflows, including encryption, access control, and auditing.
• Ensure compliance with relevant regulations (e.g., GDPR, HIPAA) and internal policies.

Collaboration and Documentation:

• Work closely with cross-functional teams, including Data Scientists, DevOps, and Business Analysts, to align data engineering efforts with organizational goals.
• Document data schemas, pipeline designs, and operational procedures for maintainability and knowledge sharing.

Required Skills & Experience:

• Bachelor’s degree in Computer Science, Engineering, or a related field (or equivalent work experience).
• Experience:
• 3+ years of hands-on experience as a Data Engineer or in a similar role.
• Proven track record of working with cloud platforms, particularly AWS .
• Experience supporting machine learning workflows and integrating data pipelines with ML systems.
• Proficiency in AWS services such as S3 , Glue , Athena , Redshift , Lambda , Kinesis , SageMaker and EMR .
• Strong SQL skills for querying and manipulating data.
• Experience with programming languages such as Python , Scala , or Java .
• Familiarity with big data technologies like Apache Spark , Hadoop , or Flink .
• Knowledge of containerization and orchestration tools like Docker and Kubernetes .
• Understanding of data modeling, schema design, and ETL processes.
• Excellent problem-solving and analytical abilities.
• Strong communication and collaboration skills.
• Ability to work in a fast-paced, agile environment.

Preferred Qualifications:

• Experience with Airflow , Prefect , or similar workflow orchestration tools.
• Knowledge of Terraform or CloudFormation for infrastructure-as-code.
• Exposure to real-time data streaming and event-driven architectures.
• Familiarity with MLOps principles and tools.
• Certifications in AWS (e.g., AWS Certified Solutions Architect, AWS Certified Data Analytics Specialty).

What we look for beyond required skills
In order to thrive at VentureDive, you
…are intellectually smart and curious
…have the passion for and take pride in your work
…deeply believe in VentureDive’s mission, vision, and values
…have a no-frills attitude
…are a collaborative team player
…are ethical and honest
Are you ready to put your ideas into products and solutions that will be used by millions?
You will find VentureDive to be a quick pace, high standards, fun and a rewarding place to work at. Not only will your work reach millions of users world-wide, you will also be rewarded with competitive salaries and benefits. If you think you have what it takes to be a VenDian, come join us ... we're having a ball!    
#LI-Remote