Job Title:
Job Description
We are seeking an experienced Data Engineer to join our team. The ideal candidate should have a strong background in data engineering
The project in which the person will be assigned is a data engineering effort to support the current Engineering team.
The application , hosted in AWS, includes EC2, Event Bridge, S3, Lambda, Step Functions, Athena, DymanoDB, OpenSearch, CloudWatch, GLUE, Elastic Cache, SSM Commands, Bedrock, SageMaker, Kendra, Amazon Q, Claude from Anthropic Titan Embeddings from in AWS, Python 3, Langchain, and Streamlit.
Must Have Primary Skills:
· 5+ years of experience as a Data Engineer working with data integration and ETL/ELT pipelines
· Strong hands-on experience with AWS Glue for building and maintaining data pipelines on AWS
· Expertise in using AWS data services like S3, Athena, Redshift, etc. alongside Glue
· Handson Experience in Python
· Handson Experience in SQL (MySql \ MSSql Document DB etc).
· Handson Experience in AWS Lambda
· Experience in PySpark or Scala for developing data transformations in Glue
· Knowledge of Glue Data Catalog
· Experience in working with various files formats like csv, json, parquet, Hudi
· Experience in version controlling using GitHub
· Experience in creating secure IAM Policy and Roles
Desirable & Secondary Skills:
· Experience with other AWS data tools like Kinesis, EMR is a plus Strong programming abilities in languages like Java, SQ
· Experience in schema version controlling tools like Liquibase
· Proficient with DataWeave for data transformations and integrations in Mule
· Solid understanding of data modeling, schema design, and data warehousing principles
· Knowledge of CI/CD, infrastructure as code, monitoring for data pipelines· Experience in Terraform
· Knowledge of crawlers, job monitoring, and orchestration· Ability to build reusable connectors, templates and integration assets
· Good understanding of data architectures, distributed systems, and real-time processing
· Problem-solving skills to troubleshoot and optimize data pipelines
· Ability to collaborate across teams and translate requirements to technical designs
Location:
Language Requirements:
Time Type:
If you are a California resident, by submitting your information, you acknowledge that you have read and have access to the Job Applicant Privacy Notice for California Residents