At Nielsen, we are passionate about our work to power a better media future for all people by providing powerful insights that drive client decisions and deliver extraordinary results. Our talented, global workforce is dedicated to capturing audience engagement with content - wherever and whenever it’s consumed. Together, we are proudly rooted in our deep legacy as we stand at the forefront of the media revolution. When you join Nielsen, you will join a dynamic team committed to excellence, perseverance, and the ambition to make an impact together. We champion you, because when you succeed, we do too. We enable your best to power our future. ABOUT THIS JOB Platform engineers strike a balance between precision and disruption, between reliability and innovation. Nielsen is a tech company backed by over a century of forward momentum to show the world what’s next—and we couldn’t do it without our platform engineers. Nielsen Media is the market leader in content ratings across virtually all entertainment platforms including local and national TV, streaming, digital advertising, and more. In the role of Data Engineer on the Identity Team, you'll be responsible for developing the systems that estimate the audience consuming the content. This effort deals with massive data sets, demanding keen focus on both performance and design.
RESPONSIBILITIES
Use your knowledge of open source, cloud, and data technologies to architect better identity data products for our internal customers.
Perform architectural reviews for critical business products to ensure they are properly built for reliability, scalability, and cost optimization.
Design and implement low-latency large-scale data pipelines cleaning and transforming identity data.
Review and refine the fundamental Data Engineering and SRE practices in the Identity platform and help define a standard for use around Nielsen.
Mentor other platform team members to increase their technical and soft skills
A LITTLE BIT ABOUT YOU
You have a genuine passion for data engineering at scale and absolutely love learning new technologies. You don’t just want to blindly use platforms and libraries; instead you have a desire to tear them apart and figure out how they work inside. You love scaling and optimizing and seeing just how far or how fast your projects can go. You see all of the building blocks in the community as legos and want to use them to build innovative new solutions at a faster pace than ever.
You are also familiar with many of the following technologies and programming languages:
Apache Spark (spark-sql, graphframes)
Trino/Presto
Airflow 2.x
Python
Scala
Jupyter (Notebooks)
Postgres
AWS S3, EMR, IAM, KMS, EKS, RDS
QUALIFICATIONS
5+ years of experience in a MLOps, Data Engineering, DBA or Database Developer role leading critical projects of high complexity.
Capable of proposing/evaluating data system architectures for high-scale projects.
In-depth knowledge and experience implementing core big data concepts and tools spanning source control, CI/CD, Infrastructure as Code, monitoring, alerting, logging.
Deep knowledge of Apache Spark is a must - spark-sql internals specifically is a significant plus.
Proficiency in AWS, with strong knowledge of S3, KMS and IAM.
Experience training and mentoring junior team members.
Experience with Apache Spark graphframes is a plus.
