Job Description
The Databricks Administrator/Engineer will serve as an important member of LabCorp’s Analytics team focused on supporting Tier 1 Business Intelligence (BI) analytic server administration. This person is part of a team of application administrators and will be responsible for the administration and support of internal-facing Databricks servers. Additionally, the selected candidate will be responsible for supporting Databricks developers/development, application integration and to end-user setup and administration.
Primary Responsibilities- The Databricks Administrator will be responsible for providing technical expertise in deploying applications on AWS/Azure Databricks platform.
- The Databricks Administrator must be self-motivated and apply knowledge of Azure to drive solutions, support the development team, and create documentation to support and describe technical solutions.
- Strong experience with Databricks administration, managing cloud-based big data platforms.
- Hands-on experience with managing permissions, AD group integration, and service principals within Databricks.
- Experience in setting up and managing permissions for Databricks catalogs, schemas, and jobs.
- Familiarity with cloud platforms (AWS, Azure, GCP) and integration with Databricks environments
- Install, configure, and maintain Databricks clusters and workspaces.
- Monitor cluster performance, resource utilization, and troubleshoot issues to ensure optimal performance.
- Implement and manage access controls and security policies to protect sensitive data.
- Perform regular software updates and patches.
- Provide technical support to Databricks users, including troubleshooting and issue resolution.
- Conduct training sessions to educate users on platform best practices and capabilities.
- Optimize cluster configurations for performance and cost-effectiveness.
- Monitor and manage auto-scaling to ensure resources are allocated efficiently.
- Implement and enforce security policies, access controls, and encryption mechanisms.
- Stay up-to-date with security best practices and compliance requirements.
- Develop and maintain backup and disaster recovery strategies to ensure data integrity and availability.
- Collaborate with data engineers to integrate Databricks with other data sources, data warehouses, and data lakes.
- Monitor and manage platform costs, including optimizing resource utilization to control expenses.
- Maintain detailed documentation of configurations, procedures, and best practices.
- Collaborate with cross-functional teams, including data scientists, data engineers, and business analysts,
- To understand their requirements and provide technical solutions
Required Skills/Experience- Bachelor’s Degree in computer science or an analytical discipline or equivalent experience
- Proven experience with Databricks server administration
- 7+ years overall IT experience
- 5+ years Databricks Administration preferably in a high availability architecture
- Must be able to demonstrate ability to lead system and application upgrades
- Proficient with troubleshooting techniques to determine root cause analysis and formulate solution recommendations
- Experience in Memory Management, Handle Scheduling queues & Messages, Performance Management, Supervise Clusters Health check, Regular Back up & recovery, Node Commissioning & Decommissioning, Hive connections, Kafka, Spark, Map Reduce, Tez, User Group Administration, Privilege Checks, Code Deployment to Production, Server Log Analysis and Scheduled Report Monitoring & Back up Monitoring
- Excellent written and verbal communication skills
- Works under minimal supervision with a high degree of self-motivation, commitment and integrity
- Willingness to learn new technologies at an accelerated pace
- Provides clear communication to senior management on issue identification and resolution
- Leverage in-depth business knowledge to translate complex business analysis and reporting requirements into technical designs
- Practical skills in deploying large scale environment processing millions of records per day in a high-availability environment
Labcorp is proud to be an Equal Opportunity Employer:
As an EOE/AA employer, Labcorp strives for diversity and inclusion in the workforce and does not tolerate harassment or discrimination of any kind. We make employment decisions based on the needs of our business and the qualifications of the individual and do not discriminate based upon race, religion, color, national origin, gender (including pregnancy or other medical conditions/needs), family or parental status, marital, civil union or domestic partnership status, sexual orientation, gender identity, gender expression, personal appearance, age, veteran status, disability, genetic information, or any other legally protected characteristic. Additionally, all qualified applicants with arrest or conviction records will be considered for employment in accordance with applicable law.
We encourage all to apply
If you are an individual with a disability who needs assistance using our online tools to search and apply for jobs, or needs an accommodation, please visit our accessibility site or contact us at Labcorp Accessibility.
For more information about how we collect and store your personal data, please see our Privacy Statement.
