Sr. Staff System Test Engineer

Company:Qualcomm India Private LimitedJob Area:Engineering Group, Engineering Group > Systems Test Engineering

General Summary:

Job Description

We are seeking a Senior Staff AI System-Level Test Engineer to lead end-to-end testing of Retrieval-Augmented Generation (RAG) AI systems for Hybrid, Edge-AI Inference solutions. This role will focus on designing, developing, and executing comprehensive test strategies for evaluating the reliability, accuracy, usability and scalability of large-scale AI models integrated with external knowledge retrieval systems.

The ideal candidate needs to have deep expertise in AI testing methodologies, experience with large language models (LLMs), expertise in building test solutions for AI Inference stacks, RAG, search/retrieval architecture, and a strong background in automation frameworks, performance validation, and building E2E automation architecture.

Experience testing large-scale generative AI applications, familiarity with LangChain, LlamaIndex, or other RAG-specific frameworks, and knowledge of adversarial testing techniques for AI robustness are preferred qualifications

Key Responsibilities:

Test Strategy & Planning

• Define end-to-end test strategies for RAG, retrieval, generation, response coherence, and knowledge correctness
• Develop test plans & automation frameworks to validate system performance across real-world scenarios.
• Hands-on experience in benchmarking and optimizing Deep Learning Models on AI Accelerators/GPUs
• Implement E2E solutions to integrate Inference systems with customer software workflows
• Identify and implement metrics to measure retrieval accuracy, LLM response quality

Test Automation

• Build automated pipelines for regression, integration, and adversarial testing of RAG workflows.
• Validate search relevance, document ranking, and context injection into LLMs using rigorous test cases.
• Collaborate with ML engineers and data scientists to debug model failures and identify areas for improvement.
• Conduct scalability and latency tests for retrieval-heavy applications. Analyze failure patterns, drift detection, and robustness against hallucinations and misinformation.

Collaboration

• Work closely with AI research, engineering teams & customer teams to align testing with business requirements.
• Generate test reports, dashboards, and insights to drive model improvements.
• Stay up to date with the latest AI testing frameworks, LLM evaluation benchmarks, and retrieval models.

Required Qualifications:

• 13+ years of experience in AI/ML system testing, software quality engineering, or related fields.
• Bachelor’s or master’s degree in computer science engineering/ data science / AI/ML
• Hands-on experience with test automation frameworks (e.g., PyTest, Robot Framework, JMeter).
• Proficiency in Python, SQL, API testing, vector databases (e.g., FAISS, Weaviate, Pinecone) and retrieval pipelines.
• Experience with ML model validation metrics (e.g., BLEU, ROUGE, MRR, NDCG).
• Expertise in CI/CD pipelines, cloud platforms (AWS/GCP/Azure), and containerization (Docker, Kubernetes).

Why Join Us?

• Work on cutting-edge AI retrieval-augmented generation technologies
• Collaborate with world-class AI researchers and engineers.

If you are passionate about AI system testing and ensuring the reliability of next-generation generative models, apply now!

Minimum Qualifications:

•    Bachelor's degree in Engineering, Information Systems, Computer Science, or related field and 6+ years of Systems Test Engineering or related work experience.
OR
Master's degree in Engineering, Information Systems, Computer Science, or related field and 5+ years of Systems Test Engineering or related work experience.
OR
PhD in Engineering, Information Systems, Computer Science, or related field and 4+ years of Systems Test Engineering or related work experience.

Applicants: Qualcomm is an equal opportunity employer. If you are an individual with a disability and need an accommodation during the application/hiring process, rest assured that Qualcomm is committed to providing an accessible process. You may e-mail disability-accomodations@qualcomm.com or call Qualcomm's toll-free number found here. Upon request, Qualcomm will provide reasonable accommodations to support individuals with disabilities to be able participate in the hiring process. Qualcomm is also committed to making our workplace accessible for individuals with disabilities. (Keep in mind that this email address is used to provide reasonable accommodations for individuals with disabilities. We will not respond here to requests for updates on applications or resume inquiries).

Qualcomm expects its employees to abide by all applicable policies and procedures, including but not limited to security and other requirements regarding protection of Company confidential information and other confidential and/or proprietary information, to the extent those requirements are permissible under applicable law.

To all Staffing and Recruiting Agencies:Our Careers Site is only for individuals seeking a job at Qualcomm. Staffing and recruiting agencies and individuals being represented by an agency are not authorized to use this site or to submit profiles, applications or resumes, and any such submissions will be considered unsolicited. Qualcomm does not accept unsolicited resumes or applications from agencies. Please do not forward resumes to our jobs alias, Qualcomm employees or any other company location. Qualcomm is not responsible for any fees related to unsolicited resumes/applications.

If you would like more information about this role, please contact Qualcomm Careers.