Quality Engineer (AI)
Intelligent Operations
Description:
Overview:
The Quality Engineer collaborates with Data Scientists and Machine Learning Engineers to assess and validate machine learning models for production use. They ensure that models meet quality standards not only in terms of functionality and performance, but also with respect to trade-offs such as thresholds, data variability, latency, and real-world feasibility.
What you will do:
- Develop manual and automated test designs (test automation scripts)
- Design and implement manual and automated tests for machine learning and deep learning models across training, validation, and inference pipelines.
- Develop CI/CD-integrated testing workflows to automate model evaluation at every release stage.
- Execute model validation strategies including functional, regression, performance, robustness, and fairness testing.
- Conduct adversarial and edge-case testing to identify brittle behavior in ML models.
- Evaluate large language models using prompt-based test suites and metrics such as coherence, factuality, and hallucination rate
- Employ a variety of testing techniques to successfully deliver product releases including functional, regression, performance, and system tests.
- Work closely with data scientists and machine learning engineers to ensure quality in model deployment and continuous integration pipelines
What we are looking for:
- 3–5 years of experience in QA, with at least 1–2 years working with ML models or AI systems, LLM or conversational agent experience is a plus.
- Experience in Backend Testing and App Testing
- Experience with test automation frameworks for data and model validation
- Working knowledge of Python and familiarity with ML libraries (Scikit-learn, TensorFlow, PyTorch, Langchain)
- Understanding of ML concepts including classification, regression, overfitting, model drift, and evaluation metrics. Also has a baseline understanding of generative model risks (e.g., hallucinations, toxic outputs).
- Experience in testing applications in different domains (e-commerce, banking and finance)
- Working knowledge on at least 1 test automation framework (BDD, KDT, DDT)
- Established foundation in different basic tools for Test Documentation, Bug Logging, and Agile practice
- Basic understanding of fairness, bias detection, and explainability in ML models is a plus