# QA Architect - R01571456 at Brillio

QA Architect - R01571456 at Brillio in Bangalore, Karnataka, India. Posted October 6, 2026.

- Company: Brillio
- Location: Bangalore, Karnataka, India
- Experience: 7+ years
- Posted: 2026-10-06
- Apply: https://jobs.lever.co/brillio-2/e717b34c-e4de-4ffe-bff0-b9df05bc7714/apply
- Source: https://jobmagic.ai/jobs/qa-architect-r01571456-brillio-jtw9m4

## Requirements

- Design and implement automated testing strategies for AI and data science outputs, ensuring accuracy and reliability across models and pipelines
- Develop and maintain robust evaluation and validation frameworks for backend and frontend components, leveraging statistical and machine learning techniques
- Collaborate with data scientists and engineers to define test cases, hypotheses, and statistical metrics for model assessment and improvement
- Integrate advanced statistical tests such as T-Test, Z-Test, and regression analyses into automated QA workflows to validate model performance
- Utilize tools like Great Expectations, Evidently AI, and specific machine learning frameworks (TensorFlow, PyTorch, Sci-Kit Learn, CNTK, Keras, MXNet) to monitor, track, and report on model drift, anomalies, and forecast accuracy
- Optimize testing processes for scalability and efficiency using Python, PySpark, R, and related technologies in large-scale data environments
- Configure and manage testing infrastructure using platforms such as KubeFlow and BentoML to streamline deployment and evaluation cycles
- Troubleshoot and resolve issues in automated testing pipelines, driving continuous improvement and high-quality deliverables
- Advanced proficiency in Python and PySpark for test automation and statistical analysis
- Expertise in statistical testing methods including Hypothesis Testing, T-Test, Z-Test, and Regression (Linear, Logistic)
- Strong experience with machine learning frameworks such as TensorFlow, PyTorch, Sci-Kit Learn, CNTK, Keras, and MXNet
- Hands-on knowledge of Great Expectations and Evidently AI for data validation and monitoring
- Proficiency in SAS and SPSS for statistical computing and analysis
- Deep understanding of probabilistic graph models and classification algorithms including Decision Trees and SVM
- Experience with forecasting techniques including Exponential Smoothing, ARIMA, and ARIMAX

## Description

**QA Architect**

### Job requirements

**Experience Range:** With at least 7 years of quality assurance experience, including substantial hands-on work with data science and machine learning testing frameworks **Key Responsibilities:**
- Design and implement automated testing strategies for AI and data science outputs, ensuring accuracy and reliability across models and pipelines

- Develop and maintain robust evaluation and validation frameworks for backend and frontend components, leveraging statistical and machine learning techniques

- Collaborate with data scientists and engineers to define test cases, hypotheses, and statistical metrics for model assessment and improvement

- Integrate advanced statistical tests such as T-Test, Z-Test, and regression analyses into automated QA workflows to validate model performance

- Utilize tools like Great Expectations, Evidently AI, and specific machine learning frameworks (TensorFlow, PyTorch, Sci-Kit Learn, CNTK, Keras, MXNet) to monitor, track, and report on model drift, anomalies, and forecast accuracy

- Optimize testing processes for scalability and efficiency using Python, PySpark, R, and related technologies in large-scale data environments

- Configure and manage testing infrastructure using platforms such as KubeFlow and BentoML to streamline deployment and evaluation cycles

- Troubleshoot and resolve issues in automated testing pipelines, driving continuous improvement and high-quality deliverables
**Required Skills:**
- Advanced proficiency in Python and PySpark for test automation and statistical analysis

- Expertise in statistical testing methods including Hypothesis Testing, T-Test, Z-Test, and Regression (Linear, Logistic)

- Strong experience with machine learning frameworks such as TensorFlow, PyTorch, Sci-Kit Learn, CNTK, Keras, and MXNet

- Hands-on knowledge of Great Expectations and Evidently AI for data validation and monitoring

- Proficiency in SAS and SPSS for statistical computing and analysis

- Deep understanding of probabilistic graph models and classification algorithms including Decision Trees and SVM

- Experience with forecasting techniques including Exponential Smoothing, ARIMA, and ARIMAX

- Familiarity with distance metrics such as Hamming, Euclidean, and Manhattan Distance

- Advanced skills in R and R Studio for statistical modeling and QA scripting

- Experience configuring testing platforms such as KubeFlow and BentoML
**Preferred Skills:**
- Experience automating evaluation pipelines for AI/ML in production environments

- Expertise in integrating QA processes with CI/CD workflows and cloud-native architectures

- Knowledge of emerging ML testing tools and frameworks beyond industry standards

- Ability to develop custom statistical metrics for model evaluation

- Experience with QA automation for distributed systems at scale
**Desired Qualifications:**
- Bachelor's degree in Computer Science, Data Science, Statistics, Mathematics, or a quantitative discipline

- Certification in Quality Assurance, Data Science, or Machine Learning (e.g., ISTQB Advanced Test Analyst, TensorFlow Developer Certificate)

- Certification in statistical analysis tools or platforms (e.g., SAS Certified Specialist, SPSS Certification)