# Senior AI QA Automation Engineer at Encora

Senior AI QA Automation Engineer at Encora in Bolivia. Remote. Posted October 12, 2026.

- Company: Encora
- Location: Bolivia
- Workplace: Remote
- Experience: 5+ years
- Posted: 2026-10-12
- Apply: https://job-boards.greenhouse.io/encora10/jobs/5263300007
- Source: https://jobmagic.ai/jobs/senior-ai-qa-automation-engineer-encora-n82k2y

## Requirements

- 5+ years of experience in Software Quality Engineering, Test Automation, or Reliability Engineering.
- Strong Python skills with experience building reusable automated testing frameworks.
- Hands-on experience with Playwright and/or Selenium.
- Strong experience with API, UI, integration, regression, end-to-end, and CI/CD-based test automation.
- Experience with Azure AI Foundry Evaluations, RAG evaluation, and prompt testing.
- Experience defining evaluation metrics, benchmark datasets, regression suites, and measurable quality thresholds.
- Knowledge of AI quality concepts such as hallucination testing, grounding, citation accuracy, prompt injection, and agent behavior validation.
- Experience implementing CI/CD quality gates and production monitoring.
- Ability to translate business requirements, workflows, and AI behaviors into measurable quality criteria.
- Bachelor’s degree in Computer Science, Software Engineering, Information Systems, or a related field.

## Description

 

**Job Title:** Senior AI QA Automation Engineer

**Key Skills:** Python, AI Testing, Test Automation, Playwright, Selenium, API Testing, Azure AI Foundry Evaluations, RAG Evaluation, Prompt Testing, CI/CD, Monitoring

**Experience:** 5+ YOE.

**Location:** Costa Rica, Peru, Colombia, Bolivia.

**Mode:** Remote

We at Coforge are hiring Senior AI QA Automation Engineer (#006fu00000bWsOCAA0-4) with the following skill set.

**Key Responsibilities**

- Build automated evaluation frameworks for AI agents, RAG solutions, prompts, tools, workflows, and model outputs, including benchmark datasets, golden responses, regression suites, evaluation metrics, and release thresholds.

- Test AI-specific behaviors including hallucinations, grounding, citation accuracy, prompt injection, tool selection, and multi-agent coordination.

- Develop agent-powered QA solutions for test-case generation, API and workflow validation, and intelligent UAT, while building UI, API, integration, performance, and end-to-end automation.

- Implement CI/CD quality gates and continuously monitor production AI quality, drift, reliability, and recurring failure patterns.

**Required Skills & Qualifications**

- 5+ years of experience in Software Quality Engineering, Test Automation, or Reliability Engineering.

- Strong Python skills with experience building reusable automated testing frameworks.

- Hands-on experience with Playwright and/or Selenium.

- Strong experience with API, UI, integration, regression, end-to-end, and CI/CD-based test automation.

- Experience with Azure AI Foundry Evaluations, RAG evaluation, and prompt testing.

- Experience defining evaluation metrics, benchmark datasets, regression suites, and measurable quality thresholds.

- Knowledge of AI quality concepts such as hallucination testing, grounding, citation accuracy, prompt injection, and agent behavior validation.

- Experience implementing CI/CD quality gates and production monitoring.

- Ability to translate business requirements, workflows, and AI behaviors into measurable quality criteria.

- Bachelor’s degree in Computer Science, Software Engineering, Information Systems, or a related field.

**Preferred Skills:**

- Experience testing Generative AI, RAG solutions, copilots, or AI agents.

- Experience with LLM evaluation frameworks and advanced prompt testing.

- Experience developing agent-powered test generation, workflow validation, or intelligent UAT assistants.

- Experience with adversarial testing and Responsible AI testing.

- Experience monitoring AI quality, drift, reliability, and failure patterns in production environments.

**Posted On:** 12-10-2026

At Coforge, we hire professionals based solely on their skills and do not discriminate based on age, disability, religion, gender, sexual orientation, socioeconomic status, or nationality.