GenAI QA

Top Required Skills

1. Python Testing Frameworks - pytest with async support, fixture management, parametrized tests, mocking/patching, pytest-asyncio for LangGraph workflows

2. AI Agent Testing - LangSmith integration, automated evals, LLM-as-judge evaluation patterns, prompt validation, LangGraph multi-node workflow testing, semantic similarity scoring

3. API Integration Testing - FastAPI endpoint validation, REST API contract testing, JSON schema validation, authentication flows (OAuth, API keys), Langfuse/observability platform integration, ServiceNow REST API testing

4. Test Data Management - SQL, data migration, JSON test data generation, data normalization strategies, production trace capture/sanitization

5. Workday & ServiceNow Integration Testing - Workday payroll data validation (earnings, deductions, PTO), ServiceNow ticket creation/updates, REST API/RaaS testing, cross-p...