Center for Applied Linguistics

QA Engineer

• Oversee development of Speaking test items and ancillary materials for the WIDA suite of grades 1–12 English language proficiency assessments
• Manage staff who write, review, revise, and select Speaking test items
• Evaluate items for task design, linguistic complexity, grade-level appropriateness, correctness, accessibility, scaffolding, fairness, sensitivity, and style-guide adherence
• Provide feedback to test developers and approve content at critical development stages
• Develop test-refreshment plans, project plans, timelines, procedures, and item specifications
• Coordinate with managers, directors, clients, vendors, partner organizations, and external reviewers
• Lead stakeholder reviews and resolve feedback on content, fairness, and grade appropriateness
• Oversee actor casting, recording sessions, and quality monitoring for Speaking test audio
• Write, review, and edit reports, procedures, foundational documents, style guides, and checklists
• Ensure quality-control and test-security procedures are followed
• Coordinate and approve quality checks of field-test QTI packages and ancillary materials
• Oversee qualitative analysis of Speaking field-test responses and report findings
• Conduct pilot testing and cognitive lab interviews
• Collaborate on Speaking scoring materials, rubrics, rater-training content, and scoring guidance
• Support selection of anchor and training responses; review score justifications, appeals, and validity sets
• Maintain test and item design plans, test frameworks, databases, and related materials
• Set team goals, monitor staff performance, mentor employees, and handle hiring, retention, and performance-review responsibilities

Manager, Speaking Test Development

• Oversee development of Speaking test items and ancillary materials for the WIDA suite of large-scale English language proficiency assessments for multilingual learners in grades 1–12
• Manage staff who write, review, revise, and select Speaking test items
• Evaluate items for task design, linguistic complexity, grade-level appropriateness, correctness, accessibility, scaffolding, fairness, sensitivity, and style-guide adherence
• Provide feedback to test developers and approve item content at critical development stages
• Develop test-refreshment plans, timelines, procedures, project plans, and item specifications
• Coordinate with managers, directors, client organizations, vendors, and partner organizations
• Oversee training materials and support for external reviewers
• Lead stakeholder reviews and resolve feedback on test items
• Oversee casting, recording sessions, and quality monitoring for Speaking test audio
• Write, review, and maintain reports, procedures, foundational documents, style guides, checklists, test plans, and framework documents
• Coordinate quality checks of field-test QTI packages and ancillary materials
• Review and update field-test functional rules
• Oversee qualitative analysis of Speaking field-test responses and report findings
• Conduct pilot testing and cognitive lab interviews
• Collaborate on Speaking-test scoring materials, rubrics, rater training, courses, and scoring guidance
• Support anchor and training-response selection; review score justifications, appeals, qualifying, recalibration, and validity sets
• Maintain test maps, forms, practice items, sample items, databases, and design plans
• Set team goals, monitor staff performance, mentor employees, and support hiring, retention, and performance reviews