Senior Machine Learning Engineer, AI Evaluation

SHRM is a member-driven catalyst for creating better workplaces where people and businesses thrive together. As the trusted authority on all things work, SHRM is the foremost expert, researcher, advocate, and thought leader on issues and innovations impacting today’s evolving workplaces. With nearly 340,000 members in 180 countries, SHRM touches the lives of more than 362 million workers and their families globally.

Summary

The Senior Machine Learning Engineer, AI Evaluation builds and operates the measurement and engineering infrastructure supporting the organization's Applied AI Research (AAIR) function, a continuous experimental environment designed to evaluate how artificial intelligence models perform real-world HR and workplace-related tasks against established professional standards.

This role is responsible for designing and maintaining the engineering infrastructure used to conduct rigorous, reproducible AI model evaluations and benchmarks. The Senior Machine Learning Engineer develops the systems that run multiple AI models against structured, domain-specific evaluations; builds scoring and evaluation frameworks; maintains reproducibility across model versions; and creates the data infrastructure necessary to analyze and track model performance over time.

Working closely with HR subject matter experts and Applied AI Research colleagues, this position translates expert-defined standards and evaluation criteria into technically rigorous, measurable specifications. Subject matter experts establish the domain-specific ground truth and standards for correctness, while the Senior Machine Learning Engineer owns the technical systems and methodologies used to measure model performance against those standards.

The position serves as a shared technical engineering resource across multiple Applied AI Research workstreams and helps ensure that published findings, benchmarks, and research conclusions are supported by reliable,