Senior Artificial Intelligence Evaluation Engineer

Ciklum is looking for a Senior AI Evaluation Engineer to join our team full-time in the US .

We are a custom product engineering company that supports both multinational organizations and scaling startups to solve their most complex business challenges. With a global team of over 4,000 highly skilled developers, consultants, analysts and product owners, we engineer technology that redefines industries and shapes the way people live.

About the role

As a Senior AI Evaluation Engineer, become a part of a cross-functional development team engineering experiences of tomorrow.

Responsibilities

Build the central eval harness and templates every pod uses; hub-and-spoke: central standards, pod-written tests

Create golden datasets with business and clinical reviewers, including safety and medical-accuracy suites

Set and defend pass thresholds; audit pod evals; report eval pass rates and incidents monthly

Stand up production monitoring for drift and regression with the pipeline engineer

Requirements

4+ years in ML/LLM evaluation, QA engineering for AI systems, or applied research engineering

Hands-on with eval frameworks, LLM-as-judge patterns and their failure modes, statistical rigor on small samples

Independent spine

the team shipping a thing does not set its own pass bar

Desirable

Healthcare or safety-critical evaluation experience; red-teaming background

What’s in it for you?

Strong community

Work alongside top professionals in a friendly, open-door environment

Growth focus

Take on large-scale projects with a global impact and expand your expertise

Tailored learning

Boost your skills with internal events (meetups, conferences, workshops), Udemy access, language courses, and company-paid certifications

Endless opportunities

Explore diverse domains through internal mobility, finding the best fit to gain hands-on ex