AI Infrastructure Engineer

Role: AI Infrastructure Engineer

Location: Remote Role

Duration: 6-12 Months

Interview: MS Teams

About the Role

We're looking for a hybrid AI Infrastructure Engineer to help stand up and operate a new GPU compute cluster — including hands-on work managing an 18 HGX-enabled worker node cluster with 144 GPUs, built on NVIDIA InfiniBand and Ethernet switching. This role spans the physical/network fabric layer, broader network operations, and the platform/orchestration layer, making it a good fit for someone who wants ownership from the switch port up through the workload scheduler.

What you'll be doing

*Fabric & Networking*

· Help manage and operate the InfiniBand XDR / Quantum-3400 rail-aligned fabric topology supporting the 18-node, 144-GPU HGX cluster

· Administer UFM and NetQ for subnet management, telemetry, and firmware across the fabric

· Support 800G optics, diagnose and resolve link-flap issues, and manage congestion isolation

· Run NCCL / HPL validation and performance regression testing to confirm cluster health and throughput after changes

· Operate Cumulus SN5600 and Arista border switching for the Ethernet side of the fabric

*Network Operations*

· Drive operational improvements in change management and daily operations

· Manage and operate large-scale IP network technologies and infrastructure

· Work with peering and datacenter interconnect technologies: PNI, Transit, Exchange, Passive DWDM, Wave circuits

· Monitor and support network health across on-premises and cloud infrastructure

· Document best practices and build workflow enhancements

*Platform & Workload Enablement*

· Manage NVAIE and GPU Operator lifecycle, including driver/firmware compatibility matrices across the cluster

· Administer Run:ai for quotas, project structure, and tenant onboarding as teams ramp onto the new nodes

· Configure and support Slurm on carved-out worker node pools for scheduled batch workloads

· Ma