ML Infrastructure Engineer Jobs

Discover the latest remote and onsite ML Infrastructure Engineer roles across top active AI companies. Updated hourly.

Check out 20 new ML Infrastructure Engineer opportunities posted on The Homebase

Staff Strategic Sourcing Manager (Hardware)

New
Top rated
Together AI
Full-time
Full-time
Posted

Advance inference efficiency end-to-end by designing and prototyping algorithms, architectures, and scheduling strategies for low-latency, high-throughput inference. Implement and maintain changes in high-performance inference engines including kernel backends, speculative decoding, and quantization. Profile and optimize performance across GPU, networking, and memory layers to improve latency, throughput, and cost. Design and operate reinforcement learning (RL) and post-training pipelines to jointly optimize algorithms and systems where most of the cost is inference. Make RL and post-training workloads more efficient with inference-aware training loops such as asynchronous RL rollouts and speculative decoding. Use these pipelines to train, evaluate, and iterate on frontier models on top of the inference stack. Co-design algorithms and infrastructure to tightly couple objectives, rollout collection, and evaluation with efficient inference, identifying bottlenecks across the training engine, inference engine, data pipeline, and user-facing layers. Run ablations and scale-up experiments to understand trade-offs between model quality, latency, throughput, and cost, and feed insights back into model, RL, and system design. Own critical systems at production scale by profiling, debugging, and optimizing inference and post-training services under real production workloads. Drive roadmap items requiring engine modification including changing kernels, memory layouts, scheduling logic, and APIs. Establish metrics, benchmarks, and experimentation frameworks for rigorous validation of improvements. Provide technical leadership by setting technical direction for cross-team efforts, and mentor engineers and researchers on full-stack ML systems work and performance engineering.

$200,000 – $280,000
Undisclosed
YEAR

(USD)

San Francisco
Maybe global
Onsite

Tech Lead, Android Core Product - Nairobi, Kenya

New
Top rated
Speechify
Full-time
Full-time
Posted

Work alongside machine learning researchers, engineers, and product managers to bring AI Voices to customers for various use cases; deploy and operate core ML inference workloads for the AI Voices serving pipeline; introduce new techniques, tools, and architecture to improve performance, latency, throughput, and efficiency of deployed models; build tools to identify bottlenecks and sources of instability and design and implement solutions to address the highest priority issues.

$140,000 – $200,000
Undisclosed
YEAR

(USD)

Nairobi, Kenya
Maybe global
Remote

Tech Lead, Android Core Product - Islamabad, Pakistan

New
Top rated
Speechify
Full-time
Full-time
Posted

Work alongside machine learning researchers, engineers, and product managers to bring AI Voices to customers for diverse use cases. Deploy and operate the core machine learning inference workloads for the AI Voices serving pipeline. Introduce new techniques, tools, and architecture that improve the performance, latency, throughput, and efficiency of deployed models. Build tools to identify bottlenecks and sources of instability and design and implement solutions to address the highest priority issues.

$140,000 – $200,000
Undisclosed
YEAR

(USD)

Islamabad, Pakistan
Maybe global
Remote

Tech Lead, Android Core Product - Karachi, Pakistan

New
Top rated
Speechify
Full-time
Full-time
Posted

Work alongside machine learning researchers, engineers, and product managers to bring AI Voices to customers for diverse use cases. Deploy and operate the core ML inference workloads for AI Voices serving pipeline. Introduce new techniques, tools, and architecture to improve the performance, latency, throughput, and efficiency of deployed models. Build tools to identify bottlenecks and sources of instability, then design and implement solutions to address the highest priority issues.

$140,000 – $200,000
Undisclosed
YEAR

(USD)

Karachi, Pakistan
Maybe global
Remote

Tech Lead, Android Core Product - Leipzig, Germany

New
Top rated
Speechify
Full-time
Full-time
Posted

Work alongside machine learning researchers, engineers, and product managers to bring AI Voices to customers for various use cases. Deploy and operate the core machine learning inference workloads for the AI Voices serving pipeline. Introduce new techniques, tools, and architecture to improve the performance, latency, throughput, and efficiency of deployed models. Build tools to monitor bottlenecks and instability sources, then design and implement solutions to address the highest priority issues.

$140,000 – $200,000
Undisclosed
YEAR

(USD)

Leipzig, Germany
Maybe global
Remote

Staff Machine Learning Engineer

New
Top rated
Adaptive Security
Full-time
Full-time
Posted

Define Adaptive's ML strategy including where ML should be applied across products, required infrastructure, and build vs. buy decisions. Design and build production ML systems end-to-end including data pipelines, model training, evaluation frameworks, and inference serving. Establish evaluation methodology to measure model quality, catch regressions, and make data-driven decisions about model changes. Own the strategy for acquiring and formatting necessary data, including labeling, feedback loops, and model improvement over time. Partner with product engineers to integrate ML into the product by writing production code and working within existing codebase. Help build and lead the ML team as scope grows.

Undisclosed

()

New York, United States
Maybe global
Onsite

Machine Learning Engineer, Distributed Data Systems

New
Top rated
OpenAI
Full-time
Full-time
Posted

Design, build, and maintain data infrastructure systems such as distributed compute, data orchestration, distributed storage, streaming infrastructure, and machine learning infrastructure while ensuring scalability, reliability, and security. Ensure the data platform can scale by orders of magnitude while remaining reliable and efficient. Partner with researchers to deeply understand requirements and translate them into production-ready systems. Harden, optimize, and maintain critical data infrastructure systems that power multimodal training and evaluation.

$295,000 – $445,000
Undisclosed
YEAR

(USD)

San Francisco, United States
Maybe global
Hybrid

AI/ML 2026 Internship

New
Top rated
Brain Co
Full-time
Full-time
Posted

As an AI/ML Engineer Intern at Brain Co., you will assist in designing and deploying large language model (LLM)-powered applications to automate complex, real-world workflows. You will build and improve data pipelines and support model training, evaluation, and optimization. Your work involves handling structured and unstructured data, such as text, documents, and logs. You will also help prepare models and systems for production deployment and monitoring. Collaboration with senior engineers, AI researchers, and product teams is expected, along with learning best practices through code reviews, design discussions, and hands-on mentorship. Additionally, you will gain exposure to customer-facing and real-world constraints, including working with public-sector institutions.

Undisclosed

()

San Francisco, United States
Maybe global
Remote

Principal Machine Learning Engineer

New
Top rated
PhysicsX
Full-time
Full-time
Posted

The role involves building a platform used by Data Scientists and Simulation Engineers to build, train, and deploy Deep Physics Models. The candidate will work on a focused, stream-aligned, and cross-functional team that includes back-end, front-end, and design members, empowered to make its own implementation decisions towards meeting its objectives. Responsibilities include gathering and leveraging domain knowledge and experience from the Data Scientists and Simulation Engineers using the product, taking ownership of work from implementation to production, ensuring quality, scalability, and observability at every step, which includes testing, containerization, continuous integration and delivery, authentication, authorization, telemetry, observability, and monitoring.

Undisclosed

()

Singapore
Maybe global
Hybrid

Software Engineer, macOS Core Product - Minneapolis-St. Paul, USA

New
Top rated
Speechify
Full-time
Full-time
Posted

Work alongside machine learning researchers, engineers, and product managers to bring AI Voices to customers for a diverse range of use cases; deploy and operate the core ML inference workloads for the AI Voices serving pipeline; introduce new techniques, tools, and architecture that improve the performance, latency, throughput, and efficiency of deployed models; build tools to provide visibility into bottlenecks and sources of instability and design and implement solutions to address the highest priority issues.

$140,000 – $200,000
Undisclosed
YEAR

(USD)

Minneapolis or St. Paul, United States
Maybe global
Remote

Want to see more ML Infrastructure Engineer jobs?

View all jobs

Access all 4,256 remote & onsite AI jobs.

Join our private AI community to unlock full job access, and connect with founders, hiring managers, and top AI professionals.
(Yes, it’s still free—your best contributions are the price of admission.)

Frequently Asked Questions

Have questions about roles, locations, or requirements for ML Infrastructure Engineer jobs?

Question text goes here

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.

[{"question":"What does a ML Infrastructure Engineer do?","answer":"ML Infrastructure Engineers design, build, and maintain systems that support machine learning workflows from development to production. They create scalable platforms for model training and serving, implement distributed training systems, and develop monitoring solutions to track model performance. These engineers also build data pipelines, optimize ML systems for performance, and implement automated testing and deployment processes while collaborating with data scientists and researchers to productionize ML models."},{"question":"What skills are required for ML Infrastructure Engineer?","answer":"ML Infrastructure Engineers need strong programming skills in Python and sometimes Go, Rust, or C++. Proficiency with ML frameworks like PyTorch and TensorFlow is essential, alongside expertise in cloud platforms (AWS, GCP), containers (Docker), and orchestration (Kubernetes). They should understand distributed systems, data engineering concepts, and model serving techniques. Experience with infrastructure-as-code tools and monitoring systems rounds out the technical requirements, complemented by problem-solving abilities and collaboration skills."},{"question":"What qualifications are needed for ML Infrastructure Engineer role?","answer":"Most ML Infrastructure Engineer positions require a Bachelor's or Master's degree in Computer Science or related field, plus 4-5+ years of experience building production ML systems. Employers typically expect demonstrable experience with cloud platforms, containerization tools, and ML frameworks. Strong understanding of system-level software, machine learning concepts, and resource utilization is necessary. Experience with distributed systems and high-throughput workloads is highly valued, especially for senior positions."},{"question":"What is the salary range for ML Infrastructure Engineer job?","answer":"The research provided doesn't specify salary ranges for ML Infrastructure Engineer jobs. Compensation typically varies based on factors like location, company size, experience level, and specific technical expertise. Organizations like Anthropic, Scale AI, Apple, and other technology companies actively hiring for these positions likely offer competitive compensation packages reflecting the specialized nature of ML infrastructure skills and the current market demand."},{"question":"How long does it take to get hired as a ML Infrastructure Engineer?","answer":"The hiring timeline for ML Infrastructure Engineer positions isn't specified in the provided research. The process typically includes technical interviews focused on systems design, ML fundamentals, and programming skills. Given the specialized nature of the role, companies often conduct thorough evaluations of candidates' experience with production ML systems, distributed computing, and relevant technologies. The specialized requirements may extend the hiring process compared to more general engineering roles."},{"question":"Are ML Infrastructure Engineer job in demand?","answer":"Yes, ML Infrastructure Engineer jobs show strong demand based on active openings at major companies like DataXight, Scale AI, Anthropic, Apple, and Character.AI. The field is growing particularly in specialized areas such as LLM serving infrastructure, on-device ML optimization, and safety-critical ML systems. These positions are distributed across major tech hubs with opportunities ranging from mid-level to senior roles, reflecting industry's increasing need for engineers who can build reliable ML systems at scale."}]