Software Engineer - Data Infrastructure

US.svg
United States
Location
Palo Alto United States
Palo Alto United States
Salary
(Yearly)
(Yearly)
(Yearly)
(Yearly)
(Yearly)
Salary information is not provided for this position.
Undisclosed
-
Category
Data Engineer
Date posted
February 11, 2025
Job type
Full-time
Experience level
Mid level

Job Description

We are looking for people with strong Backend Data Engineering capabilities to build highly efficient, resilient systems & pipelines for large-scale data processing. You’ll be part of Luma’s applied research team and work directly on mission critical work-streams utilizing thousands of GPUs.

Responsibilities

  • Design, build and automate infrastructure for processing data across multiple clusters of thousands of GPUs.

  • Work with researchers to identify and implement technical data requirements, and optimize distributed loading for model training.

  • Work cross-functionally for diverse backend engineering needs.

  • Design & build performant infrastructure to manage and leverage large-scale datasets for our model training.

Experience

  • Very strong generalist python coding.

  • Requirement of 5+ years of engineering, including 2+ years of work experience in petabyte-level data processing.

  • Experience engineering large-scale systems that process and serve petabytes of data.

  • Deep understanding of Kubernetes, SLURM, Ray and other cluster orchestration systems.

  • Experience working with visual data.Experience working closely with ML is a strong plus .

Apply now
Luma AI is hiring a Software Engineer - Data Infrastructure. Apply through Homebase and and make the next move in your career!
Apply now
Companies size
101-200
employees
Founded in
2021
Headquaters
San Francisco, CA, United States
Country
United States
Industry
Software Development
Social media
Visit website

Similar AI jobs

Here are other jobs you might want to apply for.

US.svg
United States

Director of Data Engineering

Full-time
Data Engineer
US.svg
United States

Senior Data Engineer (Intent)

Full-time
Data Engineer
MX.svg
Mexico

Freelance AI Agent Assistant

Part-time
Data Engineer
US.svg
United States

Senior Data Engineer

Full-time
Data Engineer
US.svg
United States

Data Engineer

Full-time
Data Engineer
US.svg
United States

Staff Data Engineer

Full-time
Data Engineer