Software Engineer (m/f/t) - Machine Learning Pipelines

RWE

2mo ago

Applications are closed

Job
Full-time
Junior & Mid Level
Software Engineering
Seattle

Requirements

2+ years of experience building, deploying, and monitoring complex pipelines that operate at an hours-level cadence
Experience in large scale data processing and distributed systems such as Apache Spark
Proficiency in Python, Julia, and/or Mojo
Experience with cloud computing platforms such as AWS, Azure, or Google Cloud Platform
Strong understanding of containerization technologies (Docker, Kubernetes) and orchestration tools
Familiarity with version control systems (e.g., Git), continuous integration/continuous deployment (CI/CD) pipelines, and infrastructure-as-code tools
Excellent problem-solving skills, attention to detail, and a passion for learning and innovation
Strong communication and collaboration skills, with the ability to work effectively in cross-functional teams and contribute to a positive team climate
Advantageous, but not a must:
Experience with stream processing frameworks (e.g., Apache Kafka, Apache Flink) for real-time data ingestion and processing
Knowledge of serverless computing and event-driven architectures for building scalable and cost-effective pipelines
Experience with Google Cloud Platform MLOps tools (e.g., Vertex AI)
Familiarity with monitoring, logging, and debugging tools for distributed systems (e.g., Prometheus, ELK Stack)
Proficiency in database technologies for data storage and retrieval (e.g., PostgreSQL, MongoDB, BigQuery)
Understanding of security best practices for securing data pipelines
Expertise working with large geospatial weather and climate datasets as well as the Pangeo stack

Responsibilities

If you're passionate about building scalable pipelines for state of the art production machine learning models and thrive in a fast-paced, collaborative environment, we'd love to hear from you! Your responsibilities will include:
Design, develop, and operate complex machine learning and multi-faceted data fusion pipelines in a production environment
Collaborate closely with data scientists, machine learning engineers, and scientists to integrate machine learning models into scalable production systems
Develop reusable components, libraries, and frameworks to streamline pipeline development and maintenance
Optimize pipeline performance, reliability, and resource utilization
Continually expand your knowledge of the latest advancements in machine learning technologies, cloud services, and software engineering best practices with an eye towards identifying and integrating exceptional new components into the AI lab’s production repertoire
Contribute to a team culture where diverse viewpoints, backgrounds and expertise are welcomed

FAQs

What kind of experience is required for the Software Engineer position in Machine Learning Pipelines?

The ideal candidate should have at least 2+ years of experience in building, deploying, and monitoring complex pipelines operating at an hours-level cadence. Experience with large scale data processing, distributed systems like Apache Spark, cloud computing platforms, and containerization technologies is also required.

What programming languages should the candidate be proficient in for this role?

The candidate should be proficient in Python, Julia, and/or Mojo for building machine learning pipelines. Familiarity with other languages may be advantageous but not required.

What are the key responsibilities of the Software Engineer working on Machine Learning Pipelines?

The responsibilities include designing, developing, and operating machine learning and data fusion pipelines in a production environment, collaborating with data scientists and engineers, developing reusable components, optimizing pipeline performance, and staying updated on the latest advancements in machine learning technologies.

What skills are valued for this role beyond technical expertise?

In addition to technical skills, we value traits like passion, collaboration, diversity in backgrounds, willingness to learn and innovate, and a determined work ethic. Strong communication and collaboration skills are also important for working effectively in cross-functional teams.

Are there any preferred qualifications for the Software Engineer position in Machine Learning Pipelines?

While not mandatory, experience with stream processing frameworks, serverless computing, Google Cloud Platform MLOps tools, monitoring tools for distributed systems, database technologies, security best practices, and expertise in working with geospatial weather and climate datasets are advantageous.

RWE

Our energy for a sustainable life.

Energy

Industry

10,001+

Employees

Mission & Purpose

RWE is leading the way to a green energy world. With an extensive investment and growth strategy, the company will expand its powerful, green generation capacity to 50 gigawatts internationally by 2030. RWE is investing €50 billion gross for this purpose in this decade. The portfolio is based on offshore and onshore wind, solar, hydrogen, batteries, biomass and gas. RWE Supply & Trading provides tailored energy solutions for large customers. RWE has locations in the attractive markets of Europe, North America and the Asia-Pacific region. The company is responsibly phasing out nuclear energy and coal. Government-mandated phaseout roadmaps have been defined for both of these energy sources. RWE employs around 19,000 people worldwide and has a clear target: to get to net zero by 2040. On its way there, the company has set itself ambitious targets for all activities that cause greenhouse gas emissions. The Science Based Targets initiative has confirmed that these emission reduction targets are in line with the Paris Agreement. Very much in the spirit of the company’s purpose: Our energy for a sustainable life.