Logo of Huzzle

DevOps Engineer - Generative AI Team

image

Adyen

Aug 23

Applications are closed

  • Job
    Full-time
    Senior Level
  • Software Engineering
    Engineering
  • Madrid

Requirements

  • 5+ years of professional experience as a DevOps Engineer, MLOps Engineer, ML Engineer, Data Engineer
  • Strong software development skills, including: version control (e.g. Git and preferable on Gitlab), coding best practices, debugging, unit and integration testing
  • Proficient in Python, Airflow, MLflow, Docker, Kubernetes and ArgoCD
  • Proficiency with observability tools, such as: Prometheus, Logsearch, Kibana and Grafana
  • Knowledge of data pipelines and ETL processes to prepare and manage data for ML training and inference. As well as model development and deployment frameworks
  • Solid understanding of DevOps best practices and tools to automate software development and deployment processes, and CI/CD concepts and experience in implementing these practices
  • Ability to diagnose and resolve model performance, scalability, and deployment issues
  • Familiarity with monitoring tools to track model performance, resource utilization, and system health. Experience in logging and error monitoring for ML models and applications
  • Knowledge of ETL pipelines using PySpark and Airflow for data preprocessing and model training
  • Clear understanding of the end-to-end machine learning lifecycle
  • Experience with Helm for packaging and deploying applications on Kubernetes and with Kustomize for customizing and managing Kubernetes configurations
  • Familiarity with infrastructure as code with tools like Terraform
  • Experienced with Open-source Machine Learning frameworks like Huggingface Transformers
  • General LLMOps experience is a plus, including model deployment, monitoring, resources, and infrastructure management, including GPU knowledge

Responsibilities

  • Collaborate with the team to design and build the infrastructure to host LLMs in-house while thinking about scale, performance and reliability.
  • Own the deployment strategy of ML models for downstream tasks such as ticket routing (text classification), summarization, sentiment analysis, and question-answer retrieval.
  • Automate the ML pipeline using MLOps tools and practices and optimize it for scalability and performance.
  • Containerize applications and manage the Kubernetes deployments as well as the infrastructure needed to deploy LLMs internally; from GPUs to vector databases and inference components.
  • Develop observability best practices for the whole LLM infrastructure and build the internal framework which allows the team to monitor the LLM behavior to ensure their robustness under real life conditions.
  • Design and implement APIs, services or frameworks to facilitate the seamless integration and usage of LLMs within various applications and services.
  • Stay up to date with the latest advancements in MLOps tools and practices.

FAQs

Do we support remote work?

This role is office-based and we do not offer remote-only roles; we value in-person collaboration at our Madrid office.

What technologies are primarily used in this role?

We mainly use Open Source frameworks like HuggingFace and LangChain, along with models such as Llama or Mixtral. Additionally, proficiency in Python, Docker, Kubernetes, and MLOps tools like Airflow and MLflow is required.

What are the main responsibilities of the DevOps Engineer in the Generative AI team?

Key responsibilities include designing and building infrastructure to host LLMs, owning deployment strategy for ML models, automating ML pipelines, managing Kubernetes deployments, developing observability best practices, and designing APIs for seamless integration of LLMs.

What qualifications are needed for this role?

Applicants should have 5+ years of professional experience as a DevOps Engineer, MLOps Engineer, ML Engineer, or Data Engineer, strong software development skills, proficiency in various tools and languages, and a solid understanding of DevOps best practices.

Is there a commitment to diversity and inclusion at Adyen?

Yes, Adyen emphasizes diversity, equity, and inclusion as crucial components of our workplace culture and encourages applicants from diverse backgrounds to apply.

What is the typical duration of the interview process?

The interview process typically takes about 4 weeks to complete, but this may fluctuate depending on the specific role.

What additional qualifications could be beneficial for this position?

Desirable additional qualifications include knowledge of ETL pipelines with PySpark and Airflow, end-to-end machine learning lifecycle understanding, and experience with Helm, Kustomize, and infrastructure as code tools like Terraform.

How can I contact you if I need flexibility during the application process?

You can let us know if you need more flexibility, and we will do our best to accommodate your needs during the application process.

Meet the financial technology platform helping the world’s leading businesses achieve their ambitions faster.

Finance
Industry
1001-5000
Employees
2006
Founded Year

Mission & Purpose

Adyen is a global payments company that provides businesses with a unified platform to process payments across online, mobile, and in-store channels. Its technology allows companies to accept payments from anywhere in the world and manage transactions securely and efficiently. Adyen's ultimate mission is to help businesses grow by providing them with the tools to simplify payments and improve customer experiences. Their purpose is to drive innovation in the payments industry, making transactions seamless and accessible for both businesses and consumers globally.