Logo of Huzzle

Python Pyspark

image

Virtusa

3mo ago

  • Job
    Full-time
    Senior Level
  • Data
    Software Engineering
  • Pune

AI generated summary

  • You should have 6+ years in Bigdata/Hadoop, expertise in data ingestion with Hadoop tools, strong Linux/Unix skills, SQL/ETL experience, and familiarity with Agile, DevOps, Python, and R.
  • You will implement and automate data ingestion using Hadoop and Spark, debug and optimize big data pipelines, execute shell scripts, and collaborate in Agile teams using DevOps tools.

Requirements

  • Excellent knowledge of Bigdata and Hadoop frameworks (Cloudera preferable).
  • Hands on experience in Implementing & Automating Data ingestion solutions using Hadoop, Sqoop, Hive, Impala and Spark.
  • Hands on experience of Linux/Unix Shell scripting(mandatory), SQL & ETL.
  • Experience in Debugging, performance tuning, troubleshooting bigdata pipelines.
  • Good to have knowledge of Service now, Jenkins, Git, Bitbucket, JIRA and DevOps tools.
  • Experience in working in Agile Scrum methodology.
  • Knowledge of Python and R can be a plus
  • Relevant Experience :- 6+ years

Responsibilities

  • Excellent knowledge of Bigdata and Hadoop frameworks (Cloudera preferable).
  • Hands on experience in Implementing & Automating Data ingestion solutions using Hadoop, Sqoop, Hive, Impala and Spark.
  • Hands on experience of Linux/Unix Shell scripting(mandatory), SQL & ETL.
  • Experience in Debugging, performance tuning, troubleshooting bigdata pipelines.
  • Good to have knowledge of Service now, Jenkins, Git, Bitbucket, JIRA and DevOps tools.
  • Experience in working in Agile Scrum methodology.
  • Knowledge of Python and R can be a plus.

FAQs

What is the primary focus of the Python Pyspark job role?

The primary focus is on implementing and automating data ingestion solutions using Hadoop, Sqoop, Hive, Impala, and Spark, while also handling Linux/Unix Shell scripting, SQL, and ETL processes.

What qualifications are required for this position?

A minimum of 6+ years of relevant experience in Big Data and Hadoop frameworks, hands-on experience with data ingestion tools, and proficiency in Linux/Unix Shell scripting are required.

Is knowledge of Python and R mandatory for this job?

Knowledge of Python and R is a plus but not mandatory for this position.

What methodologies does this role require experience in?

Experience in Agile Scrum methodology is required for this role.

Are there any additional tools or technologies I should be familiar with?

Good to have knowledge of ServiceNow, Jenkins, Git, Bitbucket, JIRA, and other DevOps tools is beneficial for this position.

Is experience in debugging and performance tuning of big data pipelines required?

Yes, hands-on experience in debugging, performance tuning, and troubleshooting big data pipelines is a necessary requirement.

What work environment does Virtusa promote?

Virtusa promotes teamwork, professional and personal development, and values collaboration and a dynamic environment for nurturing new ideas.

Does Virtusa have a non-discrimination policy?

Yes, Virtusa has a firm non-discrimination policy and evaluates all employment decisions based on qualifications, merit, and business needs, without discrimination based on race, gender, or other protected categories.

Business transformation that lasts starts with Engineering First.

Technology
Industry
10,001+
Employees
1996
Founded Year

Mission & Purpose

Virtusa Corporation provides digital engineering and technology services to Forbes Global 2000 companies worldwide. Our Engineering First approach ensures we can execute all ideas and creatively solve pressing business challenges. With industry expertise and empowered agile teams, we prioritize execution early in the process for impactful results. We combine logic, creativity and curiosity to build, solve, and create. Every day, we help clients engage with new technology paradigms, creatively building solutions that solve their most pressing business challenges and move them to the forefront of their industry.