Attention prospective job seekers! Beware of fraudulent offers Read more
Data Engg Python Pyspark - CREQ187455
Description
Experience in pharma domain would be preferred.
At least 5-8 years of experience in PYSPARK, AWS glue) and capable of configuring data pipelines. Possess the following technical skills AWS,SQL, Python, Pyspark, Hive, Unix, ETL, Control-M (or similar) Data Extraction from Teradata, HIVE, Oracle, and Flat files Ability to work independently on specialized assignments within the context of project deliverables Take ownership of providing solutions and tools that iteratively increase engineering efficiencies. Design should help embed standard processes, systems and operational models into the BAU approach for end-to-end execution of Data Pipelines Proven problem solving and analytical abilities including the ability to critically evaluate information gathered from multiple sources, reconcile conflicts, decompose high-level information into details and apply sound business and technical domain knowledge Communicate openly and honestly. Advanced oral, written and visual communication and presentation skills the ability to communicate efficiently at a global level is paramount. Ability to deliver materials of the highest quality to management against tight deadlines. Ability to work effectively under pressure with competing and rapidly changing priorities. Excellent communication skills team player.
Roles and responsibilities of a PySpark developer include:
Developing unit tests for all the Spark transformations and helper methods
Building Spark jobs for data aggregation and transformation
Designing and developing data processing pipelines
Collaborating with team members to determine best practices and client requirements for software.
Developing intuitive software that meets and exceeds the needs of the company.
At least 5-8 years of experience in PYSPARK, AWS glue) and capable of configuring data pipelines. Possess the following technical skills AWS,SQL, Python, Pyspark, Hive, Unix, ETL, Control-M (or similar) Data Extraction from Teradata, HIVE, Oracle, and Flat files Ability to work independently on specialized assignments within the context of project deliverables Take ownership of providing solutions and tools that iteratively increase engineering efficiencies. Design should help embed standard processes, systems and operational models into the BAU approach for end-to-end execution of Data Pipelines Proven problem solving and analytical abilities including the ability to critically evaluate information gathered from multiple sources, reconcile conflicts, decompose high-level information into details and apply sound business and technical domain knowledge Communicate openly and honestly. Advanced oral, written and visual communication and presentation skills the ability to communicate efficiently at a global level is paramount. Ability to deliver materials of the highest quality to management against tight deadlines. Ability to work effectively under pressure with competing and rapidly changing priorities. Excellent communication skills team player.
Roles and responsibilities of a PySpark developer include:
Developing unit tests for all the Spark transformations and helper methods
Building Spark jobs for data aggregation and transformation
Designing and developing data processing pipelines
Collaborating with team members to determine best practices and client requirements for software.
Developing intuitive software that meets and exceeds the needs of the company.
Primary Location
Chennai, Tamil Nadu, India
Job Type
Experienced
Primary Skills
MongoDB, Kafka, AWS Glue, Python
Years of Experience
6
Travel
No
Job Posting
01/05/2024
Syncing your Virtusa Information
Syncing Complete
Exporting your personal Virtusa brand items
Export Complete
Check your downloads folder for files and implementation instructions.
Assets are now available in your profile for future editing and use.
Loading your existing headshots
Please select any one of the images below
Downloading Secure Assets
Signing out