Attention prospective job seekers! Beware of fraudulent offers Read more

× Success! Job has been saved successfully.

Senior Data Engineer

Bangalore, Karnataka, India
Posted on: 29-10-2024
Apply
Save job Click to Save job
Back to results Click to Back to results
Job description
Develop and Maintain ETL Pipelines: Design, develop, and implement scalable ETL workflows using PySpark, Python, and AWS Glue, Databricks.
Data Transformation and Integration: Extract, transform, and load data from various sources to AWS S3 and Redshift.
Performance Optimization: Identify and resolve performance bottlenecks in ETL processes, ensuring optimal performance across large datasets.
Debugging & Reverse Engineering: Should be able to debug PySpark programs/Job and reverse Engineer the code.
Automation and Monitoring: Implement automation scripts using AWS Lambda, Step Functions to schedule and monitor data pipelines.
Data Quality: Ensure data integrity and quality across all stages of the ETL pipeline.
Collaboration: Work closely with data architects, analysts, and stakeholders to understand requirements and provide clear communication throughout the project lifecycle.
Documentation: Create and maintain technical documentation, including data mapping, workflow designs, and ETL processes.
Knowledge of CI/CD pipelines and best practices in deployment automation.
Qualification

Develop and Maintain ETL Pipelines: Design, develop, and implement scalable ETL workflows using PySpark, Python, and AWS Glue, Databricks.
Data Transformation and Integration: Extract, transform, and load data from various sources to AWS S3 and Redshift.
Performance Optimization: Identify and resolve performance bottlenecks in ETL processes, ensuring optimal performance across large datasets.
Debugging & Reverse Engineering: Should be able to debug PySpark programs/Job and reverse Engineer the code.
Automation and Monitoring: Implement automation scripts using AWS Lambda, Step Functions to schedule and monitor data pipelines.
Data Quality: Ensure data integrity and quality across all stages of the ETL pipeline.
Collaboration: Work closely with data architects, analysts, and stakeholders to understand requirements and provide clear communication throughout the project lifecycle.
Documentation: Create and maintain technical documentation, including data mapping, workflow designs, and ETL processes.
Knowledge of CI/CD pipelines and best practices in deployment automation.

Apply
Back to results Click to Back to results

 Key job details

Primary Location
Bangalore, Karnataka, India
Job Type
Experienced
Primary Skills
PySpark
Years of Experience
8
Travel
No
Job Posting
29/10/2024

Join Virtusa

 
Ensure that you open the links to enable the Next button, assuming you have read all the documents.
Okay & NextClick to Next

Please enter a valid email address to begin your application.

Ensure that you open the links to enable the Next button, assuming you have read all the documents.
Okay & NextClick to Next
Submit Click to Submit

Thank you for verifying your email. Please proceed with the steps below to apply.

Ensure that you open the links to enable the Next button, assuming you have read all the documents.
Okay & NextClick to Next

We only accept the following file extensions: .pdf, .docx or .doc
Maximum file size: 1 MB
File name must not include special characters or spaces (e.g. “name_resume.pdf”)

Please attach your CV/Resume, ensure it is in the correct format and smaller than 1MB.
We only accept the following file extensions:

Thank you. You already have an active account with Virtusa's hiring system. Please login to our portal to proceed with your application or apply for more opportunities.

LoginClick to Login

Related jobs

About Virtusa

Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join us, you join a team of 36,000 people globally that cares about your growth — one that seeks to provide you with exciting projects, opportunities and work with state of the art technologies throughout your career with us.

Great minds, great potential: it all comes together at Virtusa. We value collaboration and the team environment of our company, and seek to provide great minds with a dynamic place to nurture new ideas and foster excellence.

Virtusa is an Equal Opportunity Employer. All applicants will receive fair and impartial treatment without regard to race, color, religion, sex, national origin, ancestry, age, legally protected physical or mental disability, protected veteran status, status in the U.S. uniformed services, sexual orientation, gender identity or expression, marital status, genetic information or on any other basis which is protected under applicable federal, state or local law.

Applicants may be required to attend interviews in person or by video conference. In addition, candidates may be required to present their current state or government-issued ID during each interview. All candidates must be authorized to work in the USA.

Learn more

Awards and recognition

Ensure that you open the links to enable the Next button, assuming you have read all the documents.
Okay & NextClick to Next

2023

UK’s Best Workplaces in Tech

2023

UK's Best Workplaces for Wellbeing 2023

2022

Virtusa Earns 2022 Great Place To Work Certification

2017-2022

Candidate Experience (CandE) Awards
View all awards and recognition Click to View all awards and recognition

Find us on Glassdoor.

Ensure that you open the links to enable the Next button, assuming you have read all the documents.
Okay & NextClick to Next

Have any questions?

What is the best way to find and apply for positions at Virtusa?

To join our bright team of professionals, you can apply directly to our website under the Careers tab and search all open jobs. https://www.virtusa.com/careers

Ensure that you open the links to enable the Next button, assuming you have read all the documents.
Okay & NextClick to Next

Can I apply for more than one position at the same time?

Yes, you can. Virtusa gives you the flexibility to apply for multiple open positions that excite you about your future and align to your experience and career goals.

Ensure that you open the links to enable the Next button, assuming you have read all the documents.
Okay & NextClick to Next

Can I apply for a position across multiple geographical locations?

Yes, you can. Virtusa is a global Company, and we serve our clients through our global delivery model.

Ensure that you open the links to enable the Next button, assuming you have read all the documents.
Okay & NextClick to Next

What happens after I’ve submitted my resume?

Our dedicated recruitment team will review your online application and match it to all our open jobs. We update our open jobs on a daily basis and encourage you to check back often.

Ensure that you open the links to enable the Next button, assuming you have read all the documents.
Okay & NextClick to Next

How will I be evaluated for a career opportunity with Virtusa?

Our team of recruiters will review your application, relevant job experience, and skills to appropriately align it to our open jobs. From there, the recruitment team will contact the qualified candidate to start the interview process.

Ensure that you open the links to enable the Next button, assuming you have read all the documents.
Okay & NextClick to Next

Career insights

Want to explore the ways you can engineer your career in technology? Our thought leaders share key career insights for candidates from entry-level job seekers to senior technologists.