MyPage is a personalized page based on your interests.The page is customized to help you to find content that matters you the most.

I'm not curious

Data Engineer (Hadoop Admin)

Location Mumbai, India
Posted 14-September-2021

Planning and designing the Hadoop Cluster
Implement and deploy a new Hadoop Cluster
Deploy new hardware and software environments needed for Hadoop and scale up or down existing environments
Configuring and tuning Hadoop using the various configuration files available within Hadoop
Screening and monitoring of Hadoop Cluster performance, capacity, connectivity and security
Employ security using an authentication and authorization system such as Kerberos
Manage and review Hadoop log files
Install operating system and Hadoop updates, patches, version upgrades when required


You have a M.Sc. or Bachelors in a quantitative field (Information Systems, Computer Science, Physics, Mathematics, etc.) and demonstrated industry experience in data engineering4 to 5 years

Core Technical Requirements:
Expert knowledge of Linux as Hadoop runs on Linux
Knowledge of Troubleshooting Core Java Applications
Good troubleshooting skills, understanding of systems capacity, bottlenecks, basics of memory, CPU, OS, storage, and networks
Expert knowledge on setting up and maintaining Hadoop Clusters on premise and on cloud platforms
Good troubleshooting skills
Expertise in Hadoop ecosystem like zookeeper, HBase, Mahout, Hive, Spark, Pig etc
Expert knowledge of system capacity, memory, CPU, OS, storage, security and networking

Supportive Requirements:
Version Control TechnologiesExperience working with version control (e.g., Git, SVN etc.)
SDLC ModelKnowledge of SDLC models (e.g., Waterfall, Agile, CI/CD)
Containerization - Experience with container-based infrastructures (Docker/Kubernetes)
Experience in the development of API services (REST, e.g. with Flask)
Basic knowledge of common libraries and frameworks (NumPy, pandas, scikit-learn etc.)
Demonstrable knowledge of applying Data Engineering best practices (coding practices, testing (unit, system, acceptance, black box etc.), code review)

Additional Information

Are you a game changer and want to shape the future with us?

An exciting international environment of projects, customers and consultants
Extensive training opportunities in data analytics and data engineering
Competitive salary

Awards & Accolades for MyTechLogy
Winner of
Top 100 Asia
Finalist at SiTF Awards 2014 under the category Best Social & Community Product
Finalist at HR Vendor of the Year 2015 Awards under the category Best Learning Management System
Finalist at HR Vendor of the Year 2015 Awards under the category Best Talent Management Software
Hidden Image Url