Planning and designing the Hadoop Cluster Implement and deploy a new Hadoop Cluster Deploy new hardware and software environments needed for Hadoop and scale up or down existing environments Configuring and tuning Hadoop using the various configuration files available within Hadoop Screening and monitoring of Hadoop Cluster performance, capacity, connectivity and security Employ security using an authentication and authorization system such as Kerberos Manage and review Hadoop log files Install operating system and Hadoop updates, patches, version upgrades when required
You have a M.Sc. or Bachelors in a quantitative field (Information Systems, Computer Science, Physics, Mathematics, etc.) and demonstrated industry experience in data engineering4 to 5 years
Core Technical Requirements: Expert knowledge of Linux as Hadoop runs on Linux Knowledge of Troubleshooting Core Java Applications Good troubleshooting skills, understanding of systems capacity, bottlenecks, basics of memory, CPU, OS, storage, and networks Expert knowledge on setting up and maintaining Hadoop Clusters on premise and on cloud platforms Good troubleshooting skills Expertise in Hadoop ecosystem like zookeeper, HBase, Mahout, Hive, Spark, Pig etc Expert knowledge of system capacity, memory, CPU, OS, storage, security and networking
Supportive Requirements: Version Control TechnologiesExperience working with version control (e.g., Git, SVN etc.) SDLC ModelKnowledge of SDLC models (e.g., Waterfall, Agile, CI/CD) Containerization - Experience with container-based infrastructures (Docker/Kubernetes) Experience in the development of API services (REST, e.g. with Flask) Basic knowledge of common libraries and frameworks (NumPy, pandas, scikit-learn etc.) Demonstrable knowledge of applying Data Engineering best practices (coding practices, testing (unit, system, acceptance, black box etc.), code review)
Are you a game changer and want to shape the future with us?
An exciting international environment of projects, customers and consultants Extensive training opportunities in data analytics and data engineering Competitive salary