MyPage is a personalized page based on your interests.The page is customized to help you to find content that matters you the most.


I'm not curious
4

Spark / Python Developer

Location Tampa, United States
Posted 14-September-2021
Description

This position has the fully remote option. Candidates not required to be onsite

Skills Required:

Minimum 3 years of experience using Apache Spark, KafkaMinimum 3 years of experience with cloud database like SQL, and Cosmos DBMinimum 3 years of experience with streaming and structured streaming dataStrong in Hadoop, HIVE, SQL, Shell scripting, PySpark and PythonExperience with pre-processing data and development of complex business rules in Python or ScalaExperience with design and implementation of ETL/ELT framework for complex warehouses/martsExperience with CSV, JSON, AVRO, Parquet and JDBC/ODBC connectionsExperience with atomic transactions, consistent updates using Delta Lake, Hudi or IcebergKnowledge of large data sets and experience with performance tuning and troubleshootingHands-on development, with a willingness to troubleshoot and solve complex problemsStrong problem- determination skills and resolution experience.Excellent teamwork and communication skills (written and verbal)Bachelor s degree in Computer Science or related analytical field or equivalent experience

Special Requirements:

Hands-on experience in Databricks, preferredUnderstanding of common connections wasbs, adls, abfss, S3, preferredPower BI and Tableau, preferredUnderstanding of Data Modeling concepts, preferredCI/CD exposure, preferred

Work Description:

Development of Databricks (Spark), Kafka workloadsMonitor and troubleshoot Databricks workloadsAssist with architecture, and design for new workloads destined for Databricks and/or utilizing data for storage/events (Kafka, Event Hub)Mentor teams on Spark, Kafka, and complex SQL (developers, testers, data analysts)Assist with Azure DevOps pipelinesHelp teams with data schemas, and data storage detail

 
Awards & Accolades for MyTechLogy
Winner of
REDHERRING
Top 100 Asia
Finalist at SiTF Awards 2014 under the category Best Social & Community Product
Finalist at HR Vendor of the Year 2015 Awards under the category Best Learning Management System
Finalist at HR Vendor of the Year 2015 Awards under the category Best Talent Management Software
Hidden Image Url