Big Data Support Engineer

About the role

The Data Operations team at Absa is looking for a Big Data Support Engineer to maintain and enhance applications within the Hadoop ecosystem. You will ensure data pipelines run smoothly, support dataset changes, and contribute to new development initiatives.

Key responsibilities

• Support end‑to‑end data pipelines and troubleshoot production issues.
• Build, enhance, and deploy new data pipelines on Hadoop, Spark and Kafka.
• Identify and implement optimisation opportunities to improve performance and recovery times.
• Test prototypes and hand over stable solutions to the Data Operations team.
• Participate in regular team and user meetings, providing technical input.

Required profile

• Bachelor’s degree in Computer Science, Information Systems or a related field.
• Minimum 2 years of experience in a Big Data environment.
• Strong analytical and problem‑solving abilities.
• Effective collaboration and communication skills.

Required skills

• Programming languages: Java, Scala, Python.
• Big Data technologies: Hadoop, HDFS, MapReduce, Apache Spark, Kafka.
• SQL and database design, development and data modelling.
• Experience with Linux command line and workflow/scheduling tools.