Job Ref: 13875 Print Job

Senior Big Data Engineer

Adaps has been scouted to source Senior Big Data Engineer for one of our Large Insurance clients at Mel CBD

Data Engineer is responsible for the design, build and run of data pipelines to efficiently and reliably move data (both operational structured and behavioral unstructured, and in both batch and real-time modes) from primary source systems to various reporting and analytical data platforms; the configuration, customisation and integration of data components to deliver and operationally run the data platforms; and, the operationalization of predictive models/ machine learning algorithms in the data platforms.

As a Data Engineer

  • Are an analytic and strategic thinker: You take ownership and think creatively to solve complex problems, situations or opportunities.
  • Are planned and organised: You set clearly defined objectives, plan activities and projects well in advance and take account of possible changing circumstances. You manage time effectively and monitor performance against deadlines and milestones.
  • Have the ability to manage conflicting priorities: You effectively multi-task and work successfully either autonomously or in a collaborative environment.
  • Are comfortable with ambiguity: You are able to push forward with ideas that are a workin-progress.
  • Are a team player: You are able to read situations and adapt accordingly, you have a supportive and participative style.
  • Are action oriented: You are decisive whilst balancing risks, you show initiative and take accountability for your actions and projects.
  • Are innovative and proactively facilitate change: Thinking creatively for ‘out-of-the-box’ solutions comes naturally to you and you thrive on challenges requiring innovative solutions whilst being an ambassador for change.
  • Are dedicated to your craft: You have pride in your work as craft, and are always seeking to improve your craftmanship through practice of your skills, self-learning and exploration.

You have…

  • Proficiency in data management with proven experience developing DW and ETL solutions
  • Exposure to, or demonstrable selflearning of contemporary Big Data Platforms, Lambda Architecture and distributed computing principles * Experience with Hadoop and related technologies, on premise or in the cloud
  • Proficiency in the use of Microsoft Azure’s services (or transferable AWS experience)
  • Experience building data processing pipelines using Kafka/Storm/Spark
  • Expert level SQL skills, with additional experience in Python, and Java or Scala (Bonus) Experience with NoSQL databases, such as Mongo, Elastic, Cassandra or Hbase


  • Develop, maintain, test and evaluate data pipelines and solutions
  • Assess, select and integrate any data tools, components, and frameworks
  • Support the data science and advanced analytics pipelines and teams
  • Support and modernise heritage DW/ETL solutions
  • Follow DevOps practices and make use of extensive automation and infrastructure as code approaches
  • Measure and evaluate platform and pipeline performance and contribute towards capacity planning
  • Support the business in the development and implementation of its Data Strategy, Data Governance program, Data Quality and Big Data Platform approaches and frameworks

You use…

  • Contemporary best practice Big Data principles, frameworks, approaches, technologies and tools
  • Industry standard ETL frameworks, approaches, technologies and tools
  • Agile development methodologies
  • DevOps approaches

You have…

  • A generative mindset. You come from a place of possibility, think on your feet and outside the square, and adapt well to change.
  • You are comfortable with ambiguity and are able to push forward with unproven ideas. You are a positive and supportive individual with a flexible, can-do attitude and the ability to see the bigger picture.

How to apply:
Please apply using the link below or call Ricky on 03 8506 6521 for further details. Applications closes based on the volume of applications received. Only short-listed candidates will be contacted.

Adaps is an equal opportunity employer that actively embraces diversity in its workforce through accurate community representation of gender, culture; thought and work arrangements.

Connect with Adaps: