• IT & Telecomms
• Consultant - Other
• Sydney
• Contract or Temp

DWS is seeking multiple, experienced hands-on ETL Data Engineers / Senior ETL Data Engineers for a major multi-year program of work in Sydney. Working in an embedded team with our client, a major BSFI giant, you will be working towards the migration of their petabyte scale on-premises Hadoop data warehouse to a new AWS solution.

This is a career defining opportunity, to work with arguably the biggest name in their field on a massive digital transformation project that will be the anchor to all subsequent work across the enterprise in future.

Type: Max term
Length: up to 2 years
Salary: $170-200K (inc super), depending on experience
Location: CBD Sydney, NSW
Style: Hybrid, approx. 3-4 days in the office
Start: Immediate

Essential Skills and Experience:

• Significant professional experience and expertise with Scala, Spark, and Hive - building Spark/Scala pipelines on Hadoop on-prem and in the cloud
• Significant expertise in performance tuning, and building frameworks at scale
• Abule to quickly understand existing frameworks, then build automations and manage stakeholders for architectural changes
• Recent experience in an enterprise BFSI (banking, financial services, insurance) environment
• Expertise in data migration strategies
• Worked on major enterprise scale data migration projects
• Experience with CI/CD and build tools including SBT and Maven

Desirable Skills and Experience:

• Proficiency in HDFS and YARN
• Worked with AWS: EMR, S3, Glue, Lake Formation, Redshift
• Experience in data governance, security, compliance

What you’ll be doing:

• Design and implement data migration strategies from on-premise Hadoop to AWS
• Develop and optimize Hadoop jobs for data processing and migration
• Ensure data quality and integrity throughout the migration process
• Collaborate with stakeholders to understand and document data requirements
• Perform data mapping and transformation tasks as part of the migration strategy
• Implement and manage data governance and security processes
• Troubleshoot and resolve data migration issues and performance bottlenecks
• Monitor and optimise data workflows for performance and cost efficiency
• Develop and maintain documentation for data processes and architecture
• Coordinate with cross-functional teams to align migration activities with project goals
• Conduct performance tuning and optimisation of data processing tasks
• Design and implement automated data pipelines
• Collaborate with AWS data engineering teams as needed

Applicants must hold Australian Citizenship, Permanent Residency (PR), or other permanent work rights – DWS are unable to sponsor or support visas.

Working at DWS offers you:

• Ongoing professional and career development
• Paid learning and certifications (chosen entirely by you), with completion bonuses
• An active social club that host regular, subsidised events
• Corporate discounts on a range of services and products, including health insurance, gym memberships, Apple products and much more

About Us:
Established in 1991, DWS is a professional Technology Consultancy and part of HCLTech. With offices across Australia, we have over 700 employees in Melbourne, Sydney, Adelaide, Brisbane and Canberra; providing services to a broad range of blue-chip corporate clients, as well as Federal, State and Local Government agencies. We provide a wide range of services including; IT consulting, managed application services, project management, customer driven innovation, digital transformation, data and business analytics, strategic advisory and productivity, and robotic process automation.

Ready for a new challenge? Click APPLY now!
www.dws.com.au

DWS encourages applications from Aboriginal and Torres Strait Islander peoples.

No agencies please – our in-house talent acquisition team have this covered.