Descripción:

RDT Engineers is an innovative company whose field of activity is the development of advanced engineering projects. We are looking for a Data Engineer with at least 3 years of experience in Azure solutions.
The main function is to provide insights of turbine's technical performance across the business (through KPIs reporting) for the right decision-making. The KPIs analysis is based on Reliability, Availability and Maintainability metrics covering worldwide turbines under SGRE maintenance contract.The analysis requires extracting data from multiple sources (Azure SQL Database, Azure Data Explorer, Azure Data Lake Storage…), transforming and processing enormous volumes of data, using big data technologies like Azure Databricks and Apache Spark.
It involves full data management skills and proven Experience in Azure Solution, requiring the ability to:

• Design, implement, and maintain data pipelines for data ingestion, processing, and transformation in Azure from various sources (Azure SQL Database, Azure Data Lake Storage, Azure Data Explorer).
• Utilizing Azure Data Factory or comparable technologies, create and maintain ETL (Extract, Transform, Load) operations.
• Handling and analyzing enormous volumes of data, designing and implementing cloud and hybrid data solutions in big data technologies like Azure Databricks and Apache Spark.

Requisitos:

It requires technical capabilities with:

• ADVANCE knowledge in Phyton and Spark Programming.Experience in developing Spark applications using Spark-SQL in Databricks for data extraction and transformation.
• Data ingestion from Azure services (Azure Data Lake, Azure Data Explorer, Azure SQL…) and processing data in Azure Databricks.

Hands on in Spark Architecture with Databricks: Manage clusters in Databricks, Databrick Administration and ETL process.
It requires soft skills in:

• Highly developed attention to detail.
• Analytical mind with a problem-solving aptitude.
• High Communication skills in verbal and written English.