United Kingdom Job Openings
Scrumconnect Consultancy
Data Engineer
FULL TIME
November 18, 2024
- Design, develop, and optimize.*data pipelines* to support data integration and processing workflows.
- Write efficient and scalable queries using.*SQL* to extract, transform, and load data.### Job Title: Data En We are seeking an experienced Data Engineer to join our dynamic team. The ideal candidate will have a strong background in data engineering, with expertise in **SQL**, **Python**, and **Microsoft Azure**. You will be responsible for designing, building, and maintaining robust data pipelines and storage solutions to support our data-driven initiatives. This role requires a minimum of **5 years of experience** in data engineering and **3 years of hands-on experience** with the specified technologies.
Key Responsibilities
- Design, develop, and optimize **data pipelines** to support data integration and processing workflows.
- Write efficient and scalable queries using **SQL** to extract, transform, and load data.
- Develop and maintain data processing scripts and tools using **Python**.
- Leverage **Microsoft Azure** services such as Azure Data Factory, Azure Databricks, Azure SQL Database, and Azure Data Lake to build cloud-based data solutions.
- Ensure data quality and integrity through testing, monitoring, and validation processes.
- Collaborate with stakeholders to gather requirements, translate them into technical solutions, and deliver actionable insights.
- Implement best practices in data security, governance, and compliance in alignment with organizational standards.
- Monitor and troubleshoot data workflows to ensure optimal performance and reliability.
Qualifications and Skills
Essential:
- **5+ years of experience** as a Data Engineer, with a track record of delivering data solutions in complex environments.
- **3+ years of hands-on experience** in:
- Writing complex queries and optimizing performance in **SQL**.
- Developing data processing scripts and solutions in **Python**.
- Designing and implementing data workflows using **Microsoft Azure** services.
- Proficient in building scalable data architectures and handling large datasets.
- Strong problem-solving skills and ability to troubleshoot data pipeline and system issues.
- Experience working with Agile methodologies.
Desirable:
- Knowledge of data visualization tools such as Power BI or Tableau.
- Familiarity with Dev Ops practices, including CI/CD pipelines for data workflows.
- Certifications in **Microsoft Azure** or related technologies.
- Understanding of distributed computing frameworks such as Spark.
New Job Alerts
Kodiform Therapeutics Limited
Scientist (Cell & Molecular Biology)
London
FULL TIME
November 18, 2024
View Job Description
Vagabond Wines
Waiting Staff and Wine Lovers!
London
FULL TIME & PART TIME
November 18, 2024
View Job Description
Looking for similar job?
Culina Group
Business Intelligence & Data Analyst
Rugby
FULL TIME
September 23, 2024
View Job Description
Tarmac
LIS (Land Information System) Database Administrator
Quorn
FULL TIME
September 23, 2024
View Job Description
Astellas Pharma Europe
Manager, Quality Assurance, Technology & Data
Addlestone
FULL TIME
September 23, 2024
View Job Description
Chainlink Labs
Senior Analyst, GTM Data and Analytics
FULL TIME
September 23, 2024
View Job Description
Cummins Inc.
Placement Student-Service Engineering - Digital and Data Analytics
Huddersfield
September 23, 2024
View Job DescriptionSee What’s New: Scrumconnect Consultancy Job Opportunities
Scrumconnect Consultancy
QA Engineer (only SC cleared)
FULL TIME
November 18, 2024
View Job Description
Scrumconnect Consultancy
Java Developer ( SC cleared)
FULL TIME
November 18, 2024
View Job Description
Scrumconnect Consultancy
Data Engineer- R shiny skills mandatory
Coventry
FULL TIME
November 18, 2024
View Job Description
New Job Alerts
Kodiform Therapeutics Limited
Scientist (Cell & Molecular Biology)
London
FULL TIME
November 18, 2024
View Job Description
Vagabond Wines
Waiting Staff and Wine Lovers!
London
FULL TIME & PART TIME
November 18, 2024
View Job Description