United Kingdom Job Openings
Adroit People Ltd.
AWS Data Engineer
Telford
FULL TIME
November 8, 2024
• Design, develop, and maintain real-time data streaming pipelines using AWS Kinesis for processing and analyzing high-velocity data streams.
- Collaborate with stakeholders to gather requirements and design efficient data streaming architectures for real-time analytics, monitoring, and alerting.
- Build and optimize Kinesis Data Streams, Kinesis Data Firehose, and Kinesis Data Analytics to process real-time data from various sources.
- Develop and implement ETL processes to transform, enrich, and aggregate streaming data for downstream consumption in data lakes or data warehouses such as AWS S3, Redshift, and Dynamo DB.
- Integrate Kinesis with other AWS services like Lambda, S3, and EMR for real-time data ingestion, processing, and storage.
- Optimize streaming applications for performance and scalability, ensuring efficient resource usage and minimal latency.
- Develop and maintain robust error handling, data validation, and retry mechanisms within Kinesis pipelines to ensure data consistency and reliability.
- Implement security best practices for data streaming, including encryption in transit and at rest, access control using IAM roles and policies, and data retention policies.
- Monitor and troubleshoot AWS Kinesis streams using Cloud Watch metrics, alarms, and logs, ensuring high availability and minimal downtime.
- Collaborate with Dev Ops teams to automate the deployment and scaling of streaming pipelines using AWS Cloud Formation, Lambda, and CI/CD tools.
- Work closely with data analysts and data scientists to ensure real-time data is processed and delivered for immediate insights and reporting.
- Implement data partitioning, sharding, and key management strategies to handle high-throughput data streams and ensure scalability.
- Optimize the cost of real-time data processing by tuning the configurations of Kinesis and associated AWS services like Lambda and Firehose.
- Stay current with AWS updates and best practices for streaming data, recommending improvements to the architecture as new features and services become available.
- Provide guidance and mentorship to junior engineers in real-time data processing, ensuring best practices in streaming technologies and AWS cloud services.
- Create and maintain detailed technical documentation for data pipelines, including data flow diagrams, stream processing logic, and operational guidelines.
Experience:
- Kinesis Data Streams: 8 years (preferred)
- AWS: 8 years (preferred)
- United Kingdom (preferred)
New Job Alerts
Kodiform Therapeutics Limited
Scientist (Cell & Molecular Biology)
London
FULL TIME
November 18, 2024
View Job Description
Home Office
Authorising Officer
Birmingham
FULL TIME & PART TIME
November 18, 2024
View Job DescriptionFrederic Robinson Brewery
Housekeepers
Ilkley
FULL TIME & PART TIME
November 18, 2024
View Job Description
Atherley's Bakery Farnsfield
Bakery Packer
Farnsfield
FULL TIME & PART TIME
November 18, 2024
View Job DescriptionEl Rachum Care
Cleaner/Carer
Burton upon Trent
PART TIME & CONTRACT
November 18, 2024
View Job Description
LloydsPharmacy Online Doctor
Customer Support Advisor
London
FULL TIME
November 18, 2024
View Job DescriptionLooking for similar job?
Pinnacle Group
Junior Data Analyst - Holborn, England
Holborn
FULL TIME
August 22, 2024
View Job Description
RWS Group
Data Annotator for AI Models | English | Remote, Part Time, Work from Home
PART TIME & CONTRACT
August 1, 2024
View Job Description
FDM Group (Graduate)
Data & Analytics Graduate Programme - Dublin
FULL TIME
August 14, 2024
View Job Description
Alibaba
Alibaba Cloud-Data Center Asset Manager (EMEA)-London, United Kingdom
London
August 6, 2024
View Job Descriptione-Careers Limited
Trainee Data Analyst
Birmingham
FULL TIME & PART TIME
August 25, 2024
View Job DescriptionSee What’s New: Adroit People Ltd. Job Opportunities
Adroit People Ltd.
Software Asset Management Consultant
London
November 18, 2024
View Job Description
Adroit People Ltd.
Mechanical Design Engineer
Cambridge
FULL TIME
October 24, 2024
View Job Description
New Job Alerts
Kodiform Therapeutics Limited
Scientist (Cell & Molecular Biology)
London
FULL TIME
November 18, 2024
View Job Description
Home Office
Authorising Officer
Birmingham
FULL TIME & PART TIME
November 18, 2024
View Job DescriptionFrederic Robinson Brewery
Housekeepers
Ilkley
FULL TIME & PART TIME
November 18, 2024
View Job Description
Atherley's Bakery Farnsfield
Bakery Packer
Farnsfield
FULL TIME & PART TIME
November 18, 2024
View Job DescriptionEl Rachum Care
Cleaner/Carer
Burton upon Trent
PART TIME & CONTRACT
November 18, 2024
View Job Description
LloydsPharmacy Online Doctor
Customer Support Advisor
London
FULL TIME
November 18, 2024
View Job Description