Senior Data Engineer
NHS Jobs Manchester
The primary purpose of the role is to be responsible for designing, building, optimising and maintaining the infrastructure that allows data to flow from various sources to storage systems, supports data storage, processing, and analysis, enables high-quality, secure, and scalable data delivery across the organisation.
About the Role Designing, building and maintaining data pipelines. Building and maintaining data warehouses. Data cleansing and transformation. Developing and maintaining ETL processes (ELT = extract, transform, load) to extract, transform, and load data from various sources into data warehouses.
Validating charts and reports created by systems built in-house. Creating validation tools. Developing and maintaining data models, data tools. Monitoring and optimising the performance of data pipelines, data storage and processes, identifying and resolving bottlenecks to ensure efficient data processing and retrieval.
Ability to integrate data from multiple sources, including databases, APIs, websites, and flat files for efficient data processing. Selecting and integrating appropriate tools and technologies Troubleshooting and resolving data-related issues.
Architect scalable data solutions to support advanced analytics, machine learning, and real-time processing needs. Design and implement data governance frameworks, including metadata management, lineage tracking, and cataloging. Ensuring data quality and reliability; ensuring data security and compliance, including GDPR.
Document data engineering standards, design decisions, and infrastructure setup for operational transparency. Collaborating with other teams to ensure data availability. Collaborate with data scientists and analysts to ensure data availability for experimentation, model training, and deployment.
Defining and delivering clear data engineering work packages and deliverables. Supporting the implementation of new infrastructure to improve the quality and efficiency of projects delivery. Identify opportunities for innovation, improved productivity, efficiency, and quality.
About the Role Designing, building and maintaining data pipelines. Building and maintaining data warehouses. Data cleansing and transformation. Developing and maintaining ETL processes (ELT = extract, transform, load) to extract, transform, and load data from various sources into data warehouses.
Validating charts and reports created by systems built in-house. Creating validation tools. Developing and maintaining data models, data tools. Monitoring and optimising the performance of data pipelines, data storage and processes, identifying and resolving bottlenecks to ensure efficient data processing and retrieval.
Ability to integrate data from multiple sources, including databases, APIs, websites, and flat files for efficient data processing. Selecting and integrating appropriate tools and technologies Troubleshooting and resolving data-related issues.
Architect scalable data solutions to support advanced analytics, machine learning, and real-time processing needs. Design and implement data governance frameworks, including metadata management, lineage tracking, and cataloging. Ensuring data quality and reliability; ensuring data security and compliance, including GDPR.
Document data engineering standards, design decisions, and infrastructure setup for operational transparency. Collaborating with other teams to ensure data availability. Collaborate with data scientists and analysts to ensure data availability for experimentation, model training, and deployment.
Defining and delivering clear data engineering work packages and deliverables. Supporting the implementation of new infrastructure to improve the quality and efficiency of projects delivery. Identify opportunities for innovation, improved productivity, efficiency, and quality.
Evaluate and prototype new tools or technologies, recommending enhancements to the existing data stack.
NoirManchester
Data Engineer - Leading Pharmaceutical Company - Manchester
Tech Stack: Data Engineer, Databricks, Python, Power BI, Azure, TSQL, ETL, Agile Methodologies
About the Role: We are seeking a talented and experienced Data Engineer on behalf of our...
Hays Specialist RecruitmentManchester
Prestigious opportunity for a Data Platform Engineer within a pioneering technology team based in Manchester. Following a period of significant growth, investment and a cloud transformation, we are inviting you to join our success story!
As our...
CMSPIManchester
As a Senior Data Engineer, you will be an integral part of a cross-functional feature development squad, dedicated to delivering innovative data solutions on our leading analytics platform. You will utilize your diverse skill set to contribute...