Data Engineer with Data Formats
London
Get The Future You Want!
Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you’d like, where you’ll be supported and inspired by a collaborative community of colleagues around the world, and where you’ll be able to reimagine what’s possible.Join us and help the world’s leading organizations unlock the value of technology and build a more sustainable, more inclusive world.
Your Role
We are seeking a highly skilled and experienced Data Engineer to join our dynamic team. The ideal candidate will have a deep understanding of data engineering principles, data technologies, and a proven track record of designing and building complex data pipelines.This role requires strong expertise in SQL, various data formats, Python, and JavaScript to support our data-driven decision-making processes and enhance our data infrastructure.
- Architect and maintain scalable data pipelines using various programming technologies.
- Use SQL to query, transform, and process data across relational and NoSQL databases.
- Integrate data from APIs, flat files, and streaming sources for consistency and quality.
- Implement real-time data processing using Kafka or Solace.
- Manage data storage in systems and warehouses, optimizing for performance.
- Design data models and apply techniques like partitioning and indexing for efficiency.
- Handle multiple data formats (CSV, JSON, Parquet) and manage unstructured data.
- Utilize Python and JavaScript (Node.js) for data processing, automation, and ETL development.
- Leverage Microsoft technologies, Apache Spark, and Airflow for distributed computing.
- Implement DevOps tools (Jenkins, Git, Docker) for CI/CD and monitor pipeline performance.
- Bachelor’s or Master’s degree in Computer Science, Data Science, IT, or related field.
- 8+ years of experience in data engineering, architecture, or related roles.
- Proven track record in building and maintaining large-scale, complex data pipelines.
- Expertise in managing data infrastructure in high-volume environments.
- Strong foundation in designing and optimizing data systems and workflows.
About Capgemini
Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world while creating tangible impact for enterprises and society. It is a responsible and diverse group of 350,000 team members in more than 50 countries.With its strong over 55-year heritage, Capgemini is trusted by its clients to unlock the value of technology to address the entire breadth of their business needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fueled by its market-leading capabilities in AI, cloud, and data, combined with its deep industry expertise and partner ecosystem.
The Group reported 2023 global revenues of €22.5 billion.
Get The Future You Want | www.capgemini.com
London
Lead Data Engineer / Manager (Python SQL AWS) London / WFH to £100k
Are you a Data technologist with leadership skills who enjoys collaborating and working on complex systems with cutting edge technology?
You could be joining a hugely profitable...
NoirLondon
Lead Data Engineer Python, PySpark - Remote
Lead Data Engineer, Team Lead, Technical Lead, Senior Data Engineer, Data Engineer, Python, PySpark, SQL, Big Data, Databricks, R, Machine Learning, AI, Agile, Scrum, TDD, BDD, CI / CD, SOLID principles...
Client ServerLondon
Lead Data Engineer / Manager (Python SQL AWS) London / WFH to £100k
Are you a Data technologist with leadership skills who enjoys collaborating and working on complex systems with cutting edge technology?
You could be joining...