Job description
Information Tech Consultants is seeking a Data Engineer to design, develop, and maintain scalable data pipelines and infrastructure in a UK-based role. The candidate should have experience in ETL/ELT, data warehousing, and cloud platforms to support analytics.
Responsibilities include batch and real-time pipelines, SQL queries, data quality, governance, and collaboration with analysts and BI teams. This role emphasizes data security, monitoring, and documentation, with a focus on performance and
Design, develop, and maintain robust data pipelines for batch and real-time data processing. Build and optimize ETL/ELT workflows to extract, transform, and load data from multiple sources. Develop complex SQL queries and optimize database performance. Work with data warehouses, data lakes, and distributed data processing frameworks. Implement data quality checks, validation, monitoring, and error-handling mechanisms. Collaborate with data analysts, data scientists, BI developers, and business stakeholders. Ensure data security, governance, integrity, and compliance with organizational standards. Troubleshoot pipeline failures and improve data processing performance. Maintain technical documentation for data models, pipelines, and workflows. Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field. 2–5 years of experience in data engineering or a related role. Strong proficiency in SQL and Python. Experience with ETL/ELT tools and data integration frameworks. Knowledge of relational and NoSQL databases. Experience with data warehousing concepts, dimensional modeling, and database optimization. Familiarity with Apache Spark, Apache Airflow, or similar data processing and orchestration tools. Experience with at least one cloud platform: AWS, Microsoft Azure, or Google Cloud Platform. Understanding of data pipeline monitoring, testing, and performance optimization. Strong analytical, problem-solving, and communication skills.