Data Engineer - AI (DV Clearable)
Job description
Sanderson Government & Defence are partnering with a market-leading National Security organisation that delivers mission-critical technology and data solutions across the UK Intelligence, Defence and wider National Security community.
As part of continued growth, they are seeking talented Data Engineers to support the design, development and operation of secure data platforms that enable organisations to make informed, data-driven decisions in highly sensitive environments. Working with cutting-edge technologies including Python, Apache NiFi, Spark, Kafka, AWS and Palantir Foundry, you will play a key role in building scalable data pipelines, integrating complex datasets and delivering secure, high-quality data solutions.
Key Responsibilities
- Design, build and maintain data pipelines and ingestion frameworks.
- Develop and support ETL processes using tools such as NiFi and Python.
- Design database schemas and data models.
- Integrate and enrich data from multiple sources while ensuring data quality and consistency.
- Support and maintain data platforms, including data ingestion and storage components.
- Investigate and resolve data-related operational issues.
- Implement appropriate data security controls.
- Work within Agile/Scrum teams, translating user requirements into epics and user stories.
- Develop clean, secure, test-driven and reusable code.
- Monitor system performance and implement improvements.
Essential Technical Skills
- Python development, particularly for ETL and data engineering.
- Apache NiFi, Spark and/or Kafka.
- Relational databases (PostgreSQL, MySQL).
- NoSQL databases (MongoDB, Cassandra).
- AWS cloud services.
- Experience with Palantir Foundry would be highly desirable.
The strongest candidates are likely to have:
- SC or DV clearance (or eligibility).
- 3+ years' experience in data engineering, data integration or ETL development.
- Strong Python skills.
- Experience building secure cloud-based data solutions.
- Exposure to big data technologies such as Spark, Kafka or NiFi.
- Experience working in defence, government, intelligence or highly regulated environments.
This is an opportunity to join a business at the forefront of National Security innovation, working alongside highly skilled technical teams on programmes that have a genuine impact on protecting the UK's national interests. The organisation offers a collaborative culture, strong career development opportunities and exposure to some of the most challenging and rewarding projects within the Defence and Intelligence sector.