Senior Data Engineer
Job description
- Job grade: Senior Executive Officer
- Contract type: Permanent
- Business area: DEFRA - FBT - Farming Countryside
- Type of role: Architecture and Data, Engineering
- Working pattern: Flexible working, Full-time, Job share, Part-time
- Number of jobs available: 1
Location
About the job
Job summary
The Department for Environment, Food and Rural Affairs (Defra) works on issues that affect daily life across England. Our work covers the environment, farming, food production, rural communities, animal health and protection from natural risks such as flooding.
The Farming and Countryside Programme helps deliver government plans to:
- Support food production
- Restore nature
- Reduce greenhouse gas emissions
- Improve animal health and welfare
The Farming Directorate is responsible for designing and managing Environmental Land Management (ELM) schemes. These schemes support farmers to farm in a more sustainable way.
We are looking for a Senior Data Engineer for the Data Science and Engineering team within the Evidence and Analysis function.
You will work with a small team of data engineers and data analysts, responsible for providing data to the reporting and statistics teams, and wider Defra. Together, this data helps the directorate understand progress, make decisions and meet reporting requirements.
Job description
The Senior Data Engineer will deliver on the data engineering vision for the organisation as set by the Lead Data Engineer. You will focus on driving the design and implementation of numerous data flows to connect operational systems, data for analytics and BI systems.
As Senior Data Engineer, you will identify and deliver new data sources and develop Extract, Transform, Load (ETL) processes. You will also collaborate with the Data Modelling, Statistics and Metrics teams to ensure timely data availability and quality, to deliver data analysis and reporting required by FCP.
You will support the Lead Data Engineer in analysing problems and data analysis issues, looking for underlying causes and assisting in developing solutions. You will also be responsible for delivering data solutions in accordance with agreed organisational standards that ensure services are resilient, scalable and future proof.
We recognise that people with protected characteristics may experience challenges in the job market and in career progression. We want to make sure that everyone has equal opportunities. Defra has many staff networks and trade unions that represent staff and help to positively change how we work. We support flexible working and want to ensure that our staff have a good work-life balance. We provide support for employees with disabilities and health conditions. If you need support at interview, please let us know so that we can provide this. We are keen to make our workforce as diverse as possible, and we hope to attract applications from underrepresented groups.
Person specification
In this role you will:
- Support the management and maintenance of the Outcomes Framework Database (OFDB); a tool that combines several important Defra datasets to support decision-making by other teams.
- Design, build and test data products based on feeds from multiple systems using a range of different storage technologies and/or access methods
- Contribute to the design, development, and maintenance of modern data pipelines and platforms that ingest, process, and publish data from government agencies and external providers
- Contribute to the build and maintenance of cloud-native data infrastructure using platforms, such as Azure
- Carry out ad-hoc analysis requests from policy and senior stakeholders
- Apply the concepts and principles of data modelling to produce, maintain and update relevant data models for specific business needs, including reverse engineering data models from a live system.
- Undertake data profiling and source system analysis and present clear insights to colleagues to support the end use of the data
- Ensure the development of user requirements from full development, test and through to deployment life cycles, while maintaining the schedule of work
- Support the development of methods to monitor and improve data quality in source systems using feedback from the Analytics team
- Undertake changes to existing routines following change control procedures and liaise with team members and other teams to communicate delivery and impact of change
- Work with data privacy and information security staff to assure that security and privacy requirements are identified and addressed in your solutions
- Work closely with the Data Engineering community to promote, challenge, collaborate with, and ensure an agile approach to working is being adopted
- Opportunity to line/task manage less senior Data Engineers
Eligibility criteria
The accepted qualifications are one of the following:
- a first or second class honours degree in a numerate subject computer science or similar field
- a postgraduate qualification, such as an MSc or PhD, in a subject such as data engineering, data science, mathematics, statistics or physics
- a level 6 apprenticeship
- professional experience in data engineering or data science, supported by a record of continuous professional development, at a level comparable to a foundation degree or Higher National Diploma (level five)
Essential Criteria
Successful candidates will need to demonstrate evidence of the following
- Advanced technical skills in Python/SQL and use version control
- You have experience in data roles and have an interest and enthusiasm to learn about the specific datasets we use
- You have experience building data flows to connect operational systems, data for analytics and business intelligence systems
- You can follow data engineering standards and testing for data quality
- You have task management experience and are comfortable delegating to a small team
- You have analysed data and can help teams to apply a range of analysis methods
- Ability to create a supportive environment where people can learn and perform well
- Ability to explain complex findings clearly to non-specialists and senior colleagues
- Understand GDPR requirements and have experience handling data securely
Behaviours
We'll assess you against these behaviours during the selection process:
- Changing and Improving
- Communicating and Influencing
We only ask for evidence of these behaviours on your application form:
- Changing and Improving
Technical skills
We'll assess you against these technical skills during the selection process:
- Data Development Process
- In Tray Exercise
We only ask for evidence of these technical skills on your application form:
- Data Development Process
Benefits
- Learning and development tailored to your role
- An environment with flexible working options
- A culture encouraging inclusion and diversity
- A Civil Service pension with an employer contribution of 28.97%
Things you need to know
Artificial intelligence
Selection process details
After submission of the first stage of your application you will be invited to complete a Civil Service Verbal Test and a Civil Service Numerical Test. If you successfully pass the test, you will be invited to complete the final stage of the application.
Please complete the online tests as soon as possible (within 24-48 hours is recommended), the closing date for the tests is 23:55 on 30th November 2026. If you fail to complete the online test before the deadline your application will be withdrawn. Guidance for the test will be available when you are invited to take the test. The tests are administered online and accessed via the CS Jobs website.
Do not leave completing the tests until the last minute in case you experience any access or technical issues. There may not be technical support available after business hours and after 2pm on the closing date of 30th November 2026.
While the tests should work on most operating systems and modern browsers with a good internet connection, we cannot guarantee that every combination will work. At the beginning, the test will check which browser you are using, and alert you if it is likely to be incompatible.
It is highly recommended that you complete the online tests on a desktop computer, not a mobile phone or tablet.
Please note that online tests associated with this vacancy are not subject to time limits; for further information visit the Reasonable adjustments for online tests - a candidate guide
Online Test Maintence
You will be directed to the Talogy - True Talent – Scheduled Maintenance Window Webpage for online tests during an outage.
Our online test supplier will be performing scheduled maintenance on Sunday 29th November 2026 between 4pm and Midnight. You will be unable to access tests during this period.
If you are in the middle of a test when the maintenance begins, your progress will be paused. You will be able to resume your test from where you left off via your application centre once the downtime has ended. Please note that you will not be able to launch any new tests while the system is offline.
As part of the application process, you will be asked to complete a CV, 750-word personal statement, 250-word behaviour and technical statements. Further details around what this will entail are listed on the application form.
The behaviour statement is Changing & Improving. The technical statement is based on Data Development Process.
If candidates require reasonable adjustments during the application process, or online test process, please allow sufficient time (at least 5 working days prior to the job closing date) to request support, to allow us to support candidates as best as possible. Please note that there is no candidate support from Friday 5pm until Monday 9am.
Sift
Should a large number of applications be received, an initial sift may be conducted using the technical statement. Candidates who pass the initial sift may be progressed to a full sift, or progressed straight to assessment/interview.
Interview
If successful at the application stage, you will be invited to interview where you will be assessed on the below behaviour questions & technical where you will be given an in-tray exercise.
Changing & Improving
Communicating & Influencing
Your interview will either be conducted face to face or by video. You will be notified of the location if you are selected for interview.
Sift and interview dates to be confirmed.
Further Information
Location
As part of the pre-employment process for this post, successful candidate(s) will be able to agree a contractual workplace from those locations listed in this advert. The agreed contractual workplace is then the substantive and permanent place of work for the successful candidate(s).
Where the location is ‘National’ the successful appointee should discuss and agree an appropriate contractual location in line with both Defra’s location policy and site capacity, prior to proceeding with pre-employment processes.
The agreed amount of time spent at a workplace for this post will reflect the requirement for Civil Servants to spend at least 60% of their working time in an organisation workplace with the option to work the remaining time flexibly from home. Working time spent at a workplace may include time spent at other organisational locations including field-based operational locations, together with supplier, customer or partner locations. This is a non-contractual agreement which is consistent with common Civil Service expectations.
Travel costs to non-contractual workplaces will be subject to departmental travel and subsistence policies. Travel costs to contractual workplaces are the responsibility of the employee.
The successful candidate is required to carry out all their duties from a UK location, and cannot do so from an overseas location at any time.
Defra includes the core department, APHA, RPA, Cefas and VMD.
Salary
New entrants to the Civil Service are expected to start on the minimum of the pay band. The internal roles rules apply to existing Civil Servants, i.e. level transfers move on current salary or the pay range minimum, transfers on promotion move to new pay range minimum or receive 10% increase. Either case is determined by whichever is the highest.
Reserve List
A reserve list may be held for a period of 12 months from which further appointments can be made.
Childcare Vouchers
Any move to DEFRA from another employer will mean you can no longer access childcare vouchers. This includes moves between government departments. You may however be eligible for other government schemes, including Tax Free Childcare; for further information visit the Childcare Choices website.
Reasonable Adjustment
If a person with disabilities is put at a substantial disadvantage compared to a non-disabled person, we have a duty to make reasonable changes to our processes.
If you need a change to be made so that you can make your application, you should:
Contact Government Recruitment Service via defrarecruitment.grs@cabinetoffice.gov.uk as soon as possible before the closing date to discuss your needs.
Complete the ‘Assistance required’ section in the ‘Additional requirements’ page of your application form to tell us what changes or help you might need further on in the recruitment process. For instance, you may need wheelchair access at interview, or if you’re deaf, a Language Service Professional.
Accessibility
If you are experiencing accessibility problems with any attachments on this advert, please contact the email address in the 'Contact point for applicants' section.
Visa Sponsorship Statement
Please take note that Defra does not hold a UK Visa & Immigration (UKVI) Skilled Worker License sponsor and are unable to sponsor any individuals for Skilled Worker Sponsorship.
Near Miss
Candidates who are judged to be a near miss at interview may be considered for other positions in Defra which may be at a lower grade, but have a potential skills match.
Merit Lists
Where more than one location is advertised, candidates will be appointed in merit order for each location. You will be asked to state your location preference on your application.
Criminal Record Check
If successful and transferring from another Government Department a criminal record check may be carried out.
In order to process applications without delay, we will be sending a Criminal Record Check to Disclosure and Barring Service on your behalf.
However, we recognise in exceptional circumstances some candidates will want to send their completed forms direct. If you will be doing this, please advise Government Recruitment Service of your intention by emailing Pre-EmploymentChecks.grs@cabinetoffice.gov.uk stating the job reference number in the subject heading.
Internal Fraud Database Check
Applicants who are successful at interview will be, as part of pre-employment screening, subject to a check on the Internal Fraud Database (IFD). This check will provide information about employees who have been dismissed for fraud or dishonesty offences. This check also applies to employees who resign or otherwise leave before being dismissed for fraud or dishonesty had their employment continued. Any applicant’s details held on the IFD will be refused employment.
A candidate is not eligible to apply for a role within the Civil Service if the application is made within a 5 year period following a dismissal for carrying out internal fraud against government.
All of these posts require the successful candidate to hold basic security clearance. Candidates posted to London may be subject to a higher level of clearance because of the security requirements for that location. Job offers to these posts are made on the basis of merit. Security vetting will only take place after the receipt of a job offer.
For further information on National Security Vetting please visit the Demystifying Vetting website.
Please be aware the levels of national security clearance are changing which may impact on the level needed for this role by the time of appointment. All efforts will be made to keep candidates informed of any changes and what that will mean in terms of vetting criteria. For more information please See our vetting charter
Feedback will only be provided if you attend an interview or assessment.
Security
Nationality requirements
This job is broadly open to the following groups:
- UK nationals
- nationals of the Republic of Ireland
- nationals of Commonwealth countries who have the right to work in the UK
- nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS) (opens in a new window)
- nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS)
- individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020
- Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service
Working for the Civil Service
We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window).
Diversity and Inclusion
Apply and further information
Contact point for applicants
Job contact :
- Name : Farming Directorate
- Email : FD_BusinessManagement@defra.gov.uk
Recruitment team
- Email : defrarecruitment.grs@cabinetoffice.gov.uk
Further information
If you are not satisfied with the response you receive from the Department, you can contact the Civil Service Commission: here
Share this page
- Gmail