Immigration Technology is one of several delivery portfolios within the Digital, Data, and Technology (DDaT) function of the Home Office. DDaT has around 1,500 staff, augmented by supplier partners.
Every year, DDaT services support more than 3 million visa applications, checks on 100 million border crossings, 5 million passport applications and 140 million police checks on people, vehicles and property.
The Immigration Technology portfolio is responsible for designing, building and delivering significant software systems so members of the public and immigration officials can apply for and process immigration applications of all kinds.
The use of digital technology is helping to realise significant changes in the handling of immigration cases, saving time and money, and improving operational effectiveness.
Adopting the practices of Continuous Delivery
As part of a drive with DDaT to adopt proven nimble approaches to software delivery, the Immigration Technology portfolio has moved towards Continuous Delivery practices for building and releasing software systems.
Continuous Delivery – reliable releases through build, test and deployment automation – is a specific set of practices put forward in the 2010 book Continuous Delivery by Jez Humble and Dave Farley. It has radically increased the effectiveness of software delivery in organisations worldwide, including in UK government departments.
Recent research into software delivery practices by experts Nicole Forsgren, Jez Humble and Gene Kim in their book Accelerate found that organisations using Continuous Delivery practices and related approaches have significantly fewer bugs and operational problems, spend more time developing new business-relevant features and have higher staff satisfaction compared to other organisations.
The core of Continuous Delivery is ensuring that software is deployable at any time, so teams can respond rapidly to requests for change, bug fixes and unforeseen problems such as zero-day security vulnerabilities.
To keep software deployable, a set of practices and disciplines is needed, including small change sizes, rapid feedback on tests and deployments, substantial automated testing and strong team ownership of software.
Concepts that many software technologists now take for granted – such as the deployment pipeline, automated acceptance tests, blue-green deployment techniques and trunk-based development – were all brought to widespread attention in Jez and Dave’s book.
A day of Continuous Delivery with Dave Farley
We’ve made good progress in some areas with Continuous Delivery practices, especially for systems on the periphery or less tightly coupled to our core data systems. In other areas, we have more work to do. Keeping software deployable requires changes to software architecture, testing, deployment and environments, and these changes take time to make safely.
To help accelerate things, we were recently joined for a day by Dave Farley, who shared his experience and ran workshops and surgeries on the theme of modern software delivery practices.
We ran a session in the morning for the extended Senior Leadership Team, covering fundamental (but often misunderstood) aspects of software delivery at scale: team size and inter-team relationships, quality as something that’s built into software rather than ‘tested in’ later, the importance of reducing cycle time and batch size, designing for testability and decoupling, and adopting a culture of continuous improvement and learning.
Following that, Dave took part in our lunchtime tech talks session, The Forge. He talked about and how – by using the principles of small, decoupled systems communicating via asynchronous messages – we can build highly scalable systems that avoid becoming coupled in complex ways. There were many useful points for how we build systems within Immigration Technology and DDaT.
In the afternoon, we ran 2 surgery sessions with people from across Immigration Technology (including teams in remote locations via a video link), with Dave answering questions on all kinds of subjects relating to software engineering and delivery, including branching, testability, change management, software architecture, test-driven development, software operability and deployment pipelines.
These sessions generated lots of healthy discussion and a long list of questions from various people, which we'll use to help guide the learning and continuous improvement roadmap for 2018 and 2019.
Actions we took away from the day
- Focus on reducing cycle time – the time taken from committing a change, to version control, to that change working in production.
- Automate almost every aspect of the build, test, and release process – including feature testing, security testing and operational testing.
- Keep our software releasable at all times – just a single button-click or command should be all that’s needed to release a new version.
- Invest in a high-quality build-test-deployment mechanism – the deployment pipeline – that provides rapid feedback from automated testing.
- Keep team sizes small (around 8 people) and match the size of software components to the cognitive load of the team.
- Work to keep separate systems decoupled using contract tests, stubs, API record/replay and similar techniques.
2 comments
Comment by Howard posted on
We're currently figuring out best practices regarding continuous delivery at DfT too. The tech and tools may be slightly different, but it would be extremely useful if different departments got together to share knowledge with each other.
Comment by David Durant posted on
Hi Howard,
I suggest speaking to the cross-gov Agile Delivery Community (https://www.gov.uk/service-manual/communities/agile-delivery-community) and / or going along to DeliverCon (https://www.eventbrite.com/e/delivercon-tickets-48336746510).
Hope that helps.