Andy Peterson, Costco
Like so many other organizations, Costco Travel has relied on stabilization or hardening sprints ensure that done does mean Done. They began their agile transformation with a 2-week stabilization sprint after a 3-week sprint. The next step was a release with two 2 week feature sprints followed by a one-week stabilization sprint. These stabilization sprints were where the teams stop focusing on delivering new features or architecture and instead shifts their time and focus on system stabilization, integration testing, performance testing, security testing and preparation of the system for release.
Costco Travels next stop on the road was to completely remove the stabilization sprint. We needed to shorten our delivery cycle by eliminating the stabilization/hardening week. To do this we will deliver value more frequently to our members and business while still keeping the quality bar high. It will also align with the internal cadence of our business by following the fiscal periods. We will also free time by moving to a more proactive planned release cycle instead of the more reactive hotfix patching mode.
QA activities during the stabilization week needed to either move or be removed. Many of the non-functional testings had to be moved and even broken up. Performance and Security runs were broken down and run earlier and more frequently. Regression testing was continually automated and also run more frequently. Costco Travel began to rely on feature flags to control the actual release of a feature and not have all features get rolled out on our release. Training and personal goal activities were moved to a quarterly planning sprint. They also began to deploy a version into pre-production weekly. Costco Travel has had to redefine final version bugs and stories so that they have a true justification that has high enough severity and have formal communication. Finally, the work in progress has been limited so that work can be brought in versus punted easier.
In the four releases that Costco Travel has had since removing the stabilization sprint, we have successfully delivered more frequently to our stakeholders. We’ve taken out 1 week for each release, making 13 where we originally had only 10 each year. In addition to this, we’ve been able to synchronize those releases with our business fiscal periods. We have gotten more stories released per sprint week. Finally, we have been meeting our quality target for deployments. It is now easier to wait for the next major release instead of including it into a hotfix with higher risk. Therefore, we have seen a reduction in average bugs per release.
