Since my previous blog, An Agile Framework To Measure Remote Team Productivity was written almost a year ago, here is a retrospective view of the agile lessons we can take away after a year of working at home. A number of things have changed since then, mostly as a result of the COVID pandemic and the subsequent changes to our business, that of our clients, and the way we work. It is fair to say that it has been a year of experimentation and discovery, with successes, some “lessons learnt”, and a great deal of hard work.
My previous blog was based on the idea that remote teams are less productive than co-located teams (agile sacred cow #1) and I outlined the framework to measure this. With 90% of our team working remotely, this has proven harder to decisively test than originally anticipated, primarily due to lacking a control group to measure against.
This leaves the evaluation of throughput over time to get an idea of how productive our teams have been.
Throughput As A Proxy For Productivity
I used holiday, public holiday, and illness records to calculate the throughput rate per person day in each team, or as I call it, “Capacity Adjusted Throughput”. These graphs illustrate the improvement in productivity controlling for hiring, and although hard to see shows a 19% increase over the year and arguably a significant increase in the maximum productivity.
Agile Metrics
As a company, we’ve paid close attention to not improving one area at the expense of another area. We’ve tried to “spread the love” across the performance framework, rather than exclusively focus on the classic agile team management metrics of throughput/velocity and cycle and lead times. Below is a summary of the changes we’ve made in 2020 across each of the areas of our agile metrics framework.
1. Quality
- Platform Engineers – We introduced the Platform Engineer role to Trayport, giving us a small team of people responsible for increasing the value gained from the automation of our testing, deployment, and live verification. This not only reduced the overhead associated with managing our automation estate, but more importantly reduced the elapsed time between identifying an issue and delivering a solution. A significant part of this improvement can be attributed to reducing the time taken to “prove” or test and releasing our changes.
- Quality Metrics – When we think about the quality outcomes we want to achieve, we think happy customers. But from a measurement perspective, these measures tend to have significant lag time so our goal is to find predictive quality metrics. Metrics that we can measure within hours have a strong causal relationship with our desired quality outcomes. Some of our more common predictive quality metrics fall into the categories of stability, code quality, performance, security, and usability.
2. Productivity
- Digital Boards – Naturally, with the abrupt switch to working from home we all needed to quickly find ways to collaborate digitally. Our first step was to switch from physical Kanban boards to digital ones. In most teams we adopted TFS or JIRA boards, but in one team we built a tool to allow the team to continue to innovate around their processes without the constraints of an off-the-shelf software package.
- Digital Collaboration – With teams unable to gather around a whiteboard or brainstorm around a problem/situation, collaboration was a key issue to address. We explored digital collaboration tools and now use Mural across many of our departments as a digital whiteboard to share ideas and engage in collaborative, creative, and innovative exercises. Interestingly, we have found people prefer this when proposing ideas anonymously – something not really possible with in-person meetings.
3. Responsiveness
- Discovery Boards – Even pre-COVID significant time and effort went into shaping upcoming work. This requires a deep understanding of the business context, our user’s wants/needs, and our current software features. We created a “discovery” board to enhance transparency and enable those with the necessary skills to contribute, thereby reducing lead time. We experimented with a single team and its proven success has led other teams to adopt discovery boards as well.
- Value Stream Mapping – As with most Kanban teams, we completed STATIK exercises to map each team’s process and workflow as a starting point. We noticed that while we were making improvements to the team’s slice of the value stream, upstream and downstream were not changing significantly. We realised the biggest ‘wait time’ and ‘unfinished work’ waste fell outside the team’s process, so we engaged the teams “up” and “down” stream and conducted value stream mapping exercises. This aided our cross-team collaboration, the gradual reduction in value stream waste, and improved our ability to change strategic direction when needed.
4. Predictability
- Rolling Planning – coming into 2020, we were using quarterly planning to align upcoming work and thereby identify, remove or at least manage dependencies. While plans were up-to-date, accurate, and useful at the start of each quarter, by the 2nd half of the quarter they were less so. To improve the timeliness and value of roadmap planning we switched to a rolling 3-month roadmap, with each team maintaining their own “sprint plan” in between monthly roadmap walkthroughs. The increased transparency and reliability has contributed to building trust internally.
- Probabilistic Forecasting – Like all software teams, we struggled with estimation and the balance between effort to produce accurate and valuable estimates of both effort and delivery dates. One of our favourite agile thinkers, Troy Magennis, introduced us to probabilistic forecasting based on monte carlo simulations, allowing better predictions of work completion times and cost. There’s always room for improvement.
5. Resilience
- Team Happiness – Arguably, the foundation on which all of our other work is based is on happy people. We have put a great deal of time and resources into people, and have sought to build on that. We’ve used simple, entertaining ways to measure team happiness on a regular basis – see for yourself.
- Open Space Tech – In addition to our Hackathons, virtual team lunches, and other social events, we recently experimented with an internal “unconference” based on Open Space Technology. This event had the dual purpose of aiding knowledge sharing as well as facilitating meaningful connections between team members. While it was an unexpected success on the social front, we plan to run another event later this year with a more focused theme.
While we have made many changes over the past year, we see this as the next steps in a continuing journey. In fact, I’d go as far as saying we don’t expect to ever reach an end-state where we stop making improvements.
The year of COVID has been a hard time for many of us, with sometimes drastic changes to our lifestyles, home/work schedules and family lives. Many of us have had to adjust to changes that were outside of our control. Even so, I like to think that we are in a better place now than we were a year ago and that we have found a way for further improvement based on raised awareness, experimentation, and our willingness to learn and listen to feedback.
;