Discovering Agile: a rollercoaster journey

This post's title was debated for a long time. Only because the word "Agile" had to be in there. If you're in tech and say this word out loud, I guarantee you'll get a wide spectrum of reactions. Some good ones, others not so great.

Agile, in the past 20 years, became the new kid on the block that promised to revolutionize how companies deliver better products faster. It promised everyone a way to learn and iterate quickly to meet their customers' ever-changing needs.

Unfortunately, when applied incorrectly it can wreak havoc on your company, teams, and individuals. Don't be surprised if this is where many organizations and teams are because it's difficult to get right on the first try. Many teams are feeling cheated and never want to hear the word "Agile" again.

Just like anything else though, we should take a step back and consider that perhaps the problem is not the framework, but our implementation of it. It's key to note there is no one-size-fits-all process for Agile delivery. It is very dependent on your company, team, where you are today in your delivery journey, the individuals involved, and sometimes just the business you're in.

I'm going to pause here and say I am no Agile guru. I'm not certified in any shape or form. I don't claim to have all the answers (because I don't). What I offer is a story. One where a team went through a rollercoaster trying to land on an Agile delivery process and came out stronger and better. If what worked for us can help you today, then I've achieved my goal. Carry on to learn more.

The Chaos Begins

I worked for a small company that was early in its product, engineering, and delivery journey. They had smart people working hard every day and delivering awesome products. I was hired to start their Quality Engineering department. One of their first moves in order to adapt to a growing business.

Early on, we'd self-manage and release new features with relative ease. Priorities came from the CEO and Product Manager, and the team tagged on additional features or fixes. Individuals played multiple roles to fill gaps where needed. We got the job done.

As the company's success grew, so did our priorities, dependencies, and team size. We realized we needed a more structured approach to delivery. So, the company hired an Agile Coach and Program Manager.

The entire team had a massive hill to climb. Everyone (including me) needed convincing that this new thing called Agile is good for them and the company. That while we would introduce a defined process, we wouldn't be bogged down and our culture will remain fun and innovative.

We set up the usual scrum meetings: standup, backlog refinement, sprint planning, and retrospective. Standups were simple because we were already doing them. Backlog refinement and sprint planning were completely different stories.

We didn't have priorities ahead of time. Our stories were more tasks. We didn't have enough details to provide good estimates. Our frontend and backend teams had different sprint schedules. We didn't know what the goal of each meeting was. We had too many cooks in the kitchen. What transpired were 2-hour backlog refinements and sprint plannings. The team would leave the meetings exhausted, angry, and more confused about what we need to do and why.

In the middle of this, our company was purchased by a large corporation. Almost immediately we began working on a multi-team, cross-country, cross-capability project with tight deadlines.  To make matters more difficult, they had their own processes and their own Agile process and their culture did not exactly match ours.

Stories would carry over every single sprint. I'm not talking 3 or 5 points. More like 15-30 points. To top it off, our priorities would shift mid-sprint. It felt like we were in constant fire-fighting mode. The team was tired and frustrated. Morale was low. Everyone was fed-up and losing faith in Agile or any process for that matter.

What no one realized in the heat of the moment is that throughout all these difficulties we were making incredible progress. We were learning more about Agile and each other. We were experimenting with different approaches and slowly adjusting our processes and roles.

Paving Our Path

It took us almost 2 years to land on a stable Agile delivery process. Keyword "stable", because there is no such thing as a final process. Our environment constantly changes. People change. Customers change. Companies change.

Our guiding principles when creating the initial process were:

• Start simple
• Define roles and responsibilities
• Clear definitions and goals for each ceremony
• Be flexible, iterate as needed – we'd only adjust the process after at least 2 sprints have gone by

Priorities Meeting

First and foremost, the constant fire-fighting and shifting priorities had to stop. We added a priorities planning meeting on the calendar with all the leaders; Software Engineering, Quality Engineering, UX Design, Product, Support, and Program Management.

In this meeting, we had one backlog of the Epics we wanted to prioritize. And it wasn't just product-specific items. Each capability came with its own items to work on. All were added to this one list. We'd spend an hour every week discussing each item and prioritizing the board accordingly. We'd talk about dependencies between teams and capabilities. We aimed to improve the business and ensure we matured our engineering and design practice (remember, we had a young team hungry to learn and experiment).

The result was a prioritized list that every capability agreed to. We promised if priorities changed we wouldn't shift the team's work mid-sprint. No more siloed discussions about priorities. Everyone agreed, everyone was on the same page, and it was all documented in Jira.

Product Owner Role

Once the Epics were prioritized with the leadership group, we wanted to detail them further and ensure stories were ready for the team. We needed someone that can bridge the gap between the business and technical folks. So we added a Product Owner role.

While we did initially try to hire for this role, it was difficult to find someone who had both a business and technical lens. So we turned to the team. Who better than someone who has shown both the interest and skill for it. A Quality Engineer and a Software Engineer stepped up. Each taking turns based on priorities, their workload, and skill-sets.

The PO (engineer really) worked with the Product Manager and the team to break an Epic into stories. When needed, they connected with the UX team to gather designs. They also worked with the tech leads to detail a high-level implementation approach. The formula was the same for each Epic:

• Break into stories
• Provide designs (technical and UX)
• Clarify dependencies
• T-shirt size the effort

All this would happen outside of any official ceremony. The team had complete freedom at this stage to achieve their goal. Spoiler, everyone excelled.

Updated Scrum Process

We had prioritized Epics and detailed stories in each one. This is what our ceremonies looked like now.

PO Review

We added a bi-weekly, 1-hour PO Review meeting to assess the readiness of an Epic and its stories. It was the leadership group plus the POs and technical leads.

Each PO presented the status of their Epics and stories and the group discussed if it was ready for the team to consume in their upcoming scrum ceremonies. This acted as another point of alignment on priorities, with a clear understanding of the work. The t-shirt sizing of effort helped leadership re-adjust priorities as needed.

When an Epic was deemed ready for the team, it became a candidate for the next Backlog Refinement.

Backlog Refinement

Backlog Refinement involved the entire working team. Leaders would sometimes join but were mostly spectators or helped answer questions. This was the working team's meeting, booked for 1 hour every 2 weeks. Actually, this meeting, sprint planning, and retrospective were all for the team.

At this point, the majority of the members had already seen or heard of the work. The Product Manager and Scrum Master walked through each Epic and story. If the team had any questions or concerns, they were discussed and mitigated as much as possible. When everyone was satisfied with the level of detail in each story, they estimated them as a group using the planning poker method.

One important note to consider is how the stories were estimated. We did not use the number of days. We used complexity.

• How complex were the dependencies? Was it internal, external, or both? Who were the teams involved and what processes did we have to consider for each? Do we have technical integrations to consider?
• How complex was the UX design effort?
• How complex was the implementation effort?
• How complex was the testing effort?

It's also important to note that a story was not ready for Sprint Planning if its dependencies were not mitigated yet. Did we need a specific version of an API or package from another team? If it wasn't ready for us to consume, we didn't take the chance. Of course, we were flexible remember, so we'd consider a story if it was time sensitive and we had high confidence the dependency would be delivered on time.

Sprint Planning

By the time a story hit Sprint Planning, every single member of the team had already seen it and had an opportunity to weigh in. Sprint Planning was booked as a 1-hour meeting every 2 weeks as well.

The team started by reviewing their current sprint and assessing how they were tracking. That made sure we did not plan more stories next sprint than the team could handle, in case any stories carried over.

Then they'd go through each prioritized Epic and story and review the estimates. This was their chance to make adjustments or clarify anything. If something unexpected came up, we did not discuss the story for more than 10 minutes. We moved on and considered the story not ready.

Remember how we always had stories carry over? Well, to curb that we looked at the average velocity of the team and aimed for a slightly lower target to start. Stories were added to the next sprint and the team left the meeting knowing exactly what was coming up and why it was important.

As for story assignments, we left it to the team to sort out based on skill-sets and interests. Usually, they'd use the same meeting or spend a little more time after to assign a story per person.

Sprints

Sprints remained on a 2-week cadence. We kept our word to eliminate priority shifts. The guidance to the team was if anyone asked for work outside of their sprint commitment, to not make any promises and to convey they need to speak to the team first. The leaders would then handle the requests and shield the team to focus on their stories.

When anyone finished their work early, they'd see if anyone needed help. If not, we had a prioritized tech-debt backlog to pick from. Or they could pick from the product backlog if they wanted. Or they could even use the time for learning. They could start on the story, but we did not bring it into the sprint.

Seeing the Light

Within 3 sprints of trying the new process, we dramatically reduced story carry-overs. We were no longer battling fires. Communication was clear across the entire team, horizontally and vertically. Everyone was settling into a rhythm. Teamwork was through the roof and we were starting to increase our velocity.

Take a look at your team and your current process and assess if any of the learnings here can help you. Start with the guiding principles and work your way through each ceremony. Feel free to drop or add meetings. You need to find what works for you. It's a difficult challenge to overcome for sure. But there is nothing better than a well-functioning and happy team.