The Essence of Agile

January 21, 2013


I love to talk about Agile software development, Lean principles, and am currently trying to get a better grip on Rightshifting. However, when people ask me for advice it can be hard to recall details and specifics, especially if it’s either an area you haven’t thought about for a while, or something that you maybe only have limited perspectives or experience of. When this happens, it’s invaluable to be able to reach for some guiding principles.

The Agile Manifesto is a wonderful resource for this and I hear it’s even being refined (continuous improvement is a wonderful thing). Generally, when asked about them, I’ll end up mis-quoting them…even though I understand what they are about, why they were written down, how they are generally interpreted, etc. Who would have though that those 4 statements would be so difficult to remember? So I wanted to capture an even simpler interpretation of them. I was also keen to include as much from Lean as possible too. It struck me that the strongest, most often recurring theme is “feedback”…and the more I look at it, the more I’m convinced that Feedback is the very essence of Agile. I should probably back that up with evidence of some kind, or at the very least, my opinions and observations.

Feedback

If we examine the Agile Manifesto, we can see that it is all about favoring rich, prompt feedback over other options.

  • Individuals and interactions over processes and tools

This is all about interacting with people, and in general, people are pretty good at giving prompt, quality feedback. Sure, we could all be a little better at it, but you generally get much more feedback talking to someone than using a tool or following a process to accomplish the same goal. For example, handing over a piece of work in a team distributed by time-zone. You could follow the process and update the story with the latest information, and you could use a tool, JIRA perhaps, to capture that information and make it available remotely to the rest of the team. However, having a conversation and brief guided tour of the feature in it’s current state imparts information much more effectively, and provides opportunity for questions and clarification.

  • Working software over comprehensive documentation

The documentation for a project is rarely a good guide to whether the software works or not…even when it gets completed. The problem is that documentation generally lags behind the software product and is often quite far down the priority list when identifying new work. Ultimately it provides poor quality feedback, due to it generally being out of date, and it doesn’t provide it in a useful time frame.

On the other hand, working software is a very rich feedback environment with every click and gesture providing more feedback instantaneously. Continuous integration, yet another feedback mechanism, should allow you to get access to such rich feedback each day (or even more frequently).

  • Customer collaboration over contract negotiation

Our customers write stories in the form of “As a…, I want to…, So that…”, and that’s a good thing. However, it’s just a starting point for a much more detailed conversation about the details. It provides opportunity for change too. Instead of asking for every feature they can think of up-front, they are involved in the evolving product, which should allow them to select and prioritise work in an intelligent manor.

Contract negotiation is a much slower process and much more guarded in the language used. I always see it as both sides trying to make sure they are not going to be punished for failure rather than a collaborative process to deliver something. So it’s not particularly prompt, and the language used tends to disguise the true meaning…so not very high quality either.

  • Responding to change over following a plan

Prompt feedback in essential as the longer you go without feedback, the more opportunity you have to go down the wrong path…or inversely, the more often you get feedback the more opportunity you have to perform a course correction based on the latest information at hand.

Following a plan when you know it to be inaccurate or incorrect (or even a complete fantasy) leads to a culture of no longer questioning, just doing.The act of planning is invaluable, it forces you to consider options and think about how things will work out. However, the plan itself diminishes in value rather quickly as the assumptions and estimates it was built around meet up with reality. Sometimes it’s value drops to zero (or in fact below zero and becomes a burden) even before you start following it.

Test Driven Development

High quality, prompt feedback turns out to be the key concept to all sorts of other Agile software development practices. My favorite example would of course be Test Driven Development (TDD). This provides feedback on a number of levels:

  • Seeing the test results go from failing (red bar of shame) to passing (green bar of joy) is the most obvious form of feedback provided by TDD and should start a round of refactoring now you are in a “good” state.
  • By writing the test first, you get feedback on your understanding of the task. It is very difficult to devise a test when you are unclear on the desired behavior.
  • By writing the test first, you get feedback on the state of the existing code-base. If the test is difficult to write, that feedback is telling you that you should first look at refactoring/redesigning the surrounding code.
  • By running all the tests frequently, you are getting feedback on the state of the entire project and how your current piece of work is integrating with it.

The huge advantage of TDD is that you get this feedback really quickly, it’s incredibly targeted  and it’s very specific. That is an almost perfect definition high quality, prompt feedback.

Pair Programming

I’ve heard code reviews talked about as a poor man’s substitute for pair programming. While I agree with the sentiment and acknowledge that both provide feedback on code, I think the gap between them in practice is much, much bigger than most people realise.

The obvious difference is that code reviews happen after the code has been written…sometimes quite a long time after the code has been written. So code reviews do not provide prompt feedback. This is bad…actually, I’m not sure that “bad” the right word. Maybe I should use “disappointing“? Yes, this is disappointing because the effort is being spent to review the code but the opportunity for getting appropriate returns on that investment have passed by already. In my experience, code reviews generally provide a lower quality of feedback than pair programming too. This is partly because they are conducted some time after the code was written, but also because it’s much harder to really understand a piece of code when you were not there at it’s creation.

So there you go, it’s all about the feedback. Make it high-quality, and seek/provide it promptly. I’ll grant you, it’s not exactly an epiphany, but it was an observation I thought I should share. Feel free to share or suggest your own observations on feedback in the comments section…or even to disagree with me.


Oh dear, it’s November now and I’ve not written the promised second part of my Lean Agile Scotland review. So no more excuses or procrastinating…

People

People Patterns by Joe O’Brien

I’d heard that Joe’s People Patterns talk was really good. I can now confirm that “really good” doesn’t do it justice. I could relate to so many of the issues he talked about and cannot agree more that projects don’t succeed or fail for technical reasons, they do so because of people. If you get the chance to see his talk, go see it. Failing that, listen/watch to it at Lean Agile Scotland 2012 videos.

Slightly off-topic, I had the pleasure of meeting Joe at a lunch with some friends a year or so ago. I was having a bad day and most likely came across like an angry dick, and I really regret that. Completely ignoring some of the points from People Patterns about talking to people, I put off going to speak to him to apologise for my behavior until it was too late and by then he’d already left.

Respect For People by Liz Keogh

It turns out that I can be very disrespectful and I didn’t even notice (see lunch with @objo above for details). Liz examined the roots of various words common in the Lean & Agile vocabulary and related it to the sorts of interactions we’re all familiar with in software development.

I tried the suggestion of not asking people to go next at our stand-up (it’s dis-respectful because I am demanding an update from them, rather than them providing it on their own). This led to a long awkward silence after my turn. Once I’d explained my thinking and talked about Liz’s keynote, the silences got a little shorter and I like to think the rest of the team understood what I was getting at.

Rightshifting track

On the second day, track one had 3 rightshifting sessions back to back. I’d read a little about it, talked a little about it at the Lean Agile Glasgow meet-up, but I didn’t really understand it. To my untrained eye, it looked like a variation on CMMI maturity model…but on it’s side rather than a pyramid. I’m glad to say that once Bob Marshall had assembled us into a circle and talking with us, it all became a lot clearer. I also left with an understanding of why large, established firms can find moving to an Agile process to be so painful.

Ian Carroll’s “Rightshifting in action, using Kanban for organisational change” was also riveting. I’d seem his webcast on Systemic Flow Mapping before and after that session I am even more motivated to perform the exercise in the business unit I work for. Maybe nearer Christmas I’ll find the time. I suspect a first attempt will be more of a learning exercise but even asking the questions and trying to describe the current process should be very revealing.

I didn’t manage to attend Torbjörn Gyllebring’s session, but I’ll be watching the video soon.

Why Agile Fails – Matt Wynne

I liked Matt’s presentation style and found the content engaging and interesting. My key points to take away were:

Wynne’s 1st Law of Software Delivery: If it isn’t fun, you’re doing it wrong.

I’ve used that a number of times myself now 🙂

The other key point was around Cargo Cults and Shu Ha Ri, which describes the learning process as (please excuse my poor rephrasing):

  • Shu – we practice the forms rigorously and without deviation
  • Ha – once disciplined, make innovations and question the forms
  • Ri – completely depart from the forms, open the door to creative technique, and arrive in a place where we act in accordance with what our heart/mind desires, unhindered while not overstepping laws

That last step really does sound like the phase we all want to be in, but the first phase is also very similar to when you see Scrum cargo cults in large organisations. Matt stressed that we shouldn’t look down on those teams as the first stage certainly does show similarities with Cargo Cults. However, the differentiation here is that a Cargo Cult doesn’t realise that it’s just the first step on the path to understanding…and maybe they just need some appropriate assistance or coaching to make that mental leap.

So, what now?

Well, those are my highlights of Lean Agile Scotland 2012 from the perspective of what gave me cause to stop and think/re-think about how I approach things. I enjoyed it all, and I can’t wait until Lean Agile Scotland 2013.

My team is moving from a Scrum process (which we’d tailored somewhat over the years) to something more Kanban, dropping estimation…or at least only giving very brief, initial estimates or deciding to break stories into smaller pieces.

I am determined to construct a Systemic Flow Map of my business area. I think it would reveal all sorts of valuable information about how our teams could work more collaboratively and streamline the work we do. I’m likely to blog about that for advice because I still have questions about exactly how to do it.