Is Agility Making You Less Innovative?
Sunday, March 21, 2010 at 11:37AM It has now happened four times in the last 9 months: A Senior Executive of a company called us up because they were unhappy with their Agile (typically Scrum) practices. It has taken a bit of time to understand the pattern, but I think we've got it: Agility is making some companies less innovative. Let's explore some of the reasons why this can happen and how to fix it.
The Slippery Slope to Less Innovation
When an executive calls us looking for help, we typically ask a few broad questions. Executives at organizations who have been practicing Agile for some time usually report that quality is greatly improved, the development organization is happier, releases are more predictable, and the overall organization, including such things as putting software into production, just runs more efficiently.
So what's wrong? When I ask executives "Are you being innovative?" - meaning - are you creating and delivering "big chunk" innovations that continue to establish or maintain market leading positions, the executives typically exclaim "That's it! We're more Agile, but we're less innovative!". In one case, an executive admitted that he didn't think they'd delivered anything genuinely innovative in over a year. That's unacceptable, and it was a good thing that this executives "spidey sense" tripped.
The slippery slope to less innovation begins when teams adopt many common - and *very good* agile practices: Creating and prioritizing a backlog and producing potentially shippable, working software in 2-4 week time boxes (in Scrum-speak, a Sprint).
Sounds good, right?
Well, it _can_ be more than good. It can be _great_. Teams creating working software in predictable chunks? What can be better than that?
Innovation is Not Predictable
One problem is that innovation isn't predictable. It is a bit messy. As your teams gets better and better at hitting every iteration, they can inadvertently start to favor small pieces of work that always get done. Heck - we tell people that SMALL stuff is good and have a wonderful acronym invented by Bill Wake: INVEST. So, people create small stuff. Big chunky innovations get dropped. They are, after all, risky. They don't always work. They can negatively impact velocity. In fairness, it is entirely understandable that teams start focus on small, safe stuff.
From a product improvement perspective, customers do indeed enjoy small, incremental improvements along a number of dimensions. My friend Jeff Patton teaches this brilliantly when he explains why iterations are critical to successful Agile development.
Unfortunately, while customers often _enjoy_ small, incremental improvements, what they _really want_ are "big chunky innovations" to solve their ever-changing landscape of hard problems. And big chunky innovations are not small, incremental, are not iterative, and don't fit into one sprint. (And don't waste your time flaming me about the idea that customers enjoy small improvements. Sure, we all enjoy small improvements to our favorite applications. But that's not enough. And you know it.).
Epics are NOT big, chunky innovations
The Agile community has readily adopted the term "epic" to mean something that takes longer than a sprint. Typically decomposed into many user stories and related requirements (in creating our online Innovation Games® some of our own epics have exploded into more than 50 user stories and even more related requirements), epics are important for capturing big pieces of work.
But epics are not equivalent to the big, chunky innovations that I'm talking about.
Roadmaps capture big, chunky innovations
The big, chunky innovations that I'm talking about are best captured by your roadmap. Which is needed, because your roadmap is your antidote to the lack of innovation that can creep into Agile teams.
Which brings me full circle to the start of this post. Inevitably, when executives tell me that their Agile teams are being _less_ innovative, they also tell me that they don't have a roadmap, or that their roadmap only extends a meager 6 months into the future.
Keep all the power and greatness of Agile AND maintain your ability to deliver big, chunky, innovations into your market. Create a market-driven agile roadmap, make sure it captures big, chunky innovations, and use it as the first step in setting the direction of your team.
Finally, don't get so hung up on "perfect" velocity. In fact, if your teams are _always_ hitting "done, done" in every iteration, start to worry.
PS. Thanks to my friend Ron Lunde for pointing out much needed edits to the original post.
Reader Comments (13)
Yes, we completely see this in video game development. So true. Like evolution, innovation seems to be "punctuated". Good insight. Thanks.
Luke,
I like that you want to maintain product road-map and product backlog as two separate entities. The road map is populated through market-research or by playing games as 'Prune the Product Tree'. The product backlog is derived from the product road map and is written in language which is close to the product and can be understood by end-users and developers. May be we need to play the game again to have the customer’s input on the release defined features and to get its feedback on the proposed innovation.
My understanding about innovation is that it is something that the user can not state as requirement. He likes when he sees it. It's something that has to come from the product development organization through experience, market research and strategic design. Toyota's Chief Engineer is the person responsible on the features (including innovation) of the car model. It would be interesting to read about the profile of the Chief Engineer.
Thanks for the excellent post.
I think that one of the challenges that "agile" teams face is that they adopt the simplistic Scrum rhetoric at face value and effectively park their brains at the door. Disciplined agile developers adopt techniques from a variety of agile methods, including Agile Modeling. Agile Modeling, www.agilemodeling.com, promotes the concept that you need to do initial envisioning to explore the scope and potential solutions, to do model storming throughout the project to explore the details. Particularly during envisioning you want to consider innovative ideas, but you'll also do so during other modeling efforts throughout the project. Most agile modeling occurs between a few people around an inclusive modeling tool such as paper or whiteboards to help promote the involvement of everyone there.
The best advice that I can give is to think outside the Scrum box (egads, be innovative in your approach to agile!) and consider the true needs of the situation that your team faces. Consider the full life cycle. Consider how to address the scaling factors that your team faces (such as team size, distribution, domain complexity, and others). Go beyond the narrow confines of your certified mastery. To get you thinking along these lines you may find my paper about the Agile Scaling Model (ASM) to be of interest.
Luke,
Thanks for a post full of insight. I also see the risks you are pointing out.
When implementing Agile, I often see the need for a product vision, and The need for long term planning. Your thoughts is valuable complement to what I have learned through years of working in product management.
Great Post. I agree that a lot of Scrum-in-Practice and Kanban is very much focussed on delivering working software in small bites. I would contend that Scrum with 1 month Sprints, big goals, and a culture that supports failure can deliver innovation. Maybe what these companies need is a workshop on Artful Making to get creativity going.
Michael, we contend that big goals are necessary but not sufficient. You need roadmaps.
Great post. I think there are definitely times where a big initiative has to be broken into small chunks and those chunks may sit in staging for some time while the rest of the chunks land in subsequent sprints.
This can often feel unnatural for some teams who are so intent on getting stuff to production every sprint.
IMO, this underscores the need to get extra buy in for these big roadmap bets since everyone needs to be a lot more patient and be ok with nothing "shipping" for a few sprints. Some helpful tactics I've seen work include lots of milestone updates, internal demos, and shipping minimally viable features. I'd also recommend spreading out these big meatballs every couple of quarters to avoid the perceived quagmire of big bets so you can get some meat sauce in between ;-)
Hi Luke,
Thanks for posting about this. It is something that interests me, so I'm afraid you got me typing away ;-)
Fully agree that the quest for stable velocity is not fully compatible with the desire for innovation, since to get stable velocity you need to stay with what you know how to do, and this would by definition not be innovation/discovery.
Your proposed solution seems to address another problem though(?); the disability to align longer term business goals with work producted in sprints. I agree also that most organizations also have this problem(!) I would say that one common reason is the way they manage products. I see very few product owners that actually strive to make cohesive releases (e.g <n> sprints) to adress business goals (might this be what your big chunky innovations are?) They are not used to do this since before agile teams arrived they only option was just pushing in a big chunk of work every year or so and then pray...
Now with agile they can start having actual business strategies and formulate releases to adress these. But many have not started to use this new option. My cure for this would be increasing the skill set of agile product managers (who then would start creating release plans/roadmaps to meet their newly crafted business goals). Similar to your solution, but maybe a different root cause idea?
From my point of view lack of innovation is different problem, often created by some commmon scrum implementation problems:
1. Proxy product owners focused on writing user stories. We do not want this, but the habit may have been created by the initial type of training offered for PO's. This was focused on backlog and user stories. Real PO's/Product manages do not have time to writing user stories in backlogs, they have markets and business things to pay attention to. Also, they may not be skilled in picking good solutions. This is not their jobs. So, they introduced proxyPOs to do this, and with this also delays andmisunderstanding in the process. In Scrum, instead we want real business people to meet directly with the team and explain what their goals are and what probles/need they want to adress. The TEAMS then write their own backlogs with thier expert solution proposals to the issues to be adressed. Innovation comes from team members with domain knowledge and knowledge about what can be done.
2. No slack in process. No innovation can take place if teams are viewed as an engine that needs to be fully utilized to produce backlog items. To produce real value people need time to think and be creative.
Sugggested solution a): Introduce slack in process, e.g. <n> days per sprint where team members explore whatever they like as long as its for the good of the .
Suggested solution b) Educate POs and team on the importance of pull and sustainable pace. Nobody can force a scrum team to overcommit to the level where no innovation do occur. They decide themselves how much work to take on and should keep the pace right so that they can also think ans innovate. This is not our habit in sw dev though, so a change is needed.
Well, that's my take on it!
This is article is interesting, as most people who don't have a clear idea of what Agile really is think that Agile will make the team members, and thus the company, more innovative because of the flexibility and the leeway it gives to the team.
Thanks for sharing...
All you have to do to encourage serendipitous innovation is open some space and time for it to happen. Just look at what Google and IDEO have done with this concept. If you want innovation, open some space and time for it to happen. Scrum is not the culprit here, it is the spirit of the management implementing it. My hunch is that the time-boxes have no space for this aspect of the Agile Manifesto:
"Simplicity-- the art of maximizing the amount of work NOT DONE-- is essential"
Bet you dinner that these "uninnovative" Scrum teams are getting little space and time to be simple, messy etc. In other words, I bet their Sprints are very crowded, with no room for thinking, reflection, silliness, being messy with stuff etc.
Further, and a much deeper discussion, is how we seek single-cause explanations (Scrum's fault, managements fault, etc) when the world is multi- and omni-causal and creates multi- and omni- results.
INTENTION equals results. If there is a lack of innovation, then the intent is exactly that. Scrum is probably NOT the single-source of this problem.
Great post. I agree with Dan's comments and Henrik's comments also.
There tends to be an excessive focus on velocity, to the extent that some large companies fail to innovate adequately, damaging their own futures in the process.
Time to think is key. We need to cherish the innovators, the Apple way, and provide slack in our working day, the Google way. They're not short of innovation are they?
I agree with these points. I think they boil down to one thing "people take agile too seriously". Most of agile was not developed by people who know the first thing about how humans really think or how teams really function. This means the rules are not well tuned to real world situations. The fix is not to take them too seriously, but use them as guidelines.
A lot has already been said above, thanks for the good article and insightful comments. IMHO, this stems from a biased view of Agile/Scrum which I sum up as "it's enough if the team gets it". The roadmap is clearly the PO's responsibility, and if he doesn't have a vision of the product he can share with the team, the team will busily and efficiently deliver features going nowhere...
So ScrumMasters / Agile Coaches: let's stop telling POs that their job is just the bean counting they excelled at when being Project Managers, and let's open them to their real responsibility: guiding the product ship towards new, unknown horizons without getting caught up in the nitty gritty of daily tech decisionmaking (and velocity checking - yuck!).
Don't confuse the Product Cycle (share the vision, plan the releases, review the releases) with the Team Improvement Cycle (Plan a Sprint, Review it, Retrospect)...
And remember: good ideas come when you're having fun, not when you're trying to think harder (or faster) :)
François