Why Agile Teams Often Don’t Thrive

tumblr_mddkdtUC1a1r9qnhbo1_400

On TUE May 19, we hosted Ron Jeffries (yes the guy who has been building software longer than you have been alive) at DFW Scrum. Ron is a frequent visitor of our group and has provided unbelievable guidance and direction to my own teams in our transition to Scrum and agile development practices.

 Ron spoke to our group on “Why so many agile teams often don’t thrive”. I think someone who actually helped manufacture the “Agile Manifesto” would probably be a good source of information on helping us identify our pitfalls in adoption and progress.

 Ron specified 2 main points to this actual topic:

  1. Teams don’t know how to ship software every sprint

  2. Cultural changes necessary do not take place (like helping a team become better at shipping software each sprint)

 Ron’s points above aren’t really the meat to this topic in my opinion. I had a few people struggle with what Ron was trying to convey and I immediately came to his defense and asked “don’t you guys get it?”. Obviously not, therefore, this blog post. Ron’s message hit me right in the sweet spot and validated my fairly close alignment with his views on agile development practices (he obviously has been a big influence in my journey).

 The point? We spend so much “flipping” time (Ron would have used a different word) on making sure our Scrum Masters, Product Owners, and Organization Stakeholders are prepared for the agile journey that you know who we leave out of this preparation? You got it…THE TEAM!

 If we break Scrum down into its components, we “should” have 1 Product Owner, 1 Scrum Master, and 3-9 Development Team members. Where do you think the force lies in where we spend time in equipping the teams? That’s right, the backlog (which is important), how will the business react, will they engage, do we have the roles required, how will we estimate, how we will plan a roadmap, so on and so forth.

 Those are all important questions to ask, but you know which artifact in Scrum we have lost focus on? The actual increment. The increment is the most important artifact in Scrum. If a team can’t ship software each sprint, they can’t tell a story; thus they can’t get the trust and confidence needed to let the business people focus on the what; letting the teams focus on the how. Too often the teams retreat into a hell where they need to know everything up front because we leave no room for failure, mistake, or God forbid, a misunderstanding in the actual requirement.

 Ron started the talk with relating to why Kent Beck created XP (eXtreme Programming). Basically to protect the team. I have a similar mindset in my transitions to agile and will usually build an API translation layer of communication over the team to protect them as much as possible from the cruft the organization might place on them. Our goal? Ship software! Not time-sheets, estimations, roadmap planning, meetings; SHIPPING SOFTWARE!

 Product Development is a collaborative game. Each team has different skill sets, each human brings a particular experience and skill-set to the team. What we should focus on in our agile adoption is “do these teams have the skills necessary to refactor and work incrementally to deliver slices of software to a tested/integrated environment every sprint”?

 Most of the time, the answer is no, but we will spend all kinds of money on the process, the roles, and the PMO type communication abstraction layer to ensure our teams can be left alone. As they are left alone, they struggle greatly in the organizations deployment, branching, merging, and build strategies. Please don’t misunderstand, I truly believe a team needs to provide the level of transparency to the business and PMO group on our progress, but too often this turns into an inordinate amount of work for the team.

 Ron painted a picture about a company party. Most of the time you find people socializing and talking about business. You usually find the development team folks bunched up together talking about technology and cool patterns they learned over a hack-a-thon project. They do this because they are passionate about the technology in their job and solving problems. Let’s incubate that environment and provide more opportunities for these folks to learn more about technical practices that encourage quick delivery. Branching/Merging techniques that allow us to receive changes frequently to minimize conflicts. The nitty gritty of software development.

 Ron encourages the various groups in Scrum (Scrum AllianceScrum.org, etc…) to think of ways we can foster growth in engineering practices. Let’s build more programs around the team and not just the Scrum Master / Product Owner.

When you go back and look at your teams, let’s see how much extraneous work we have around these folks that could be making them unhappy and uninspired in their quest to build innovative products. If your development environments are a large part of the problem, are we equipping the team with knowledge on how to incrementally refactor, incrementally make better paths to production, etc…?

 Back to Ron’s message, teams often don’t thrive because we don’t equip them with the knowledge and skills of how to incrementally build software. We focus on so much other activities and events in Scrum, that we tend to loose site of the most important thing, the increment. Ron is on a mission to help the Scrum Alliance and other organizations to start focusing on these types of activities that allow Developers an opportunity to show skills based on industry practices that can be learned through various techniques.

 I have an old saying in service organizations that says “if you aren’t servicing the customer directly, you better be servicing someone who is”. In Scrum, I can say “if you aren’t servicing the development team directly, you better be servicing someone who is”.

 We are in the business of shipping software, therefore, we better be the best shippers of software that we can. Are you?

Role of a Project Manager in Scrum

New-Project-Manager

We had a great turnout at DFW Scrum last TUE for “The Role of the Project Manager in Scrum”. Many organizations struggle with what to do with the Project Manager when they have a Scrum Team. Chris Eberhardt and myself facilitated this fascinating conversation. The reality is, there is not “standard” or “cookie-cutter” approach to the answer. Every culture, company, value system, etc… are a little different (or a lot for that matter). So we took the approach of facilitating conversation more than giving answers, because honestly; most people in the group have the answers, they just need validation if they are violating Scrum values.

The one big point that does apply to each company is that while the role itself is not described in Scrum, the responsibilities and duties still have to be done. Typically those duties and responsibilities are absorbed by the 3 roles in Scrum (Product Owner, Scrum Master, or Development Team). We also found some duties that aren’t necessarily part of Scrum, but needed by the organization to drive visibility, transparency, and coordination. Those burdens should be placed on the team and we should work together to figure those things out and who is best to serve those duties.

There are some organizations that exist where Development (the Scrum Team) is just a small piece of the overall “project”. In that case, we might have a project manager taking the metrics from the Scrum Team and providing visibility to the overall timeline for the other pieces. In Software, many of the Go To Market (GTM) activities might not involve the Scrum Team at all. As such, a Project Manager might piece together the finished software with the sales, support, finance, etc…

In other cases, perhaps a Scrum Master fulfills those duties in tandem with the Scrum Team duties. It really just depends. Whatever the case, the word “Project Manager” should not be a deterrent to doing Scrum. Scrum is very adamant about some things and does not outline a role for Project Manager, that doesn’t mean that we can’t figure out how to carry out the functions of the role in its absence (or presence). Be pragmatic and work as a team to figure out what works best. A traditional Project Manager tends to have duties and responsibilities that violate the tenants of Scrum. That is why aspects of that role are transferred to specific roles on the team.

Don’t let compliance and other “governance” terms deter you either. Those can be built into the definition of done for the team and considered in all aspects of our planning and completion. Naturally there are some governance processes that actually hinder productivity of the team, let’s be transparent about that and show the impact of the team and then have the team propose some possible solutions.

Attached is our Deck used to facilitate the discussion, the big take-away is this: “What kind of organization are you”? Each one presents benefits and challenges of Scrum.

Types of Organizations

  • All Scrum Masters (0:n)
    • No Project Managers, only Scrum Masters
    • Not the same as all Project Manager titles have been replaced by the title of Scrum Master
  • One Project Manager and One Scrum Master (1:1)
    • Every project has a Project Manager and a Scrum Master
  • One Project Manager and Several Scrum Masters (1:n)
    • Every project has a Project Manager but there are several Scrum Masters

Certificate of Occupancy

Image

Have you ever been in your building and noticed a “Certificate of Occupancy”? These gems enforce a safe occupancy number that are built around fire codes. How many people could realistically exit this building in the case of a fire? My question, as it relates to software development and Scrum Teams is; why don’t we use this same kind of approach to protecting the sustainable development pace for our team?

This came to me really quickly as our office is expanding. No matter what WE want to do, we are bound by the occupancy certificate stating that we can only fit X number of people in this building. If we violate that (and are caught) the consequences would be great with the city, but even greater if we did have a fire and were not able to get out. Part of Scrum’s role in our teams is to help us identify and stick to a constant and sustainable development pace (throughput). As such, we should have “certificates of occupancy” in place to help us legislate that rate. Most often however, we are constantly pressed by management or our Product Team to do more, regardless of the cost. Most teams are actually rewarded for their outstanding effort and exceeding their capacity. Don’t get me wrong, we want our Product Team to want the world and push us. But, what happens if you run your engine at 15,000 RPM for a long time? What happens when your CPU is at 100% capacity? What happens if the highway system is at 100% (or even more)? Things come to a screeching halt and permanent damage ensues.

People (like computers and engines) are not built to operate consistently at 100% capacity or more, and doing so has grave consequences for productivity in the long term. Helping a team become aware of their sustainable rate, and predicting delivery off of those metrics inherent in Scrum; help maintain that level of stability. The team has control over what to commit to and what to hold. The pressure is less because the backlog and roadmap are built with their sustainable pace in mind. Believe me, if you don’t give dates for delivery, someone will. So it might as well be the development team, armed with great historical evidence of their ability to deliver as well as planning deliveries based on those realities and accomplishments.

I am certainly not saying there isn’t times where we do come together and work extra hard to get things done. I’m just saying those times have to be throttled and well understood by everyone. In addition, we should be really good at what we do and have discipline and rigor in our processes. As we continuously improve and search for better and better ways to do things, we will be productive and increase our output within our sustainable rate.

If we wanted to have more than X people in our building, we could petition the city for a permit that would increase that number (or not). The burden of proof would be on us to indicate the value of such exercise and the safety measures we put in place to allow the increase. The point is, we just can’t arbitrarily state “we need to work harder this sprint to get Y done”. Next time it will be “Z”, “A”, and “B”. That will never go away. Put checks and balances in place to ensure when you violate the sustainable and constance pace in Scrum, that we recognize the impact of that and that we are doing it for real reasons. If you want to increase the certificate of occupancy in your backlog and sprints, do so with the team and build roadmaps based on sustainable pace. Running our engines at high RPMs should be the exception, not the rule.

The Flat Tire

20140512

In regards to Scrum, many people struggle to wrap their heads around the process. While it is a fairly simple framework with very little rules and prescription, it is very difficult to actually do; logistically speaking. Build a backlog…ok, well how do I do that? Have a sprint review…ok, well how do I do that?

What actually holds this thing together is the vision and goal of Scrum. The goal is to enable us to get / manage / expose our work in a consistent process while consistently increasing our Agile Engineering practices that will allow us to ship the most valuable software we can for our customers when we want to ship it. This implies that whatever is getting in the way of you shipping software needs to be resolved. Scrum’s goals are to actually expose all of those reasons why we can’t get our code from our developer’s machines into Production with little to no human interaction along the build, test, and deploy path.

Manual testing will eventually show you that you aren’t able to move quick enough each and every time you want to ship. Not having a good backlog before your next sprint will show you that you are very reactionary and as such aren’t able to ship valuable software each sprint. Scrum exposes those leaks and that’s a good thing, celebrate when you find an inefficiency; that is something that we can all fix and improve.

One of my close colleagues in the industry has always said that Scrum is like a syringe that injects Agility into the organization. I love that analogy; I have also thought of many myself. Scrum is like going to the tire repair center. If you suspect a leak in your tire, a lot of the times, you just don’t know where it is coming from. We take it to the professionals as they can expose the leak with a tried and true technique. Take the tire off, soak it in some soapy water, and voila’ your leak appears (or maybe several leaks appear). Now, we can actually fix those leaks and get the car back on the road. What if, we found those leaks, and simply just put the tire back on the car and said “yes, there are leaks, we just don’t have time to fix them now”. At that point, you would likely drive off and continue to waste time by putting air in your tires; only to have it leak out later. Worse, you could be left stranded on the side of the road because of a flat. The right thing to do is to stop down, fix the leak, and then move on.

Scrum exposes those leaks. We the people, still have to fix our problems. No one thing or process can fix our problems, we humans have to dig in and find out the best way to solve the problem. Scrum is that agreement between the team to react to things in an empirical way, not a planned/predictive way. Take the time to take the tire off the wheel, soak it in water, find the leaks, and fix them. Some leaks are so big that we have to replace the tire. That’s ok too, celebrate that we learned of that problem. Our goal is to ship valuable-quality software when we want to and leaks prevent that from happening. Fix the leaks.

Does Your Team Have a Definition of Done?

Does your team have a definition of done? In the dictionary, the term done means:

“Having been carried out, accomplished, or finished”

Many teams struggle to know when they are done, whether it is from the user story clarity itself or the team’s engineering paradigms paired with the diligence they have established in DONEness.

Have you ever seen anyone say “That’s Done Done”, or even “That’s done, done, and done”? We should have one singular version of what DONE is and the entire team (all roles) stand behind that version as a filter for things they choose to demonstrate in their sprint review sessions. If the user story is not done, then don’t call it done, don’t split it up, just move it to the backlog for re-prioritization.

Having a definition of done allows you to start your strength training exercises in Agile Engineering. When you first start a work-out program, you have a fairly light regimen that allows you to lift weights, do cardio, and stretch without getting hurt. As you strengthen yourself, you start changing your regimen to have more discipline and rigor. You might add more days to the work-out, increase the weights, or even do complex aerobics.

The point is, if you don’t continue to escalate your work-out, you won’t get stronger. You have to exercise those muscles. At some point, they will reach their comfort zone and be able to perform your current work-out with minimal exertion. The point of increasing your work-out regimen is to get stronger and stronger.

The same holds for your definition of done. You have to start out where you are, but have the idea of where you want to go. Your definition of done should change overtime as you strengthen your Agile Engineering practices. Work those development muscles and become stronger and more Agile the more you add to your rigorous definition of done. Your customers, the business, and your team will thank you for it, I promise.

If your definition of done is the same now as it was 1-2 years ago, you aren’t Agile and you certainly aren’t improving. Measure it like velocity, the trend over time should show an upward slope, always increasing your team’s strength in doneness.

Empirical Process Control

Image

​In Scrum Training, I started touching on the difference between the “Waterfall” method of project tracking (Defined Process Control​) vs. Empirical Process Control​. At the root of the Waterfall system is the idea that for the same inputs into a system, I can expect the same outputs and that the work ahead of us is well defined and known. In software, that usually is not the case, as we really don’t know that what we have built is what the customer wants until they can “experience” it in the way of working software. The other idea with Scrum is that we focus on getting the teams so disciplined in the logistics of shipping software, that is no longer the pain point or hold up in the process, we can ship software when we want (which to me is the definition of “Agile”).  

It is easy to get caught up in the “logistics” of a process and just focus on the mechanics, I think this article by Barry Hawkins does a great job conveying more of the “why” behind Scrum. If we can focus on the ideas and principles of the process, we can use that as leverage to get better instead of saying “these are the ceremonies in Scrum, go do them”.
 
There is another article that brings good insight into how people have turned waterfall into something it wasn’t originally intended. I bring this up as I regularly speak to University students regarding Software Development approaches (typically traditional SDLC is the first process they learn). They were in awe of the article I show on Dr. Royce’s “Managing the Devleopment of Large Systems​“. Fig 3 of the Waterfall process shows iterations are recommended. He describes the traditional SDLC model in Fig 2 inviting risk and failure. Dr. Royce’s paper is great and offers a great perspective on a big problem we were trying to solve in the 70’s. We also seem to always be in a rush and sometimes miss what an article was trying to convey. Thus began a formal process that the government originally initiated to help “control” projects.
 
To sum this up, I love the diagram below (provided by Ralph Stacey). When people ask “when should we use Scrum”?, I display this diagram and say, “anything past Simple are best suited for Scrum” (although I would still use Scrum on the simple ones as well). If any process involves humans and creativity, it is automatically a “complex” project in my humble opinion.
 
Image

Sail or Fail? Navigating Test Automation Pitfalls

Image

Paul Grizzaffi joined at DFW Scrum last night to go over the topic above. Too often, automation has to be justified and bought into before we spend resources on it. I like how Paul compared it to an Insurance Policy. No one likes to pay for insurance unless of course the tornado takes your house down or the car is totaled. Look at automation in that it will save us from potential catastrophes and customer implications of bad software. However, unlike insurance, automation can make us more efficient and “Agile” at the same time.

I like to say that being “Agile” in software development, simply means that you can ship software when you want. If a feature was demo’d and thought it was good to release to customers, could you do it? Or is it a lengthy / painful process to push code from our developers’ machines to production?

Automation is definitely a specialized skill set at this time, we all know it would be better and a good idea to do, but how do we go about doing it? Paul’s message was clear that you can’t template an approach to automation, but he can share with you some good practices / questions to ask. Other valuable lessons are what NOT to do.

Quality is everyone’s job, automation is everyone’s job. How do we fit it into our development processes? Paul’s presentation is available for download, check it out and  hit him up for questions if you have any.

Quick take-aways

  • Don’t try to automate a broken process (garbage in and garbage out)
  • Must know your process and job before you can automate
  • Think of your automation as a safety net / alarm system (confidence in making changes)
  • Test cases are your contract between the requirements and the interface
  • Try not to change Test Cases for your automation unless the contract changes
  • Automation is an insurance policy, you want a high deductible you get low cost (risky)
  • Discuss automation in context of opportunity cost (not ROI)
  • A human used to do x now its automated they can go to y
  • Exploratory testing is opportunity cost, computers are good at repetitive tasks, humans are not