First, I am guilty and struggle to loose weight that I need to loose. Having had our 4th child, sleep seems to trump the early morning work-out. However, most of the time it is because I lack discipline to prioritize my eating habits and getting to the gym. The point is, I know what I need to do, but don’t do it very well. How does that apply to this topic you ask? Well…
I have spent the last 5 years learning and experiencing everything I can with Agile Software Development practices and more specifically helping organizations change with implementing Scrum. I believe Scrum is definitely a project management process. It provides the engine or context for how the team gets and manages its work; which inherently takes a level of distraction away from those involved. We are no longer wondering what we will work on next nor are we wondering who is responsible for facilitating the gathering of requirements to feed the process. Many organizations are plagued by chaos, not necessarily because the projects they are working on are inherently chaotic, but because it involves many humans and humans tend to make things complex anyway. We all have our experiences, knowledge, personalities, and opinions (to name a few). Perhaps sometimes we let that actually get in the way of our ability to succeed; when in fact we should embrace our differences, celebrate the diversity, and use that as an advantage to build the greatest products we can for our customers. I can use the same analogy on software development in general. Can we agree that software development is complex, risky, and uncertain? Can we agree that humans are horrible at estimating things and that our accuracy only decreases with distance? It is tragic that we let all of the factors contribute to our chaotic and stressful world. We even go as far as to work extra hours and put in extra time to manage the chaos. What I love about Scrum is that it provides us a mechanism to get and manage our work in a way that helps reduce the risk, complexity, and uncertainty by learning from experience as we go (empirical process control model). We get to celebrate the things humans are not necessarily good at and build upon a process that helps reduce complexity, risk, or uncertainty. It doesn’t ever completely remove it, I can’t think of any process that is perfect in estimation and completely removes risk.
I also find it disturbing when I see teams in the world that would classify themselves as “Agile” but state that they can’t predict a timeline or release schedule (because they are “Agile”) or don’t know what they will work on until the next sprint (because the are “Agile”). I believe any Agile process is really focused on discipline and behavior and sometimes the lack of these disciplines get pawned off as “We are Agile” when they are actually just lacking focus, discipline, and collaboration. I also think there are some organizations where that is perfectly fine. Which is to my next point. You can’t simply take a template and apply it across all organizations. I tend to teach in my training courses that the software problem is in fact to build the right system, build it the right way, and deliver it at the right time. I have added on to that the fact that it is our goal to make the organization happy as we go. That actually covers a lot of ground. I don’t mean bending to the will of the organization when they challenge the Agile ideals and principals, but I also believe that they help should have input on any Agile process to meet their needs. After all, they need instrumentation to properly run the business that funds all of our paychecks. If the organization wants a year-long roadmap, then why are we afraid of that or think that is the opposite of Agile? I agree that one year is probably the max I would support (I think 9 months is reasonable). We can still utilize the Scrum process to help us determine that roadmap given we are disciplined and stick to the foundational metrics inherent in the Scrum process. The main reason teams can’t simply go off and do this tomorrow is that historical accomplishments are the foundational building blocks to forecasting the future in the Scrum process.
I actually like using Story Points (for many reasons we won’t get into here), but one reason is that they allow our sizing to be independent of duration. Duration is derived based on the sizes we complete each iteration over time. Simply put, if you have 100lbs of stone to move and you can (on average) move 10 lbs of stone per cycle, then you can estimate you will be done in about 10 cycles. If 10 cycles in your world is 10 minutes, then you can estimate that you would be complete in about 100 minutes (or 1 hour and 40 min). If you have 100 Story Points on your backlog and you complete (on average) 10 Story Points per sprint, then you can estimate you would be done in about 10 sprints. If 10 sprints in your world is 2 weeks, then you can look out 20 weeks and start gaining an understanding on duration. I would certainly not just go off and set a complete roadmap based on that metric alone, but it gives you a guideline to discuss with your teams as you start looking at all of the real work to be completed. I believe in triangulation of metrics, use many different scenarios and ranges to finally settle on delivery date that the team can actually rally behind without killing themselves. Constant and sustainable development paces are often the first thing to go in software development, mainly because we are poor at planning (because we are “Agile” right?). The Agile Manifesto itself states that we value responding to change over following a plan, but it doesn’t say we don’t plan. In fact I would contend we spend more time planning than actually following “the plan”. That is the Agile part about it. The real point is to become very disciplined in our engineering practices that we can actually ship software when we want / need to. We must focus on “DONE” and stick to working vertical slices of features each iteration that encapsulate all of the work required to deliver the software (including automated testing, builds, and deployment). Scrum will certainly expose each iteration why we would struggle with those pieces and we must constantly get better at building software. This also allows us to release software to some environment other than our local machines to gain feedback from the user to incorporate into the product. We want to ensure each iteration we are responding to the customers’ needs and are in-fact building a solution that solves their problems. If we are not, we get to talk about where we missed the mark and get better next time or we get to understand better what the customer wants (because frankly, they might not even know what they want). Feedback is the key and the only way to get true feedback is to let the customer use the real software system instead of showing them a booklet of requirement docs when they really can’t experience the workflow of the system.
One of the artifacts of the Scrum process is a product backlog. What a great idea! We have the single source to list the infinite list of things to do, paired with a finite list of resources to accomplish them. Scrum provides a role that is focused on this effort that goes under-valued in many Scrum implementations. The Product Owner is the facilitator and gather of requirements and is there to synchronize all of the inputs into the backlog to make final decisions on the most value to the customer and prioritize that work. In my opinion; they should be the expert on the customer’s problem and be able to represent that problem to the team. Scrum, however, doesn’t specifically prescribe how to build a product backlog. It does guide us in the direction of User Stories, Epics, and Themes, but what does that mean? It can be different across organizations and type of work. This is usually the first point of contention teams feel in Scrum. If we don’t have a disciplined approach that allows us to be predictable, then often times we loose the ability as the team to set the roadmap dates of delivery. How many times has your team been faced with meeting a date on the roadmap that really was manifested out of thin-air, or better yet, an Executive commitment to the customer. Scrum doesn’t tell you how to deal with that, it is an organizational problem that requires discipline to resolve. Scrum can provide the context for how to get and manage our work to produce a reliable derivation of dates for a roadmap. You still have to get good at building a backlog and keeping Executives at bay (directing them to the Product Owner should they want work placed in the team’s span of care). Often times, Software Envisioning grid-locks a new Scrum team from being better. We try to squeeze every single envisioning item into a 1-4 week iteration in which case we spend more time designing than developing software and that is where people start seeing a break-down. I recall back in 2009 when I was new in my Scrum journey and approached Ron Jeffries about this very subject. He point blank told me that Software Envisioning is not part of the Scrum process. I was now freed and understood that there perhaps is a process that lives on its own in our Product Management teams that constitute their ability to understand what their customers want and work on very high-level envisioning ideas to bring to the team in the way of design work. Whether that means that some developers are working ahead in the backlog on design in the user stories or they have their own process to conceptualize ideas. None are particular in violation of Scrum. The point is that the best folks to design and envision the software are doing so, and then bring them to the development team. The development team is inherently involved in the design and can provide feedback as well as plan (not build) the architecture required to support anything that might challenge current architecture. I am hinting to the fact that usually the first place we expose as a problem area in Scrum is the lack of discipline on our Product Management teams to get solidified in envisioning their product and providing a product vision to the team. The logistics of Scrum will take care of themselves as we begin to break large, risky ideas into Epics, Themes, and User Stories that are worked on in the sprints.
Back to complexity, risk, and uncertainty to describe software development. Basically you have a human that has an idea (Product Owner), they have to communicate that idea and vision to a group of people (the development team) who basically write text files of instructions that get interpreted by a computer and displayed on a screen in a workflow that a user can understand to accomplish their tasks (from DFW Scrum’s co-lead Gary McCants). The point is to start working on those ideas and release those text files to the computer for presentation to the user to make sure we are constantly on the right track. If we are not, we didn’t waste an inordinate amount of time to understand that. We inspect, adapt, and improve the next time. Often the burden of the process (discipline) takes center stage and gets used as the reason Scrum didn’t work. Solving problems and being the best engineers we can requires discipline. It is like trying to get your body in the best possible shape. You have to put the cupcake down, get to the gym, and be disciplined on your day to day activities. Some days you will do well, some days you won’t, but at least you can track and understand where you didn’t do well and get back to good. Agile is basically signing up to be a body builder in Software Development. It requires focus, discipline, and hard work. Any organization can get on board with that if they understand the goal, the process, and are provided the instrumentation to make decisions. I actually think we have it wrong when managers demand productivity, productivity, and productivity. Don’t get me wrong, I believe in being the most efficient we can be and deliver the most output we can, but I believe we first have to focus on predictability. In the end, that is the valuable metric that our Executives want and Scrum is just the process that we can leverage to help us wrap our head around it.
