Tuesday, May 3, 2016

Ideas to improve documentation in AGILE

AGILE isn’t about documentation” I am sure many of us must have heard it. Believe me, it’s not the truth. There are many documents required to be created as a part of project management in AGILE. Product owners and scrum masters are the ones who primarily work on the documentation related tasks. But it is a collective responsibility of the team and you shouldn’t be surprised to see many documents being worked upon by the scrum team. When the main target (in AGILE) is to deliver the product in the shortest possible time, it becomes essential to know some ideas and tips that help practitioners spend less time on documentation without impacting its quality. Here are some cool tips –
  • Keep it short and simple (KISS): While in AGILE documentation, the idea is not to decrease the number of documents (or reduce the importance of documentation), the idea is to have crisp and short documents which will speak only about the key items that are essential for the document and not linger around providing unnecessary details.
  • Frequency of document updates: Number of updates happening in AGILE are much more than that in traditional processes. This is because updates keep happening almost in every iteration; however that in traditional mainly happens only in the beginning of the cycle. If you don’t see updates happening on the documents (or anyone document) then try to understand the reason behind it?
  • Pair review: Quality has to be maintained not just at the product level but also at the documentation level. It is hence important to review and improve documents. Pair review is a concept followed for the same purpose. These are informal reviews usually done within the team by other team member and suggestions are implemented in the document.
  • Automate exhaustively: Try to develop tools, (on excel / other office tools) that can automate the tasks for documents and save time. This saved time can be used in other project management / product development activities.
  • Non-functional requirements: Agile teams have a practice of treating the non-functional requirements as story points. Creating separate stories make those tasks appear as separate entities. However it is not the right practice to adopt. Making performance and other non-functional requirements core to the functionalities is the right strategy and that’s what managers should practice to save time in the cycle.
  • Avoid overlap: AGILE is about focusing most on the product that is being developed. Hence minimizing the number of documents, their contents and yet covering all essential aspects in the documents should be targeted. It hence becomes necessary to plan documentation in advance, have clear and crisp templates and avoiding overlap across multiple documents.
  • Simplify content display: A lot of times teams end up creating multiple documents because it becomes difficult for them to find the right content at the right time. Having easier ways to display documents and their contents is the key. Using technology and thus document management systems comes handy in such scenarios.
Thus these tips will help practitioners save a lot of time on the documentation and process related items and focus more on the final output i.e., product delivery and frequent updates (thus improvements). Business will see more features and functionalities in the product and will find more returns to their investments. End users would be happy to see better quality product with good performance and usability (points that are usually neglected due to lack of time in product development). Thus, these tips may not seem to show direct impact but will indirectly help business get more returns on their investments and help in improving the quality of the software.

Waterfall vs Agile Documentation

It is said that Agile document is ‘just barely good enough’ and is produced ‘just in time’ and with ‘just the optimally right amount of content’. Less documentation and time spent in it saves project cost however this shouldn’t come at the cost of increased project risk of not documenting essentials. In this article we will notice how Agile and lean philosophies have shaped the documentation practices too and how those are different from the ones as observed in Traditional approach.

What are the documents created in waterfall (and other traditional) methods?
  • Project charter: Gives information about the scope, objectives and participants in a project.
  • Vision statement: It’s a 1-2 page document that focuses on the plan or main vision behind a particular project.
  • BRD: It’s an acronym for business requirement document. It focuses on questions like ‘what’ to deliver? ‘How’ to delivery? & ‘When’ to deliver?
  • Functional requirements: It consists of a list of all functional requirements for the software being developed.
  • Non-functional requirements: All the nonfunctional requirements like performance, usability, security, etc. are enlisted in this document.
  • Project plan: Contains the entire project schedule and plan. The document enlists the different resources that are working on a project and
  • Architecture & design: Describes at high-level the outline of the software and includes its major components. It also includes the list of drivers that can impact the architecture.
  • Wireframes: It’s a blueprint that gives skeletal framework of a website.
  • Test plan: Contains the test strategy, methodology, approach and list of test cases as will be covered while undergoing the testing cycle.
  • Deployment and maintenance plan: Talks about the deployment schedule, the team members who would be assisting the developers in deployment cycle. The approach that the team members will follow and risks.
How AGILE documentation is different?
Light documentation: In Agile stakeholders try to keep the documents light and low in terms of content. Teams don’t beat around the bush and try to cover all essential points in smaller content. The document is nimble and stakeholders keep in mind the rule of ‘too little documentation and you are in trouble, too much of documentation and you are overloaded’.
Evolving document: As the team progresses into sprints, the requirements and thus the documents keep updating and evolving. Every stakeholder is responsible for appropriate documentation for that project and in daily scrums; any changes to documents get discussed. Also since the frequency of updates is very high, a document management system is used to manage the various versions of the documents.
Document communicates: Communication is an essential key when it comes to managing projects involving cross boundary teams. In such scenarios documents play a key role and are used to track communication. In a lot of AGILE projects, the same concept is followed and laid a huge emphasis on.
Documentation throughout the cycle: In traditional methodologies a lot of time in the initial part of the cycle is invested into documentation and large documents are created. In AGILE documentation and changes on the same keep happening throughout the cycle. Thus agile projects can focus on implementation and delivery right from the beginning of the cycle.
User manuals: Agile doesn’t produce documents like requirements documents or design documents. However it does produce documents like user manual, operations manual, etc. The focus is not on producing ‘to-be-built’ document but on creating ‘as-built’ document.
Document ownership: In Agile the ownership of document lies with not just the project / test manager but it lies with every team member. In sprints as and when different tasks are identified for the team members, document related tasks can be assigned to them.
Thus Agile brings in a lot of differences when it comes to the documentation strategies. These strategies are further built upon the agile principles and inculcate a feeling of collective responsibility (on documentation), continuous improvement and focus on end product. Creating user manuals instead of requirement documents shows Agile team’s focus on serving the end product and its customer even while building documents that are essential to manage the projects. Collective ownership can inculcate a feeling of responsibility amongst the team members and can help each get more clarity on the project and the timelines. Agile thus brings a lot of improvements in the documentation methodologies and saves a lot of time when compared with the traditional approaches.

Software Development World - Agile vs Waterfall Methodologies

There are several ways to develop software, two of the most prominent methods being waterfall and Agile. And as anytime there are two ways to go about something, a debate rages about which is best. Does it matter really? Doesn’t either way give you a product?

What is the waterfall methodology?

Much like construction and manufacturing workflows, waterfall methodology is a sequential design process. This means that as each of the eight stages (conception, initiation, analysis, design, construction, testing, implementation, and maintenance) are completed, the developers move on to the next step.
As this process is sequential, once a step has been completed, developers can’t go back to a previous step – not without scratching the whole project and starting from the beginning. There’s no room for change or error, so a project outcome and an extensive plan must be set in the beginning and then followed carefully.

Advantages of the Waterfall Methodology

1. The waterfall methodology stresses meticulous record keeping. Having such records allows for the ability to improve upon the existing program in the future.
2. With the waterfall methodology, the client knows what to expect. They’ll have an idea of the size, cost, and timeline for the project. They’ll have a definite idea of what their program will do in the end.
3. In the case of employee turnover, waterfall’s strong documentation allows for minimal project impact.

Disadvantages of the Waterfall Methodology

1. Once a step has been completed, developers can’t go back to a previous stage and make changes.
2. Waterfall methodology relies heavily on initial requirements. However, if these requirements are faulty in any manner, the project is doomed.
3. If a requirement error is found, or a change needs to be made, the project has to start from the beginning with all new code.
4. The whole product is only tested at the end. If bugs are written early, but discovered late, their existence may have affected how other code was written.
Additionally, the temptation to delay thorough testing is often very high, as these delays allow short-term wins of staying on-schedule.
5. The plan doesn’t take into account a client’s evolving needs. If the client realizes that they need more than they initially thought, and demand change, the project will come in late and impact budget.

When should you use waterfall methodology?

1. When there is a clear picture of what the final product should be.
2. When clients won’t have the ability to change the scope of the project once it has begun.
3. When definition, not speed, is key to success.

What is Agile?

Agile came about as a “solution” to the disadvantages of the waterfall methodology. Instead of a sequential design process, the Agile methodology follows an incremental approach.
Developers start off with a simplistic project design, and then begin to work on small modules. The work on these modules is done in weekly or monthly sprints, and at the end of each sprint, project priorities are evaluated and tests are run. These sprints allow for bugs to be discovered, and customer feedback to be incorporated into the design before the next sprint is run.
The process, with its lack of initial design and steps, is often criticized for its collaborative nature that focuses on principles rather than process.

Advantages of the Agile Methodology

1. The Agile methodology allows for changes to be made after the initial planning. Re-writes to the the program, as the client decides to make changes, are expected.
2. Because the Agile methodology allows you to make changes, it’s easier to add features that will keep you up to date with the latest developments in your industry.
3. At the end of each sprint, project priorities are evaluated. This allows clients to add their feedback so that they ultimately get the product they desire.
4. The testing at the end of each sprint ensures that the bugs are caught and taken care of in the development cycle. They won’t be found at the end.
5. Because the products are tested so thoroughly with Agile, the product could be launched at the end of any cycle. As a result, it’s more likely to reach its launch date.

Disadvantages of Agile Methodology

1. With a less successful project manager, the project can become a series of code sprints. If this happens, the project is likely to come in late and over budget.
2. As the initial project doesn’t have a definitive plan, the final product can be grossly different than what was initially intended.

When should you use Agile methodology?

1. When rapid production is more important than the quality of the product.
2. When clients will be able to change the scope of the project.
3. When there isn’t a clear picture of what the final product should look like.
4. When you have skilled developers who are adaptable and able to think independently.
5. When the product is intended for an industry with rapidly changing standards.
Both the Agile and waterfall methodologies have their strengths and weaknesses. The key to deciding which is right for you comes down to the context of the project. Is it going to be changing rapidly? If so, choose Agile. Do you know exactly what you need? Good. Then maybe waterfall is the better option. Or better yet? Consider taking aspects of both methodologies and combining them in order to make the best possible software development process for your project.

Agile Methodologies

Agile has 6 methodology and they are XP, Scrum, ASD, Crystal, DSDM and FDD

Documentation

AGILE advocates to minimize documentation task and invest more time on core development activity, however the degree of documentation differ in different approaches. –
  • XP (Xtreme Programming), Scrum, ASD (Agile software development) & Crystal don’t put a lot of emphasis on documentation and minimum documents are created
  • Teams following FDD spend sufficient amount of time in documentation.
  • DSDM requires some documents to be created & degree of documentation is less than that of FDD and more than that of XP

End user involvement

Another significant difference of AGILE with that of traditional methods is that Agile advocates involvement of end user in the software development process. However the involvement of end-users in different formats is different. Eg., –
  • End-user is actively involved in XP
  • Product owners represent end-users in Scrum
  • In ASD, DSDM and Crystal end-users participate in all of the incremental releases
  • In FDD, end-users participate through reports

Team meetings

Extensive communication is also a core principle of Agile and the level of communication differs in the different approaches.
  • Informal daily stand up meetings happen in Scrum & XP
  • Information sharing is through documents in FDD and DSDM
  • Face-to-face meetings happen in Crystal and ASD

Size of projects

The size of projects and the teams also differ as you move from one method to another.
  • While Scrum, DSDM, FDD and Crystal can be followed for projects of any size, methods like ASD and XP are only followed for smaller projects

Sprint Cycle

The iteration time period also differ in the different methods. Here is how they differ –
  • DSDM advocates producing 80% of the solution in 20% of the time
  • XP has a sprint cycle varying from 1 week to 6 weeks
  • Scrum -> 2 to 4 weeks;
  • ASD -> 4-5 weeks
  • FDD can have a smaller sprint cycle too (2 days) and it can vary upto 2 weeks (based on project requirements)

XP (Xtreme Programming)

 Advantages: 
  1. End users (customers) are actively involved in the process of software development. Hence the product developed is very close to what the customer wants.
  2. Team feedback is also taken very seriously and there is a lot of focus on self-improvement
  3. Best practices are well-defined and religiously followed within the teams
Disadvantages:
  1. Documentation is given less emphasis and hence giving reference to an issue / instance in past (while working on projects) is a challenge
  2. At times it is difficult to bring in customer into the team since he is very distant from the development team. Hence some other team member (like sponsor / PM) plays the role of the customer. Thus lack of discipline observed at times


DSDM (Dynamic systems development method)

Advantages:
  1. Allows for efficient project management and strong control on the project lifecycle
  2. Requirement priority approach helpful in delivering most important functionalities first
Disadvantages:
  1. Documentation is complex and time consuming




Scrum

Advantages:
  1. High level of communication is observed in Scrum teams and which increases focus on problem resolution, product improvement and team improvement through feedbacks
  2. It offers certification. Hence it becomes easy for organization to know who are experts and hire people who are ‘certified’
  3. Harvard Business Review has spoken a lot about Scrum. Hence it must be something which can be taken seriously in the business

Disadvantages:
  1. Poorly documented and hence it is too easy to be abused
  2. Product owners have lose control on the project
  3. Changing requirements allow customers to get tempted to keep demanding more and more functionalities


FDD (Feature Driven Development)

Advantages:
  1. Multi-tasking is possible in case of FDD
  2. Methodology can be used in case of applications that are complex because of documentation and reports that are created
Disadvantages:
  1. Complexity is so much that there is no point in using this method for smaller projects
  2. Less communication within and out of team. Thus teams learn less from other individuals and teams



Crystal

Advantages:
  1. High risk and highly important component are delivered first
  2. Effective team communication and this facilitates learning amongst team members from each other
  3. This methodology can be adjusted as per project type and team size
Disadvantages:
  1. The planning & development are not depended on requirements, hence traceability is an issue in Crystal

Monday, May 2, 2016

Why Agile do not work - Some Reasons

Much of agile's success is due to the fact that it "sells" so well by promising solutions to perennial IT concerns: projects that run over budget and time, lack of team effectiveness, lack of true collaboration, poor product quality and dissatisfied customers.

I've been involved in a number of agile projects from all perspectives, as a team member, leader architect and overall responsible manager. I've concluded that agile has not only failed like other fad methodologies before it but, in fact, is making things worse in IT. Yes, there are certain occasions when agile does work, particularly for proof of concept (POC) work involving already well-integrated teams, but I'm talking about 80 percent of projects here.

3 Ways the Agile Methodology Is Flawed

The agile methodology produces the opposite of the promised effects. There are many reasons for this, but I'll focus on are the three most destructive agile principles.

The first, delivery over quality recasts the agile principle of "early and continuous delivery of valuable software." Yes, early visibility is an important tool for customer collaboration. What is dangerous, though, is that focusing on continuous delivery has the effect of creating an unmanageable defect backlog while developers work to put something in front of the customer.

In most agile projects I've been involved with, the defect list has grown faster than it can be dealt with. On one medium-sized project, a huge defect backlog had developed over the previous 18 months and required a full-time role just to manage it. We routinely needed groups of iterations devoted solely to trying to clear the backlog. Not only did that come at the expense of new features, but it naturally introduced new defects—which were just thrown back onto the same endless pile of things to be fixed.

Obviously, large defect backlogs are nothing new in projects, but what is new is that agile promotes the practice of ignoring defects. This leads to unmanageable backlogs and, eventually, to ineffective development. I've watched developers, initially caught up in the excitement of collaboration euphoria, end up psychologically worn down by the perpetual defect list to the point of burnout. Agile projects are responsible for burning out many a good developer, which is far from the agile promise that projects "should be able to maintain a constant pace indefinitely."

The second flaw, development over planning, hits the agile principle of "responding to change over following a plan." In theory, developers code while collaborating with stakeholders to define, refine and change requirements as the projects goes along. The methodology, however, does not distinguish between big and small changes.

Every change has a cost, but agile does not account for this. The result? People often change really big things late in the game using the rationale that since it's an agile project, it can handle it. The only way the project can handle this is by adding iterations. As that happens, defects that might have been easy to fix at one point get harder and harder to fix, since the code base keeps changing.

This principle encourages poor and irresponsible planning while hiding its effects. As iterations continue and the defect list grows, the customer becomes more frustrated—not only because of the lack of quality, but because delivery expectations aren't being met either. This is much different from more traditional practices, where you have a project based on well-defined requirements and changes are managed through a Change Management process—which, while sometimes byzantine, will at least state time and monetary costs more clearly.

The third flawed agile principle is collaboration over management. Agile emphasizes self-organizing teams that "do the right thing" in some sort of group evolutionary manner. Nothing against empowering individuals—many traditionally managed projects don't use people well—but agile isn't always consistent with responsible management.

You don't need a methodology that takes authority away from a single individual. In too many agile projects the Scrum Master is little more than a hapless cowboy waving his arms in despair while his charges go in all different directions at once. You cannot replace accountable and responsible project management with some immature utopian myth of self-organization. That's no science for success.

Changing Agile Thinking With 3 Common Sense Principles

One thing that's seductive about agile is the name. We like the idea of being agile in our thinking. Agile as a methodology cannot deliver agile thinking, however, and inevitably ends up preventing it.

Think of "agile" as the ability to take the input of all the variable elements of the project—budget, time, design patterns, re-usability, customer needs, corporate needs, precedents, standards, technology innovations and limitations—and come up with a pragmatic approach that solves the problem at hand in such a way that the product is delivered properly.

This sounds like a tall order, but really comes down to three common sense principles.

If there's one key quality a good project manager needs, it's prioritization: The ability to take the pressures of all project elements and determine which path to follow based on what's most important to achieve.

Remember, the goal is to deliver a quality product on time and to budget; as a rule, there are always some elements that have to be sacrificed to fulfill the needs of the others. It's the How to Develop Next-Generation Project Managers role of the project manager to define and maintain the project priorities so they can function as a decision framework for team members as they carry out their tasks.

One of the hardest things for many developers is pragmatism. Rather than think practically, they inevitable fall into abstract approaches to problems. I was on one very expensive project based on a purist design in which everything was reusable and could be dynamically coupled together. It was a marvel of abstraction. The only problem was that it didn't work. It had the worst performance I have ever seen in a Java system and was incapable of supporting the required load.

The core problem there was that the team attempted to solve real-world problems with an abstract approach. No matter how virtual technology gets, it's still ultimately based on the physical world that we humans operate in.

Physical problems cannot be solved abstractly. Look at any factory machine and see how many oddly shaped parts it contains, each good for just one very vital physical function. Software is no different. Sometimes things are meant for one use only. That's not a bad thing if it gets the job done and functions properly for many years. 

Finally, dynamism means the ability to switch strategies when the current one isn't working. Many times, despite our best planning, what seems like a good project, design or development strategy hits a brick wall. The most important thing in this situation is to know when to call it off.

If the brick wall is only a brick wall, a bit of brute force—i.e. one or two late nights—will get through it. But if it's really a mountain of rock, it's best not to risk the entire project but, instead, find another way around. It sounds like a simple concept, but I've watched projects go on for weeks, team members beating their collective heads against the mountain, only to conclude too late that another approach was needed.

Agile promises solutions it cannot deliver. It promotes sloppy requirements, hides the true cost of development and prevents effective management. Contrary to what we're told to expect, this leads to long-running projects, dissatisfied customers and an overall IT ineffectiveness.

What we hope to find in the methodology, however, is achievable through agile thinking. With this thinking, we can in fact solve the problems of IT project management and learn how to deliver stable products on time and to budget. If the project team knows how to think and work effectively, then we don't need to look to a methodology to save us from project failures. We can do it ourselves.