Agile Documentation: Oxymoron or Necessary Evil?

I know “documentation” is usually a double-edged sword as far as developers are concerned – it sucks when you don’t have it, but it also sucks when you have to write it. And I’m not really referring to inline comments or XmlDoc, because if a developer isn’t doing that then they’re just asking for trouble. I’m referring more to requirements, test cases, end-user documentation, and soforth. The stuff that developers usually find someone else to write for them

Agile being the fancy new development movement nowadays seems to put a lot of emphasis on being “lightweight” (aka agile), which at first glance seems to imply that you should avoid doing anything unnecessary. And by all definitions that’s what it’s about – cutting out all the extra loopholes that development departments go through to produce a better product in the end, in a more efficient manner. But another key factor of Agile is identifing and managing risk efficiently, and what better way to reduce risk than to document objectives, requirements, execution plans and results? So, how does an organization slim down the overhead of documentation without cutting it out alltogether?

According to information written by people much smarter than myself, the goal to keep in mind is “an agile document is just barely good enough for the situation at hand”. “Agilists” identify that traditional forms of documentation are infact risks in that they increase the amount of lock down and time spent on planning a project that should be subject to change and quick to implement. Because of this, an agile document is something that is a) useful, and b) able to change. There’s a few key components of this approach:

  • Use Executable Specifications – it’s all fine and dandy to say “this button should do this” in a word doc, but it’s about 100x more productive to write that in a executable way, using techniques like Behaviour Driven Development or Test Driven Development. So instead of just writing text that someone has to read and manually verify, you’ve written a user acceptance or developer-level test that other parties can execute and say “Yes, this meets the requirements”.
  • Document Stable Concepts, Not Ideas – with the old waterfall approach, everything was planned out and documented before it was executed. There was a detailed project plan, a detailed requirements document, and a detailed design document. With agile, there’s a dynamic list of project requirements, with a subset list of target cycle requirements – no one bothers writing down the project plan or design because no one knows what it’s going to be until it happens. Once you know you’re doing requirement A, THEN you document the details of it – that way your application is adaptive to change, no one requirement is locked in until it’s selected for the current cycle. That’s not to say we should be leaving everything to the end; if you return to a document after completing the project you might not remember why you made the design or implementation decisions you did. The best approach is to keep things in a “notepad” format to be finalized later when support and system overview documents are finalized – be able to quickly jot down why something was done, when you do it.
  • K.I.S.S. – Documenting the crap out of a project doesn’t have any impact on its success; rather you’re probably wasting resources and increasing the chance of failure. To keep in line with an agile methodology: use the simplest tools, create simple content, and depict models simply. Tips for this include writing in point form, use sketches of models rather than complicated visio diagrams, document only enough to provide context, and link to resources instead of repeating their information*.
  • Make Information Available – put the information in an appropriate and accessible place – this way your documentation won’t go to waste because it’s in the most obvious place for it, and having it accessible will increase knowledge transfer. Also, displaying models and other sketches in public view will increase communication on the project, which will reduce the need for detailed documentation because everyone will know what’s going on.
  • Document With Purpose – you should only be writing a document if it fulfills a clear, important, and immediate goal of the overall project. Write documents that are useful for the people reading/executing them, and keep in mind one type of document isn’t necessarily relevant to each project. Work with consumers of the documentation to figure out what the minimum-yet-sufficient amount of documentation is. And don’t write an entire document and THEN check with someone to see if it’s adequate. Check early, check often.
  • Find Better Ways to Communicate – keep in mind that Wikis or MS Word isn’t the only way to communicate. Consider using email / forum threads, IM, the phone (heaven forbid!), video conferencing, or even a whiteboard discussion to create and discuss models, develop requirements, and clarify implementations. Of course there’s only a subset of this that would be useful as a resource after the fact, but understanding and collabration is much more valuable than documentation. Also, identify forms of documentation that you actually use – if you create a UML diagram at the beginning of a cycle and then never update it afterwards, then there’s not much point creating that UML diagram, is there?
  • Treat Documentation Like a Requirement – the documentation step should be in your cycle planning right beside a user story or other requirement – estimate it, prioritize it, and put it in the sprint queue. This way the time you spend on documentation has the same metrics as the time you spend on requirments, and can be managed appropriately. Now you can manage the resources and risk associated with said documentation. Likewise, you can make people requesting documentation explain and justify their request, just as you would make a client explain a feature request, and documentation can be subject to the same scrutiny a requirement is.
  • Recognize That You Need Documentation – documents are just as much a part of the system as the source code is. Your team is not only responsible for creating a system, but also for enabling other teams to enhance, support, and maintain that system.

In summary (yes, I know this drags on) , documentation hasn’t lost any importance under this new regime of software development. It has changed, it has become more efficient and agile, but it’s still a requirement to help businesses reduce risk and produce systems that will outlive the project teams that have created it.