The flowering of Agile – since the 1940s
Today, some don’t realise that agile development principles are older than programming.
And some mistakenly think they emerged out of OO programming.
In 1943, agile design and development principles were promoted in the “Skunk Works” division of Lockheed Aircraft Corporation.
Kelly’s rules for Skunk Works say nothing about what is being designed (aircraft); they are about engineering project management.
Five of his rules correspond to four values of the agile (software development) manifesto, more than half a century later.
Kelly’s rules - 1943
The Agile manifesto – 2001
(3) Use a small number of good people.
We value individuals and interactions - over processes and tools
(9) The contractor must test his final product in flight [and] in the initial stages.
(5) A minimum number of reports required, but important work must be recorded thoroughly.
We value working software - over comprehensive documentation
(12) There must be mutual trust between the [customer] and the contractor
[and] very close cooperation and liaison on a day-to-day basis
We value customer collaboration - over contract negotiation
(4) A very simple... release system with great flexibility for making changes must be provided.
We value responding to change - over following a plan
Wikipedia speaks of further iterative and incremental development methods emerging in the 1950s.
The 1970s saw more evolutionary project management, and adaptive development approaches.
And IBM promoted Joint Application Development (JAD) workshops.
Contrarily, Royce defined a "waterfall" software development method in 1970.
But waterfall project failures in the following decades led others to promote "rapid", "iterative" and "dynamic" development.
By 1990, James Martin’s Information Engineering approach presumed Rapid Application Development (RAD).
And the Dynamic System Development Method (DSDM) was published in 1994.
So, agile methods have been around for decades.
In the last decade, almost every architect in my classes says their organisation uses SCRUM or similar, and many say Kanban as well.
Kanban is a scheduling system for lean and just-in-time manufacturing developed by Taiichi Ohno at Toyota.
Wikipedia says it “originates from the simplest visual stock replenishment signaling system, an empty box.”
This was “first developed in the UK Spitfire factories during the Second World War the late 1940s.”
Some think agile is an extension of OO thinking.
Agile (as defined in the agile manifesto) is rather an approach to engineering project management.
The values in the manifesto reflect those used in engineering and software projects from the 1940s onwards.
The first rapid/dynamic application development conferences and methods were primarily a reaction against waterfall projects.
Alongside the revolt against waterfall methods in the 1980s, came the introduction of graphical user interfaces and OO programming languages (OOPLs).
Naturally, the development of user interfaces proceeds best in a collaborative, incremental and agile way.
For this and other good reasons, OO thinkers adopted, developed and promoted agile practices.
Some agile methods, like extreme programming (XP), have grafted particular software engineering practices on to agile methods,
However, practices like pair programming and test-driven design are not specific to OO programming.
Kelly’s rule 10 (below) was that “the specifications of hardware must be agreed to well in advance of contracting”.
Similarly, agile development methods recommend stabilising the “infrastructure” in advance of software development.
Where the infrastructure can include not only platform technologies but also any persistent data structure that must be populated and maintained.
Many business applications are built on top of a large persistent data structure that records entities and events of interest to business people.
Agile system development proceeds best when the logical structure of this state data (this memory) is stable.
So, it helps to get the logical data structure as complete and right as possible before coding – to minimise refactoring later.
It also helps to implement the data structure as directly as possible, to minimise the complexity and processing overhead of any data abstraction layer.
As OO thinker Craig Larman pointed out, decoupling with no good reason is not time well spent.
Agilists are fond of promoting principles; it is important to recognise that some of them can be in conflict.
Contrary agile principle
Keep the system simple
You ain’t gonna need it
Design for future flexibility
An agile approach comes naturally to a team given flexible requirements for a system that can start out small and simple, then be extended.
The challenge is to use agile methods in other cases, when given rigid requirements for a large and complex system that must be completed as a whole.
For other obstacles to agility, scroll to the end of this Agile Architecture paper.
Again, Kelly’s rules are about engineering project management; they say nothing specific to the nature of the artifact being designed.
Agile development principles are older than programming itself.
If agile has an underlying theory, it is surely a psycho-bio-sociological one.
Agility in a software development team is a matter for socio-cultural thinking.
Flexibility in the design of a software system is a matter for general system theory – a different thing altogether – see the next paper.
Rules 3, 4, 5, 9, 10 and 12 were mentioned above; here is the full list.
1. The Skunk Works® manager must be delegated practically complete control of his program in all aspects. He should report to a division president or higher.
2. Strong but small project offices must be provided both by the military and industry.
3. The number of people having any connection with the project must be restricted in an almost vicious manner. Use a small number of good people (10% to 25% compared to the so-called normal).
4. A very simple drawing and drawing release system with great flexibility for making changes must be provided.
5. There must be a minimum number of reports required, but important work must be recorded thoroughly.
6. There must be a monthly cost review covering not only what has been spent and committed but also projected costs to the conclusion of the program.
7. The contractor must be delegated and must assume more than normal responsibility to get good vendor bids for subcontract on the project..
8. The inspection system... should be used on new projects. Push more basic inspection responsibility back to subcontractors and vendors. Don't duplicate so much inspection.
9. The contractor must be delegated the authority to test his final product in flight. He can and must test it in the initial stages. If he doesn't, he rapidly loses his competency to design other vehicles.
10. The specifications applying to the hardware must be agreed to well in advance of contracting. The Skunk Works practice of having a specification section stating clearly which important military specification items will not knowingly be complied with and reasons therefore is highly recommended.
11. Funding a program must be timely so that the contractor doesn't have to keep running to the bank to support government projects.
12. There must be mutual trust between the military project organization and the contractor, the very close cooperation and liaison on a day-to-day basis. This cuts down misunderstanding and correspondence to an absolute minimum.
13. Access by outsiders to the project and its personnel must be strictly controlled by appropriate security measures.
14. Because only a few people will be used in engineering and most other areas, ways must be provided to reward good performance by pay not based on the number of personnel supervised.