Agile 8 – Some commentary on SAFe
Copyright Graham Berrisford. One of several hundred papers at http://avancier.website. Last updated 23/09/2019 13:55
“Thanks! You article contains some really good reflections on the difference between developing with agile methods and designing solutions to be agile.”
All architecture and governance is adaptive to a degree.
But inserting "agile" or "adaptive" before "strategies", "standards", "principles", "architectures" and "governance" is oxymoronic.
Obviously, strategies, standards, principles and architectures can and should be relaxed (via waivers) and changed now and then.
But the purpose of governance against strategies, standards, principles and architectures is to steer and constrain choices and actions.
The more they are relaxed, the faster they change, the less they can be effective or useful in meeting their purpose.
At the extreme, the most flexible business is one in which there is no constraint on human choices and actions
Which implies no strategy, no standards, no principles, no architecture definition and/or no governance against them.
One author speaks of “adaptive governance” meaning the definition of "rules" to which "compliance" can be left to "self-service"
These rules might better be called "principles" or "guidelines" or "design patterns"; and "compliance" to them might better be called "conformance".
This isn't well called “adaptive governance” it is merely classifying the advice that a governing body produces.
And if that body makes no attempt to detect or measure the usage of the advice they publish, then they aren't really governing at all.
This is a supplement to a series of mostly short papers.
It is assumed here you have read paper 6.
This paper is a commentary on some of the ideas in SAFe.
The authors of SAFe have produced a lot of material, with various lists of concepts, ideas, principles and practices.
How far is SAFe scalable above software architecture?
Is it about the abstract architecture of a system, or the system itself?
And is it a coherent framework or a collection of sometimes conflicting ideas?
To begin, the authors define agile architecture thus:
· “Agile Architecture is a set of values and practices
· that support the active evolution of the design and architecture of a system
· while implementing new system capabilities.”
SAFe practices are based on nine concepts that have evolved from Agile principles and methods, Lean product development, systems thinking, and observation of successful enterprises.
The means making decisions are made in a proper economic context, wrt value, cost and timeliness
This is ambiguous since there are two very different kinds of systems thinking.
· General system theory and cybernetics applies to the systems under design and development.
· Social systems thinking applies to the teams who design and develop systems.
The means maintaining multiple requirements and design options as long as possible; commit as late as possible.
The means each iteration results in an integrated increment of a working system.
Subsequent iterations build on the previous ones.
Increments allow fast customer feedback and risk mitigation.
They also may become minimum viable products (MVPs) or prototypes for market testing and validation.
In addition, these early, fast feedback points help determine when to ‘pivot,’ where necessary, to an alternate course of action.
In Lean-Agile development, integration points provide objective milestones at which to evaluate the solution frequently and throughout the development life cycle.
There is regular evaluation of financial, technical, and fitness-for-purpose measures needed to assure that a continuing investment will produce a commensurate return.
Lean enterprises strive to achieve a state of continuous flow, where new system capabilities move quickly and visibly from concept to cash.
The three techniques do three things: match demand to capacity, enable fast completion, and minimise wait times.
Cadence creates predictability and provides a rhythm for development.
Synchronization causes multiple perspectives to be understood, resolved, and integrated at the same time.
Applying development cadence and synchronization, coupled with periodic cross-domain planning, provides the tools needed to operate effectively in the presence of uncertainty, inherent in product development.
Lean-Agile leaders understand that ideation, innovation, and the engagement of knowledge workers can’t generally be motivated by individual incentive compensation.
Achieving fast value delivery requires fast, decentralized decision-making.
However, some decisions are strategic, global, and have economies of scale that justify centralized decision-making.
Since both types of decisions occur, creating a reliable decision-making framework is needed.
Other core ideas are listed in this sentence:
“The Agile Architecture principles support Lean-Agile development through
· emergent design,
· intentional architecture,
· design simplicity, as well as
· designing for testability, deployability and releasability.”
Let us take collaboration between architects, developers and others for granted.
Intentional design would mean designing a system ahead of development.
Emergent design means that the design of a system emerges from incremental development.
Architecture versus design?
The implication here is that after a higher-level design (architecture) has been agreed, lower-level designs can emerge.
Intentional architecture implies some up-front, high-level, design documentation.
Explicit articulation of higher-level architecture(s) is the basis of all enterprise and solution architecture frameworks (including TOGAF of course).
For sure, at least some architectural oversight of agile system development is desirable.
And people ought to optimise the balance between that and emergent design, depending on various factors.
Where does agility enter the picture?
Remember agile mean “willing and able to speedily respond to change”.
Intentional agile architecture implies:
· Documenting an architecture that can be readily changed and/or
· Designing a system that can be readily changed – an agile system.
The first might be achieved by minimising the architectural documentation, keeping it lean.
The second might be achieved by generalising the architecture.
But does this mean keeping the architecture abstract (so it need not change), or designing the architecture to be changeable?
These are two different ideas.
Agile development favours the Keep it Simple (KISS) principle, but there is a fundamental design trade off.
Designing a system for flexibility usually implies (CFC) complexify for configurability
And a configurable system is likely to run slower.
Does this mean minimising the design documentation, or minimising the code written?
Or minimising the number of locations, components, messages, network hops, processes, databases?
Or minimising the size or length of components, messages, network hops, processes, databases?
There are many trade offs between these qualities.
And none of those design strategies necessarily results in an agile system.
In an article promoting Agile Architecture, a CTO is quoted as saying the following.
“The only way to manage a large distributed system is to keep things as simple as possible.”
OK, but isn’t that precisely because the "architecture" of large distributed systems is inherently complex?
If you want a simpler system, don’t distribute it!
As you may know, Fowler’s first law of distributed objects is: “Don’t distribute your objects”.
“Keep things simple by making sure there are no hidden requirements in the design.”
What are hidden requirements as opposed to visible ones?
You may know that agile development favours the You Ain’t Gonna Need It (YAGNI) principle.
The idea is not to anticipate future requirements, and to refactor when it becomes necessary.
But architecting an agile system does mean including features in the design that anticipate future requirements.
And a system cannot be agile in every possible way, it has to be designed with specific kinds of change in mind.
[And no] hidden dependencies in the design.”
What are hidden dependencies as opposed to visible ones?
We can make all dependencies visible in architectural documentation
OK, but that implies a heavy non-agile documentation overhead.
We can all remove synchronous interactions between distributed components,
OK, but that increases the complexity of maintaining data integrity dependencies and compensating transactions.
Read our Microservices paper for some decoupling trade offs.
“Cut technology to the minimum you need to solve the problem you have.”
It certainly seems a good idea not to use technologies you don’t need.
But what does this actually mean in an enterprise application design context?
Use thin clients with no application code on client devices?
Don’t use middleware when you can use REST?
Many enterprise architects promote middleware use.
Read Smart endpoints and dumb pipes for discussion of the synchronous v asynchronous trade off.
“It doesn’t help the company to create artificial and unneeded layers of complexity.”
OK, but nobody does this deliberately unless they are trying to design for maintainability.
And if asked, many sponsors will say they do want an agile (flexible, configurable) system.
To achieve this, you’ll have to make the system more complex than is needed in the short term.
For sure, test-driven design is an agile development principle.
Tests are built as “executable specification”.
But there is a trade off, since the more test cases you build, the harder it is to change the test cases.
And therefore, the longer it takes to test and release a new system.
Other papers on SAFe include six or seven principles – depending on which you read.
Some are variations on agile development principles of the kind above.
Three of the more architectural principles are discussed below.
This means having enough (say 3 to 9 month) infrastructure to complete near-term product backlog without refactoring.
What is infrastructure? The term covers many things.
It can include client devices, server devices, operating systems, database management systems, middleware, data storage etc.
To a software engineer, it includes what they use by way of a database schema and module libraries (class libraries, DLLs...).
The trouble is, 6 months is a rolling future, and 1 month from now, the 6 month future will be different.
So, surely maintaining the runway to anticipate the 6 month ahead implies continual refactoring?
What does this mean?
If it means building the simplest system that can possibly work – that is already agile development principle.
But building the simplest possible system tends to inhibit its flexibility and configurability.
If it means building the simplest architecture – that implies documenting a simple high level design.
One can always abstract to a high level design that is simple and relatively stable – but also vacuous – which doesn’t sound helpful
And it is far from clear that decoupling subsystems (as in Microservices) is well described as the simplest architecture.
How to “ simplify both the design and the communication between the teams.”
The proposal in SAFe is: “Domain-driven design, design patterns, and applying a system metaphor”
First, remember that “If you can’t draw it, it isn’t a pattern” Christopher Alexander.
Some articles on SAFe equate principles or standards with patterns.
But design patterns are not standards, and they often come in pairs.
What architects need is guidance on the trade-offs between patterns.
Wrt design patterns in general
Following design patterns tends to complexify rather than simplify.
Many common design patterns are designed to make a system more flexible by decoupling communicating parties
But, by adding a level of indirection, and perhaps allowing temporary disintegrity, they make a system more complex.
Generally, making a system more agile tends not only to make the architecture more complex, but to make the system slower as well.
Wrt domain-driven design in particular...
In 2002, Martin Fowler observed that Domain-Driven Design is difficult to learn and best reserved for complex systems with “rich domain models”.
And in January 2017, Wikipedia said: “Microsoft recommends that [domain-driven design] be applied only to complex domains.”
(Where I guess rich implies substantial use of inheritance.)
In the 1980s. the UK government’s structured design method (SSADM) had transformed an object-based domain model into event-oriented procedures.
Fowler wrote that event-oriented “Transaction Scripts” are simpler for many enterprise applications.
Both sources presumed reuse would be achieved by delegation to subroutines from event procedures or transaction scripts.
Read Domain-Driven Design, Transaction Script and Smart UI for discussion of the simple v complex trade off.
It seems likely that many if not most enterprise applications are more easily designed and maintained using Transaction Script or Smart UI patterns.
You simply factor out data object-based modules where data-centric rules are invoked from several transactions or screens.
Transaction scripts can equally well publish events (for others to consume) and log events (for subsequent query and replay).
Wrt a system metaphor...
A good system metaphor helps to facilitate communication.
Use this link to find a good discussion of the idea https://en.wikibooks.org/wiki/Software_Engineering_with_an_Agile_Development_Framework/Iteration_One/System_metaphor
Note that the metaphor for an agile system has to be generalised, which can weaken the clout of the metaphor.
And beware, of course, that metaphors can be misleading.
Analogies with the architecture of concrete buildings, airplanes and city plans have misled many about the nature of enterprise, solution and software architecture.
You may look at the architectures of buildings (in run-time reality) and at architectural drawings of them (in design-time descriptions).
You can see immediately (visually) whether the run-time reality resembles design-time description – or not.
By contrast you cannot look at the run-time reality of a software system.
You can see only a user interface that reveals nothing of the software architecture.
Enterprise and solution architects rarely look at the executable design-time description.
And looking at the code of a software system can be of little or no help anyway.
Instead, architects document a much more abstract design-time description using natural language and graphical models.
They then govern software development for compliance of the code to the intent of that documentation – rather than to its visual form.
OK, but how does this help us produce an agile architecture?
Surely here, to be agile means being able in mid-course software development to adopt an innovation with relative ease.
However useful an innovation might prove to be, adopting it is a departure from incremental development.
Until the innovation is understood, we don’t know how to anticipate it.
Until the innovation is on the road map, we don’t know we need to anticipate it.
And at that point, adopting the innovation is likely to incur the time and cost of substantially re-architecting or re-enginering the system.
So, in what sense does researching innovations enable “Agile Architecture”?
To the mirage of agile architecture we should add the mirage that after a series of YAGNI and KISS increments, substantial refactoring to resolve a major problem or introduce a major innovation can readily be accommodated without major time and cost implications.
SAFe includes a lot more than above
It includes other management-oriented ideas such as "lean portfolio management" and "flexible budgets”.
All free-to-read materials at http://avancier.website are paid for out of income from Avancier’s training courses and methods licences.
If you find the web site helpful, please spread the word and link to avancier.website in whichever social media you use.