Agile 3 – On agile businesses and agile systems

https://bit.ly/2UkJCz0

Copyright Graham Berrisford. One of several hundred papers at http://avancier.website. Last updated 12/04/2019 11:02

 

Avancier’s agile papers

This is the third in a series of papers.

1.      On the beginnings of agile

2.      On agile software development

3.      On agile businesses and systems

4.      On systems thinking ideas used in agile

5.      What is agile architecture?

6.      EA in the world of agile architecture

 

Management scientists like to discuss what a business should do to survive in a changing world.

The vision is of an agile business, enabled and supported by agile systems.

This paper offers a few thoughts on that vision, especially on trade offs that have to be made.

Contents

On business change. 1

On the Internet giants. 3

The limits to “the learning organisation”. 4

On agile systems. 5

System design tradeoffs (REPEATED) 5

Reengineering and refactoring. 7

 

On business change

This article on the Indian government’s approach to EA bemoans digital transformation initiatives as technology rather than business service and data led.

It features eight reference models: Business, Application, Data and Technology (cf. TOGAF), also Architecture, Performance, Integration and Security.

The steps for national, state and local governments to design architecture initiatives are outlined as below:

·         Performance: Identify goals, targets and indicators. What/when/how to measure and monitor? When/How/Who to analyze and evaluate results? How/when to reports results?

·         Business: Map current government services. Identify required services. Plan to transition.

·         Data: What/where/when data is needed? Who needs and owns the data? How will it be used, consumed and protected from illegitimate use? Where is it stored?

·         Integration: Identify integration requirements to realise whole-of-government perspective needed to achieve the goals.

 

This is classic, strategic, cross-organisational EA.

The results and outcomes require aggregation of data from various sources.

Realization of cross-domain, cross-sectoral services require data interchange, common data standards and communication protocols.

What does agility mean in this context?

 

This Forbes article quotes a survey finding half of senior leaders rate their business transformations a waste of time.

It suggests the issue is that most organizations are rushing to adopt the manifestations of digitisation.

For example, appointing chief digital officers and announcing programs to implement artificial intelligence.

Before getting to grips with what needs to change and therefore where their focus needs to lie.

“They don’t understand what is causing friction in the organization”

Five finding were:

·         Worry about data before technology; reliable data is vital.

·         Reengineer processes; don’t just digitise them.

·         Smash the silos (in the company structure).

·         It’s all about the business model.

·         People first, technology second.

 

A case study in business change https://lnkd.in/f5ehEF2

The reference is to a case study in business management, planning and performance management.

Note that it is a bank, a data processing business, and therefore ripe for digitization.

Note also: “We were so bad, it was easy to improve” and the transformation was radical.

 

The business was transformed over several years through a combination of:

·         Better business performance metrics (e.g. related to customer experience).

·         Better personal performance appraisals against KPIs

·         Running meetings properly

·         In-sourcing of what had been outsourced.

·         Empowering employees to make incremental process improvement

·         Migrating customers and bankers to digital operations.

 

Note that the changes made and their success can be attributed to effective top-down leadership.

It is fine example of 1) top-down leadership of strategic change and 2) encouragement of people to improve their own processes.

All organisations must strike a balance between these, and where the balance lies depends on the nature of the business.

In nuclear power station, the last thing you want is workers making up their own processes.

 

Long-term targets will evolve

There is always a target, be it for a 15 day sprint or a 5 year transformation.

In the banking case study above, the directors sponsored a 5 year legacy replacement programme.

With the target building blocks in place, it took several more years to complete the digital transformation.

However, EA does not predict the long-term future in any detail.

It does not define a target architecture that cannot change.

 

Read the 1986 PRISM report that informed EA "Dispersion and Interconnection: Approaches to Distributed Systems Architecture".

The case study concludes.

“Roger many nights in bed wondering about whether he had done the right thing.

Could he have anticipated the merger, and adjusted the architecture accordingly?

Mark and Roger decided that there was nothing they could have done about it.

Had they tried to factor in a merger, they would have been unable to make any progress at all.”

To accommodate change, an EA framework must be highly iterative, with continual requirements change management.

 

Note also: the bank’s stated ambition was (and remains) to become an internet giant, but not every business can become an internet giant.

On the Internet giants

Discussion of agility in business and in systems often draws on what are in fact rather peculiar examples.

The exemplars are application-centric internet giants like Facebook, Amazon, Netflix, Google and Spotify (FANGS).

 

These businesses depend on others for their content - and the quality of that content.

·         Amazon sell what others create; they can always replace a rejected element by another.

·         Uber piggy back on drivers managing their own cars

·         Air BnB piggy back on owners managing their own accommodation.

·         Google extract information from users and sell it back (in patterns that are not always useful)

·         Facebook, Twitter etc. share low quality data and low quality thinking (undermining the notion of authoritative sources).

 

Characteristics of FANGS

Characteristics of many enterprises

The business is based on an application or software product.

Business operations are human (healthcare), material (retail) or mechanical (mining)

The business is a self-serve platform for its users.

Business operations are end-to-end pipelines or value streams

Business employees do not interact personally with customers

Business employees interact personally with customers

Operation/transaction throughput is extremely high

Operation/transaction throughput is relatively low

Low integrity, loose-coupled operations.

Tight integrity/quality requirements. e.g. money-handling, health, and safety-critical.

Mistakes and misjudgements are resolved after they happen

Mistakes and misjudgements in business operations cannot be redressed after-the-fact

Develop their own business-specific software.

Use more packaged COTS than bespoke software.

 

Many programmers think it cool to copy what FANGS do - regardless of relevance, cost and complexity.

But most businesses are not like FANGS.

And design patterns that FANGS have developed for extreme scale web applications come with costs and downsides.

The limits to “the learning organisation”

Systems thinkers promote the idea that every business should be a "learning organisation" (after Senge).

The learning organisation diverts resources from optimising what a business does now to considering what it might do differently in future.

It facilitates the learning of its members, and continuously transform itself.

An aim is to change, adapt, and transform to meet customer’s needs, with the current workforce.

 

My small business focuses on what we do well.

That does not mean we stop exploring and changing.

Quite the contrary; we continually explore what our customers do and want.

We continually adapt and improve what we do to suit the changing world out there.

 

The case study above illustrates the success of the learning organisation idea.

It does so in the process of transforming the nature of the bank’s business operations.

But let us not fool ourselves about how far and fast a business can change.

 

Some businesses are inherently more flexible than others

The likes of Facebook, Amazon, Netflix, Google and Spotify (FANGS) are peculiarly software-centred businesses.

Changing web sites and algorithms is one thing.

Changing physical assets and workers with special abilities, knowledge or skills is another.

No amount of organisational learning can save a bricks and mortar high street store and its sales assistants from the threat posed by the likes of Amazon.

 

Remember Blockbuster? It could not compete with Netflix, and went out of business.

Blockbuster's CEO had the chance to buy Netflix in the 1990s.

Supposing he had done so; the outcome would surely have been a reverse takeover.

It would have saved the Blockbuster name - but not its stores or its employees.

 

Evolution of the wider economy

Improvements do emerge from continual iterative sense-response adaptation.

But to preserve every business forever is impossible – and undesirable.

In regretting what happened to Blockbuster, people overlook it was good for Netflix.

 

At first sight, a high rate of business closures looks like a bad thing

It often mirrors a high rate of starts ups – which may well signify a healthy economy.

Sometimes, it is better that a business becomes extinct, and is replaced by a different kind of business.

The business world as a whole evolves that way, rather by maintaining every business entity forever.

On agile systems

The notion of an agile system is not a simple or single idea

Since a system can be flexible in several ways, as described in the table below.

 

Agile variation

The system can

yet might also be

Extendable

be extended with extra features

not malleable (the Open-Closed principle)

Versatile

do many different things

complex and difficult to change

Malleable

be readily changed

simple and not versatile

Self-organising

change its own roles and rules

unpredictable and disorderly

 

Agile development does not necessarily produce agile systems.

The plain fact is, digitisation of systems is and always has been a difficult and painstaking task.

Designing an agile system is a challenge; and there are always trade offs.

System design tradeoffs (REPEATED)

Gurus in agile development and architecture frameworks promote general principles.

It is important to recognise where principles can be in conflict.

A theme here is that balancing tradeoffs is more important than following uni-directional principles.

 

For many decades, general system design principles have included:

·         Decomposition: modularise a large monolithic system into subsystems.

·         Decoupling: strive for tight cohesion within a subsystem and loose coupling between subsystems (Constantine, 1968).

 

The aim of these principles is to facilitate the design, management and change of subsystems.

Taken too far however, both principles have negative effects, and designers must strike a balance between extremes.

 

Simple subsystems or simple messaging?

This table contrasts the qualities of dividing a system into larger or smaller subsystems

 

Subsystem

Inter-subsystem

Size

Processes

Messaging

Coupling

Macro subsystems

Larger

Longer

Less

Looser

Micro subsystems

Smaller

Shorter

More

Tighter

 

Decomposing a system into small, simple subsystems creates complexity in the structure of the system.

It increases the volume and complexity of inter-subsystem messaging.

 

There are many different ways to implement messaging, ranging from tightly to loosely coupled.

Size matters in the sense that smaller, subsystems typically merit tighter coupling than larger subsystems.

 

Local agility or global disintegrities and delays?

Note first that decoupling is not a single concept - our architect classes cover a dozen or more ways to decouple subsystems.

 

In “Applying UML and patterns”, OO thinker Craig Larman said to pick your battles.

“It is not high coupling per se that is the problem; it is high coupling to elements that are unstable in some dimension, such as their

·         interface [the list of processes/services that are provided or required]

·         implementation [internal procedures, physical features or technologies]

·         mere presence [availability when needed]”

“If we put effort into “future proofing” or lowering the coupling when we have no realistic motivation, this is not time well spent.”

 

Skillful decoupling of subsystems should help people manage and change each subsystem on its own.

But increasing the agility of small subsystems can make it harder to meet higher/wider goals.

E.g. a principle of “microservices” is to “decentralise data management” by dividing one coherent database structure into smaller ones.

The aim is to divide the work of application development into smaller applications, to be developed by relatively autonomous development teams

In the right circumstances this is a good approach, but it can lead to downsides.

Generally speaking, designing a system to be flexible can lead to complexities, delays, disintegrities.

 

Agile system or simple system?

To make a system more agile usually requires a redesign that makes the system more complex.

Setting out to build an agile system adds time and cost to development.

 

Agile system principle

 

Agile development principle

Decouple subsystems

Can be contrary to

Keep the system simple

Design for future flexibility

Can be contrary to

You ain’t gonna need it

 

Agile system or fast system?

An agile system is usually slower than a rigid one.

E.g. shifting business rules from procedural instructions into data variables (that end users can change) tends to make a system slower.

 

Agile system or data integrity?

Continual improvement at a microscale risks disintegrity at a macro scale.

E.g. we change our digital course materials during a course.

Inevitably, our printed course material, digital course material and web site get out of step.

The best we can promise customers is “eventual consistency” – probably.

In other businesses (e.g. money handling, safety-critical) data integrity can be mission critical.

So, decoupling subsystems and allowing inconsistency between them can be dangerous.

Reengineering and refactoring

 

Reengineering to reduce technical debt

Reengineering an old system to use new technologies (or for scalability) does not make a system more agile in other ways.

And it can complexify; in one case, after being rewritten in Java, a single-server legacy system required a dozen servers.

 

Refactoring

Clearly, a system that emerges from incremental development is not optimised for requirements unknown at the start.

To keep the system manageable may require substantial “refactoring” of the system from time to time.

This requires investment over and above normal change requests.