General software system theory and design trade offs

Surprisingly little is new in the word of software design.

Discussions in Linkedin, January 2019, promoted me to put on record that:

· software system theory is older and broader than “object-oriented thinking”

· design tradeoffs related to decomposition and decoupling are eternal and universal.

The general principles of software system design inform OO thinking, but stand independent of it.

This paper introduces some general design tradeoffs that are relevant to agile architecture (whatever you mean by that).

Contents

General system theory – basic terms and concepts. 1

System design tradeoffs (REPEATED) 3

Where to specify and code business rules?. 5

Conclusions. 6

General system theory – basic terms and concepts

In classical cybernetics, systems exhibit regular or repeatable behaviors.

The behaviors are performed by some actor(s) and modify some structure(s) or state variables.

That is the kind of system most system theorists are interested in.

The more particular system of interest here is a software system.

It is a discrete event-driven system.

It responds to input events (messages) by performing processes.

It is deterministic; it determines what to do by comparing input data against persistent state data (memory).

The memory takes the form of a data structure.

The data structure relates data types (entities and their attributes) of interest that system users want to monitor or direct.

A system can be described from the different viewpoints shown in this table.

	Behavioral perspective	Structural perspective
External perspective	Service contracts	System interface
Internal perspective	Processes	System / subsystems

The whole quadrant defines aspects of one system; but all is recursive.

Each subsystem (bottom right) is itself a system and recursively describable using the whole quadrant.

Synonyms used by software designers include

· Service contract = method or operation

· Process = method body or procedure

· System interface = API, WSDL-defined interface

· Subsystem = module, component or class

(Unfortunately, people also use the term service for an interface or a component.)

Structural perspective

A system is a structure that can be triggered by events to perform one or more processes.

External perspective - interface

A system can be defined from the outside by its interface, which lists the services it can perform.

The interface encapsulates whatever internal processes and resources the system needs.

The encapsulation boundary is chosen by the system describer or designer, it is logical rather than physical.

(Which means that systems can overlap.)

Internal perspective- subsystems

A system may be composed of interacting subsystems

A system may be described or designed as composed of a few large subsystems or many small subsystems.

A subsystem may be further decomposed, meaning that coarser-grained subsystems contain finer-grained subsystems.

At the bottom level are atomic subsystems - not further decomposed.

Behavioral perspective

A process that runs over time from start to end.

One system-level process may span several subsystems, which each performs a part of the higher/wider process.

A system can act as either or both:

· a client, which requests another (server) system to perform a process

· a server, which performs a process at the request of another (client) system.

Here, client and server are logical terms (nothing to with hardware)

External perspective- service contracts

An invocation is an event or message that triggers a process.

The full contract for a process may be presented in the form of service contract:

· Signature (process name, input parameters and output/response message).

· Business rules (preconditions and post conditions that apply to system state variables).

To invoke a process, a client needs only its signature.

However (see the section on business rules below) a client may also test preconditions and/or test post conditions.

Internal perspective- processes

A process is a sequence of activities.

Both structured and OO methods divide them into two kinds.

An enquiry process returns/outputs a message to inform or direct the behavior of external entities (it has no effect on the system’s state).

An update process has an effect on the system’s state; it changes the value of one or more internal state variables (or else returns/outputs a failure message).

System design tradeoffs (REPEATED)

Gurus in agile development and architecture frameworks promote general principles.

It is important to recognise where principles can be in conflict.

A theme here is that balancing tradeoffs is more important than following uni-directional principles.

For many decades, general system design principles have included:

· Decomposition: modularise a large monolithic system into subsystems.

· Decoupling: strive for tight cohesion within a subsystem and loose coupling between subsystems (Constantine, 1968).

The aim of these principles is to facilitate the design, management and change of subsystems.

Taken too far however, both principles have negative effects, and designers must strike a balance between extremes.

Unfortunately, Constantine didn't explain how we must flex his principle according to the granularity of the subsystems.

There are many ways to decouple subsystems; and there are degrees of coupling.

Fine-grained subsystems are usually better more tightly coupled than coarse-grained subsystems.

Simple subsystems or simple messaging?

This table contrasts the qualities of dividing a system into larger or smaller subsystems

	Subsystem		Inter-subsystem
	Size	Processes	Messaging	Coupling
Macro subsystems	Larger	Longer	Less	Looser
Micro subsystems	Smaller	Shorter	More	Tighter

Decomposing a system into small, simple subsystems creates complexity in the structure of the system.

It increases the volume and complexity of inter-subsystem messaging.

How much complexity in messaging or event flows is a self-inflicted wound? Caused by

· Distributing subsystems that would better be co-located?

· Dividing an application into microservices where it should not be?

· Dividing an application into microservices that are too small?

· Dividing what should be an ACID transaction into several transactions that then need to be coordinated one way or another?

· Programmers directed by the CIO to use the middleware he has spent too much money on, even where it is not needed?

· Programmers eager to teach themselves Domain-Driven Design where they would better use Transaction Scripts or other pattern?

There are many different ways to implement messaging, ranging from tightly to loosely coupled.

Size matters in the sense that smaller, subsystems typically merit tighter coupling than larger subsystems.

Local agility or global complexities, delays, disintegrities?

Note first that decoupling is not a single concept.

Our architect classes cover a dozen or more ways to decouple subsystems.

In “Applying UML and patterns”, OO thinker Craig Larman said to pick your battles.

“It is not high coupling per se that is the problem; it is high coupling to elements that are unstable in some dimension, such as their

· interface [the list of processes/services that are provided or required]

· implementation [internal procedures, physical features or technologies]

· mere presence [availability when needed]”

“If we put effort into “future proofing” or lowering the coupling when we have no realistic motivation, this is not time well spent.”

Skillful decoupling of subsystems should help people manage and change each subsystem on its own.

But recognise that decoupling small subsystems can make it harder to meet higher/wider goals.

And if two subsystems are almost always used together, then co-locate them!

A principle of “microservices” is to “decentralise data management” by dividing one coherent database structure into smaller ones.

The aim is to divide the work of application development into smaller applications, to be developed by relatively autonomous development teams

In the right circumstances this is a good approach, but it can lead to downsides - complexities, delays, disintegrities.

Agile system or simple system?

To make a system more agile usually requires a redesign that makes the system more complex.

Setting out to build an agile system adds time and cost to development.

Agile system principle		Agile development principle
Decouple subsystems	Can be contrary to	Keep the system simple
Design for future flexibility	Can be contrary to	You ain’t gonna need it

Agile system or fast system?

An agile system is usually slower than a rigid one.

E.g. shifting business rules from procedural instructions into data variables (that end users can change) tends to make a system slower.

Agile system or data integrity?

Continual improvement at a microscale risks disintegrity at a macro scale.

E.g. we change our digital course materials during a course.

Inevitably, our printed course material, digital course material and web site get out of step.

The best we can promise customers is “eventual consistency” – probably.

In other businesses (e.g. money handling, safety-critical) data integrity can be mission critical.

So, decoupling subsystems and allowing inconsistency between them can be dangerous.

Where to specify and code business rules?

A software system (being deterministic) applies business rules that compare input events/messages to internal state/memory.

Business rules can be expressed in the form of pre and post conditions.

Preconditions define the state a system should be in before an event can be processed successfully

Post conditions define the state a system should be in after the event (some call them “side effects”).

Specifying business rules in service contracts

Computer scientist Charles Anthony Hoare was interested in the formal specification of a system.

His logic can be expressed in the form of what is called the Hoare triple.

· If the preconditions for a process are met.

· And the process completes successfully

· Then the system’s state will meet the post conditions of the process.

For example:

· If a customer’s debt meets the precondition that Debt + Sale Value < Limit.

· And the sale process completes successfully

· Then the customer’s debt will meet the post condition that Debt (after) = Debt (before) + Sale Value.

OO thinker Bertrand Meyer wanted to define a system’s interface in a formal, precise and verifiable way.

He proposed it should specify business rules related to system state, in way that can be tested.

So he extended interface definitions to capture system invariants and more.

A process may be presented in the form of a service contract thus:

· Signature (process name, input parameters and output/response message).

· Business rules (preconditions and post conditions that apply to system state variables).

Notice that one way or another, a client often gets to know something of server system’s internal state.

Internal state variables (or synonyms for them) may appear in the signature and in business rules.

Implementing (coding) business rules in process procedures

At run time, who tests the preconditions of a process? Client or server?

Bertrand Meyer first promoted Design by Contract in Object Oriented Software Construction, Prentice Hall, 1988.

But again there is a tradeoff to be made between design patterns.

Design by contract

Premise: a client must guarantee the preconditions of a process before invoking that process.

The server is then expected to create the post condition that the Service contract promises.

This might help to keep the system simple.

But it is impractical and/or risky in multi-user client-server database systems.

Servers cannot rely on as-yet-unknown clients guaranteeing servers’ preconditions.

And in distributed systems, clients may not be able guarantee the preconditions of a process before invoking that process.

So in the design of distributed systems, people apply defensive design instead.

Defensive design

Premise: a client is not expected to ensure preconditions hold true.

This means designing such that:

· a server handles any failure of any client to send valid data or ensure preconditions

· a client can either ignore or cope with any failure of any server to produce the desired effects/results.

The server must test the preconditions of a process are met, and if not, it likely return a failure message to explain that.

This might add some complexity to the system.

But it is the normal choice when client and server systems are designed by different teams (see the Bezos mandate in this SOA paper).

Test-driven design

There is an agile development method called “test-driven design”.

This requires developers to code test cases that test post conditions - using assertions.

Conclusions

The general principles of software system design inform OO thinking, but stand independent of it.

This paper introduces some general design tradeoffs that are relevant to agile architecture (whatever you mean by that).