An information theory (part 1)
This page is published under the terms of the licence summarized in the footnote.
All free-to-read materials on the Avancier web site are paid for out of income from Avancier’s training courses and methods licences.
If you find the web site helpful, please spread the word and link to avancier.website in whichever social media you use.
A system might be defined as a collection of actors that interact by the exchange of information.
What theory do we have for how to exchange information? Is there an information theory?
This paper is about the creation and use of meanings in social/business messages and records.
In short, a piece of information/meaning is a data/signal at the point of its creation or use.
The English language is blessed and cursed with more than enough words; the Oxford English Dictionary lists more than half a million.
In discussing information, we seem to have more words than concepts.
Consider data, description, information, knowledge, meaning, model, representation, signal, symbol, wisdom.
How many clearly distinct concepts are there?
The proposal here is that there are two essential concepts.
· Information is a meaning created or found by an actor in some matter or energy that acts as a signal.
· Data is a signal or signal element that can be created or perceived by an actor, and mapped to a language.
(See part 2 for extended discussion.)
The subject matter of an information theory
An information theory ought to address these three steps:
1. A sender encodes information in a matter/energy structure/flow
2. That matter/energy structure/flow is transported to receivers
3. Receivers decode information from that matter/energy structure/flow
Shannon’s information theory is about step 2, the transport of data from sender to receiver.
It is about signal processing operations such as compression, storage and communication.
It helps ensure data structures arrive intact, with minimal data loss or distortion
It is not about the meaning of the data.
An information theory for system architects has to address the meanings that are encoded in and decoded from physical forms of matter and energy.
Data is a physical signal or signal element created or perceived by an actor.
It is mappable to a language (V&G) by an actor.
All models and descriptions (mental, oral, hand-written and digital) are physical data forms.
They are physical data forms in which information can be communicated or stored.
Any model or data form can be translated into another kind of model or data form.
Humans are good at manipulating mental and oral models.
Computers are good at manipulating digital data forms.
Information is data at the point of creation or use.
It is a meaning created or found by an actor in some matter/energy structure/flow that acts as a signal or data form.
Information is meaning attributed to a physical signal or data form by
· the intention of its creator/sender or
· the perception of a perceiver/receiver/consumer.
Unintentionally created information
Information theorists usually focus on information that is created and communicated by intent - deliberately
But actors can find information in physical phenomena that occur without any intent to convey information.
For example, a sunflower detects sunlight, perceives the position of the sun, and turns towards it.
A mental model is some kind of memory, against which a new perception can be matched.
If two actors find the same information in a perception, then they must share mental models well enough.
(They share their logical readings/understandings of their mental models, rather than share their physical brain matter.)
Consider two actors looking at the shadow of a person cast on a cave wall.
If the two actors reckon a shadow describes a man’s outline, they must share much the same mental model of a typical man’s outline.
If both recognise the particular man whose shadow is cast, they must share the same mental model of that particular man’s outline.
Suppose one actor reckons the shadow is of a man, and the other actor says the shadow is of a woman.
The shadow is not the information – it has no meaning on its own
Its meaning depends on how well it matches a mental model
These two actors do not share mental models well enough to find the same information/meaning in the shadow.
One signal or data form can mean different things to its creator and its consumers.
(Enterprise data architecture is partly about ensuring all creators and consumers attribute the same meanings to the same signals/data forms.)
Communication involves inter-model translations.
1. A honey bee observes some pollen and its direction and distance from the hive.
2. It transcribes that observation into a private (biochemical) model.
3. Then (back in the bee hive) it transcribes that model into a public (dance) model.
4. Another bee observes that dance and transcribes the logic of it into its own private model.
5. Then, carrying that physical model, it flies off and successfully locates the pollen.
How do we know bees must store and share their mental models of pollen locations?
Because we can read their dance and predict their behaviour.
There is information potential in any matter/energy structure/flow
· The shadow on a sundial
· The pieces on a chessboard
· An open office door (meaning “you may enter”)
· The wiggle dance of a honey bee.
There is actual information when an actor creates or uses information potential
· Intentionally records information in a structure of matter: moves a chess piece, leaves an office door open.
· Intentionally puts information into an energy flow: says “come in”
· Perceives information in a matter/energy structure/flow: reads the shadow on a sundial, hears spoken words
· Uses perceived information to direct some action: enters a room when invited to do so.
1 Sender encodes information in a matter/energy structure/flow.
E.g. A speaker translates his intended meaning into words
“Send reinforcements we are going to advance”.
He translates the discrete words into continuous sound waves.
His mobile phone translates the sound waves into radio waves.
2 Transport of that matter/energy structure/flow to receivers.
E.g. Signal travels as radio waves, via intermediary receivers/transmitters.
3 Receivers decode information from a matter/energy structure/flow.
E.g. A mobile phone translates radio waves into continuous sound waves.
The hearer translates those sound waves into discrete words.
“Send three and four pence we are going to a dance”, then acts on that meaning.
In a communication stack, logical information/meaning is encoded in physical messages/signals at successively lower levels.
And in reverse, a physical signal is turned into logical information at successively higher levels of abstraction.
This table is an attempt to explain the receipt of a message using the Morse code communication stack.
Read it from bottom to top.
Logical - sentences
human reads word groups as sentences giving meaningful directions or descriptions
Logical – words
human reads character groups as words
Logical - characters
human reads impressions on the paper tape as characters
Physical - encoding
armature impresses indentations onto a paper tape
Physical - force
electromagnet pushes armature
Physical - energy
telegraph wire delivers pulses of electric current to an electromagnet
In a typical communication stack:
· the top is the level of logical human thinking;
· the bottom is the level of binary digits represented in some kind of physical matter or energy.
The physical base from which information can be abstracted might be electrical charges.
Everything above the physical medium is meaningful information abstracted by an actor of one kind or another.
Computer actors to abstract logical 1s and 0s from physical electrical charges.
At higher levels, others actors (or the same actor) can decode richer meanings from composites of 1s and 0s.
Our primary concern is with the meaning understood by human senders and receivers of information.
But every machine lower down a communication stack works at its own level of meaning.
Also, the top level of a communication stack does not have to be either human or verbal.
A machine (as in an embedded process control system) can find the meaning in a signal and act accordingly.
There are models that mimic an observable reality and data structures that encode information about a reality.
A model that mimics observable reality:
E.g. A cave wall drawing of a bull visually resembles a real bull. It has:
1. a meaning to the creating/encoding actor.
2. a meaning when viewed by an actor who interprets/decodes it.
E.g. A data structure that encodes information using a language.
The sentence “The cat sat on the mat” has:
1. a meaning to the creating/encoding actor.
2. a meaning when heard/read by an actor who interprets/decodes it.
The success of human society depends on meaning 2 matching meaning 1 well enough, often enough.
That is both why and how our social communication evolved in the first place.
Often enough for survival, meaning the encoded meaning is successfully decoded.
How do we know?
Often enough, communication receivers act in accord with communication senders’ meanings.
Building architect’s drawings are models that mimic reality.
Business architects drawings are data structures that encode reality.
The actors who create and find information
The actors may be human or computer
A postman reads the address on an envelope and delivers it
An email system finds a “To” address in a message and delivers it.
Both use a language to find information in a data structure and act on the information they find
The language used to encode/decode information in data is composed of a vocabulary and a grammar.
The same vocabulary may be expressed in spoken words, Morse code, sign language or other gestures, and digital messages and records.
People hold mental models that describe realities
(The bio-electrical-chemical form of a mental model is irrelevant here.)
These mental models are fuzzy and flexible - as they need to be
Evolution and learning make them merely accurate enough for us to direct our actions successfully.
Some animals use the vocabulary and grammar of a language to
· encode mental models into matter/energy structures/flows
· decode from matter/energy structures/flows into mental models
Encoding a mental model in a matter/energy structure/flow is a way to make the model more stable and shareable.
We attach words to discrete perceptions,
At the macro scale, reality is continuous; even solid/liquid/gas phase divisions are fuzzy-edged
But we perceive reality as chunks discrete in time and space; and we attach words to those chunks
Which works well enough for us to direct our actions.
Is a verbal proposition inherently true or false?
“The sky is dark today”
The meaning of a verbal proposition depends on the language (v&g) applied to it and the context it is used in
A proposition may be true enough to be useful to one person and not to another.
Which linguistic philosophy do we agree with?
Formalist, Non-formalist, Structuralist, Post-structuralist or Deconstructionist?
Here, natural language is no more than a side effect of biological evolution
It is inherently messy and ambiguous
It is ill-suited to perfect, unambiguous communication
It is just good enough to aid survival.
However, natural language has been refined by people in business, science and mathematics who strive to model the world more coherently and consistently.
Business systems use domain-specific languages that formalise natural languages well enough for businesses to survive.
Software systems formalise domain-specific languages further.