An
information theory for enterprise, business and software architects
This page is
published under the terms of the licence summarized in the footnote.
All free-to-read materials on the Avancier
web site are paid for out of income from Avancier’s training courses and
methods licences.
If you find the web site helpful, please
spread the word and link to avancier.website in whichever social media you use.
A system might be defined as a collection of actors that interact by the exchange of information.
What theory do we have for how to exchange information? Is there an information theory?
This paper is about the creation and use of meanings in social
and business messages and records.
Contents
How
do messages/signals differ from meanings/information?
Signal
quality and interpretation issues
The
importance of information for life
Continuous
and discrete signals
Footnote
1: Other views of “information”
Footnote
2: on the confusion between data and information
Information
Here, a piece of information
is a meaning created or found by an actor in some matter or energy that acts as
a signal.
There is information potential in any variety an actor can detect in a structure of matter or flow of energy.
There is actual information when an actor realises some information potential by recognising or describing something, or directing some action.
Information
theorists usually focus on information that is deliberately created and
communicated.
But actors can
find information in physical phenomena that occur without any intent to convey
information.
For example, a
sunflower detects sunlight, perceives the position of the sun, and turns
towards it.
Abstracting
information from physical reality
Here, a mental model is some kind of memory, against which a new perception can be matched
If two actors find the same information in a perception, then they must share mental models well enough.
Meaning: they share their readings/understandings of their mental models, rather than share their physical brain matter.
Consider two actors looking at the shadow of a person cast on a cave wall.
The shadow conveys no information or meaning until viewed by an actor who is able to interpret it as describing something in outline.
It two actors reckon the shadow describes a man’s outline, then they must share much the same pre-existing mental model of a typical man’s outline.
If both recognise the particular man whose shadow is cast, then they must share the same mental model of that particular man’s outline.
Suppose one actor reckons the shadow is of a man, and the other actor says the shadow is of a woman.
Now, the shadow has been interpreted as conveying different information, two different meanings, to the two actors.
And the implication is that they do not share mental models well enough to find the same information/meaning in the shadow.
Abstracting
information from physical communication
The process of communication often involves a series of translations
from one model to another.
·
A honey bee finds some pollen and perceives its
direction and distance from the hive.
·
It transcribes those perceptions into a private
(biochemical) model it can use later.
·
Back in the hive, it transcribes its private model
into a public (dance) model.
·
Another bee observes that dance and transcribes the
logic of it into its own private (biochemical) model.
·
Then, carrying that physical model, the second bee
flies off to find the pollen.
·
It continually matches its perception of the world
against its private model until it successfully locates the pollen.
How do we know bees
must store and share their mental models of pollen locations?
Because
we can read their dance and predict their behaviour.
Information
transmission might be characterised as a three-stage process
1 Sender encodes information in matter
or energy.
E.g. A speaker
translates his intended meaning into words: “Send reinforcements we are
going to advance”.
He translates the discrete words into continuous sound waves.
His mobile phone translates the sound waves into radio waves.
2 Transport of that matter or energy to
receivers.
E.g. The signal travels as radio waves, via intermediary receivers/transmitters.
3 Receivers decode information from
matter or energy.
E.g. A mobile phone translates radio waves into continuous sound waves.
The hearer
translates those sound waves into discrete words: “Send three and four pence we
are going to a dance”
Then acts on that meaning.
Shannon’s information theory about data
transmission
This theory is
about the limits on signal processing operations such as compression, storage
and communication.
It helps people to ensure a data structure arrives intact, with minimal data loss or distortion.
It is not about the meaning of the data.
(See footnote for 1 for discussion.)
Information versus
data
The English language is blessed and cursed with more than enough words; the Oxford English Dictionary lists more than half a million.
A mistake people often make is to assume different words mean significantly different things within one domain of knowledge.
Consider these ten words: “data”, “information”, “knowledge”, “wisdom”, “signal”, “symbol”, “description”, “representation”, “meaning” and “model”.
How many clearly distinct concepts are there? The proposal here:
Information is a meaning created or found by an actor in some matter or energy that acts as a signal.
Data is a signal
or signal element that can be created or perceived by an actor, and mapped to a
general language or ontology.
(See footnote 2 for discussion.)
On the creation and abstraction of
meaning
This theory is about the information/meaning created or found by actors.
Actors encode information in matter or energy, and decode information from matter or energy.
The physical form used to convey information may be a model that mimics reality or a data structure.
Consider a model that mimics observable reality.
An actor copies an internal mental model of a bull into the form of a drawing of a bull on a cave wall.
The drawing has meaning (1) to the creating actor and meaning (2) when viewed by an actor interprets it.
Consider a data structure, such as a spoken or written sentence.
An actor copies an internal mental model into an audible/readable sentence.
The sentence has meaning (1) to the creating actor and meaning (2) when heard/read by an actor who interprets it.
The success of
human society depends on meaning 2 matching meaning 1 well enough, often
enough.
That is both
why and how our social communication evolved in the first place.
The actors
The actors who create and find information in data can be human or computer.
A postman reads the address on an envelope and delivers it.
An email system finds a “To” address in a message and delivers it.
Both use a language to find information in a data structure and act on the information they find.
The language
A language used to encode/decode information into or out of a data is composed of a vocabulary and a grammar.
It is used to encode meaningful information in data structures.
It may be expressed in spoken words, Morse code, gestures or sign language, digital messages or records.
People hold mental models that describe realities – well enough.
They use languages to communicate mental models, or to lie about them.
General presumptions include:
Actors form,
remember and use mental models (as in the description theory paper).
Actors
communicate mental models, descriptions and directions by encoding/decoding
them into and out of messages.
The
encoding/decoding process maps elements of description and directions to a
language with a vocabulary and grammar.
Natural language is inherently messy and ambiguous, but good enough for people to survive.
Business systems use domain-specific languages that formalise natural languages well enough for businesses to survive.
Software systems formalise domain-specific languages further.
Still, communication is hindered by inaccuracies in perceptions of reality, and other limitations.
You might
think what we perceive as a discrete thing is discrete in reality
This work
presumes it can in reality be fuzzy-edged, or a slice carved out of a
continuum.
But we
perceive it as discrete, and describe it as discrete, which works well enough
for us to direct our actions.
You might
think a proposition is true or false.
This work
presumes a proposition is true or false only within a coherent and consistent
domain of knowledge.
It might be
true enough to be useful in one context and not true or useful in another.
(Others can decide if the position here matches that of a formalist, non-formalist, structuralist, post-structuralist,
or deconstructionist.
Here, language is a tool developed by biological evolution that is messy and ambiguous
Though it has been refined by people in business, science and mathematics who strive to model the world more coherently and consistently.)
Here, information is meaning
created or found by an actor in some matter or energy that acts as a signal.
There is information potential in any variety detectable in a structure of matter or flow of energy.
There is actual information when actors use some information potential to recognise or describe something, or direct some action.
|
There is information potential
in the variable |
There is actual information
when |
|
angle of the sun’s rays. |
a human reads the time from the shadow on a sundial. a sunflower perceives the position of the sun and turns to face it |
|
nerve impulses (electrical charges). |
an actor responds by removing its hand from a hot plate |
|
bending of a bi-metal strip. |
a thermostat responds by switching a heater on or off. |
|
movements of a honey bee. |
honey bees dance to communicate a location of pollen. |
|
open or closed state of an office door. |
actors share a vocabulary in which an open door means “you have permission to enter”. |
|
lengths of dots and dashes (in sound, light, braille…) |
actors use Morse code to communicate. |
|
quantity in a number. |
an actor says 20 in reply to a request for a fact (say, the speed of a bicycle in miles per hour). |
The last four examples are social systems in which senders and receivers benefit from sharing the meanings in signals.
Signal
senders encode logical meanings in physical forms; signal recipients decode
logical meanings from physical forms.
Information
is created in the act of encoding meanings in signals, and obtained by decoding
meanings from signals.
A social system depends on its members sharing information.
System actors must communicate by sending and receiving messages or signals.
All the actors in this paper are message senders or receivers (in evolutionary terms, computer actors are proxy human actors).
The system can only thrive if actors find the same meanings or information encoded in the signals they send and receive.
Actors create and find logical information in a physical signal by mapping its elements to a general pattern or language.
With no actors to look at it, a message (be it print on paper or magnetic patterns on a disc) it is just an arrangement of matter and energy.
The meaning of the message lies in the intention of the sender and the interpretation(s) made by receiver(s).
Different actors may find different meanings in one message:
·
Using
the physical medium of sound waves, a general intends to say “Send reinforcements; we’re going to advance”.
·
A
corporal interprets that physical
signal as saying “Send three and four pence; we’re going to a dance.”
When and where do those two meanings exist? Surely not in the material form of the message.
No actor can understand the material form of a message without a language, or be entirely sure how it might be interpreted by another.
Meanings arise at moments when actors create and interpret a message.
They do this by mapping elements of the message to a language known to the actor.
These three messages may help to further illustrate the difference between signals and meanings.
·
“Read
the symbols in this signal.” Since the signal quality is OK, and we
share the same language, you’ll read the meaning I intended.
·
“Read
thu simbuls in this
signal.” The signal is different (the quality is imperfect), yet the meaning
you read is surely the same.
·
“A
rose is a rose is a rose.” The signal is intended to be meaningless, yet you
may nevertheless find some meaning of your own in it.
In other
examples, ambiguities arise from the use of ambiguous words, or different
dialects of a language.
In short,
information is meanings created and found in messages or signals, by actors
using a domain-specific language.
Social systems
depend on messages – usually - conveying the same meanings to message creators
and users.
What do
meanings mean? Often, they are directions or descriptions.
Directions
may be continuous or divided into discrete instructions.
Descriptions
are usually divided into facts (tasty, tall, scary)
about things (say, food, friends and enemies) that actors perceive as
discretely identifiable.
The physical
content of a signal can be written and read with different meanings –
accidentally or deliberately.
To repeat the
example above:
A general tries
to say “Send reinforcements; we’re going to advance” using the physical medium
of sound waves.
A corporal hears
the same physical signal as “Send three and four pence; we’re going to a
dance.”
Thus, the signal
receiver obtains information, but not the information the signal sender
intended to convey.
Humans and
computer actos can use Morse code to code/encode
meanings in a physical signal.
Suppose senders
and receivers use different versions of Morse code, then they will write/read
different character strings.
At the first
level of decoding, the signal receiver can read the message as a character
string – with no apparent difficulty.
But at the next
higher level of reading, the receiver will find it the characters do not form
into words.
A poet may say
their poem means (signifies) whatever meaning each reader finds in the poem.
Some systems
thinkers take that idea to an extreme.
They say that
meaning is only found after a signal event – in the use(s) made of the signal.
This way of
thinking about communication may lead you to Luhmann's
"autopoietic social system."
There are reasons
to doubt the value of Luhmann's theory.
First, it
radically changes the meanings of “autopoiesis”,
“society” and “system” understood by others.
More importantly,
society depends on listener/readers (usually) successfully construing what
speaker/writers mean.
This means signal
senders and receivers give the same meaning to the contents of signals.
This implies the
actors must share a vocabulary and grammatical rules – be they innate or
learnt.
The physical
forms in which signals can be stored and conveyed, using matter and energy, is
not the focus here.
A dash of physics
or chemistry might help you understand how information can be stored or
conveyed.
But
information is better understood in terms of its role in biological evolution
and social systems.
It has been
said that “all life is information processing”; certainly, information is
needed for life.
Even
primitive actors need information about the state of the world they live in.
They need to
monitor and direct the state and behaviour of themselves and their environment.
An actor’s
very survival depends on it perceiving and modelling those realities well
enough.
It needs
sensors to detect the state of itself and its environment.
Those states
must be represented in energy flows sent to body parts or organs.
The importance of information for
social and business systems
The ability
of actors to describe the world evolved alongside biological evolution.
Evolution led
to actors with brains that can remember past facts about the world (e.g. where
a food item was buried).
And then to
social systems in which animate actors communicate.
“Social”
implies actors in the society cooperate, which in turn implies they
communicate, share information using a common language.
“System”
implies actors in the system follow given rules when responding to information
received.
Social systems,
information and communication existed before human actors,
However, the
range of communicable information vastly increased with the ability to describe
things in a verbal language.
Humans use a
wide variety of signals to share information that describes things, which they
need to monitor or direct.
They create
information by organising matter. E.g. by writing a letter, or painting a
picture, or placing a chess piece.
The retrieve
information from organised matter via energy flows. E.g. the light reflected
from a painting or chess board.
Social
systems range from the informal to the formal.
In
evolutionary terms, business systems evolved from social systems.
Humans
formalised social communication to standardise the transactions of government
and commerce.
In short,
information existed eons before human and computer activity systems.
Social system
members have three mechanisms for sharing the language, rules and state of a
social system.
·
Breeding
- by inheritance from parent to child (languages and rules only, not state information)
·
Communication
- by messages sent from senders to receivers
·
Recording
- by storing in a shared memory that all actors can access.
(By the way,
actors and societies may evolve as a result of copying errors in breeding,
communication and recording.)
In
evolutionary terms, all information storage must have started off as fuzzy and
imprecise.
Biological
memory mechanisms are fragile, prone to error, decay and loss.
Humans
evolved the ability to communicate logical information using physical sound
waves.
Then to store information (including the state, languages and rules of a
social system) in persistent storage materials.
In 2,500 BC
Sumerians stored their literature, laws (behavioural rules) and business
transactions (information) in libraries of clay tablets.
(Sumerians had shifted from picture-writing into
cuneiform, which means "wedge writing" in Latin.
Cuneiform was written with a wedge-shaped stylus, similar
to ones used on today's hand-held computers.
Writing was inscribed into damp clay tablets, which were
then baked until hard.)
A social
system’s state also includes the current state of inter-actor communications.
A request-reply
communication requires that signal creators and users share a language.
But moreover,
the requester must remember the question to which an answer is sought, and
associate any reply with that question.
So, social
communication requires that a requester’s brain remembers the state of an
inter-actor request-reply communication.
And in this
way, the current state of inter-actor communications is distributed between the
minds of social system members.
Information
can be extracted from a signal, even though the sender had no intent or mind to
send a message.
When you read
the time from a sundial’s shadow, you don't attribute any intent or mindfulness
to the sun.
(Though the creator of the sundial was mindfully intent on building it
to block the sun’s rays.)
However, our
focus is on social/business systems in which messages have both senders and
receivers
Some
receivers may not successfully obtain the meaning intended by the sender.
But a social system's
very survival depends on enough
receivers sharing senders’ intended meanings.
Words like
"intent" and "mind" are questionable in a general
information theory.
You might say
a human, elephant or dog "is minded" or "intends" to send a
message, within or across species.
You might
feel uncomfortable about saying the same of a bee or an ant.
You probably
see honey bees as automata without minds or intentions to do things.
It seems our
concept of mindfulness is tied up with the self-awareness we see in at least
some animal species.
And it seems
reasonable to speak of "intent" and "mind" where social
system members are self-aware.
By the way,
some regard as jellyfish as social system all to itself.
(At a brief
glance, "The Social Metabolism: A Socio-ecological Theory of Historical
Change" by Manuel González de Molina Navarro, Víctor M. Toledo looks relevant here.)
The term analogue is used to describe signals or data that vary continuously, as the hands on a clock face do.
The term digital used to describe signals that are chunked into discrete units, as you read from a clock that displays discrete numbers.
It is now common practice to convert continuous analogue signals into discrete digital signals, and vice versa.
The universe may be continuous, but animals do decode apparently continuous signals into discrete facts.
Honey bees watch a dance and decode it as showing the distance and direction of a pollen source.
You see a rainbow as distinct bands of colour; you hear a continuous stream of sound as composed of discrete words.
You perceive a stream of light as revealing your lunch to be collection of discrete food items sitting on a discrete plate.
Discriminating between these things is useful; it helps you to spear discrete food items and eat them.
“At a conscious level, humans tend to interpret the world as discrete things.
We pattern match, then label (not necessarily verbal), then move our limited attention on, which is part of a survival strategy, to categorise things as safe or dangerous.
'Chunking' may better describe the operation (https://en.wikipedia.org/wiki/Chunking_%28psychology%29 ).
Most descriptions of chunking don't do justice to the profound way in which it avoids information overload.” Ron Segal
How our brains and computers work at the neuron or electronic level is not important here.
At the level of consciousness and social communication we chunk our perceptions of the universe into discrete entities
Perceiving and describing discrete entities (you, me, your lunch, or a bicycle ride) is how we make sense of the world.
And we describe those entities in terms of discrete facts (you are tall, I am old, lunch is tasty, and this bicycle is speeding at 20 mph).
These descriptions are models or coded representations of the states of things we perceive as discrete entities.
If a signal can be perceived as composed of elements, then each element may be used to convey unitary fact (e.g. surname).
You can create or find composite facts in a structure of elements by matching it to the rules of a grammatical structure.
E.g. full name = forename + list of middle names + surname.
The entry to the information age was associated with the digital revolution that happened as business communications were computerised.
Most communications are now sent and received by computers in a binary digital form.
This means that information is encoded in signals/data flows using the binary number system as series of discrete digits 0 and 1.
The binary digits may be represented in physical signals as the presence or absence of physical things.
Or as the contrasting values of a physical quantity such as voltage or magnetic polarization.
How to encode human-level verbally-expressed information in binary digits, and decode information from binary digits?
This is usually done step-by-step, in a series of transformations, down and up a communication stack.
In a
communication stack, logical information/meaning is encoded in physical
messages/signals at successively lower levels.
And in reverse, a
physical signal is turned into logical information at successively higher
levels of abstraction.
Here is a Morse
code communication stack:
·
Top
level - logical (human reads groups of words as sentences giving directions or
descriptions)
·
Next to
top level - logical (human reads groups of characters as words)
·
Upper
middle level - logical (human reads groups of impressions on paper tape as
characters)
·
Lower
middle level – physical transformation (armature impresses indentations on
paper tape)
·
Next
to bottom level – physical motion (electromagnet pushes armature)
·
Bottom
level – physical energy (telegraph wire delivers pulses of electric current to
an electromagnet).
Typically,
the top is the level of human thinking; the bottom is the level of binary
digits represented in some kind of physical matter or energy.
And our
primary concern is with the meaning understood by human senders and receivers
of information.
But every machine lower down a
communication stack works at its own level of meaning.
The top level of a communication stack does not have to be either human or verbal.
It could be a machine, as in an embedded process control system; or even a non-human animal.
The physical base
from which information can be abstracted might be electrical charges.
Everything above
the physical medium is meaningful information abstracted by an actor.
Humans have
devised computer actors to abstract logical 1s and 0s from physical electrical
charges.
At higher levels,
others actors (or the same actor) can decode richer meanings from composites of
1s and 0s.
The table below shows three levels of abstraction away from the behaviour of entities in the material world.
At each level, actors create and use data that conveys information from or to actors at the level below.
|
At this level of abstraction |
these actors |
work to |
and so convey information
about |
|
3 meta system |
methodologists |
define notations for modelling data and process structures |
The language systems designers use at the level below |
|
2 system description |
system designers |
define data types and process flows that using the notations above |
The language systems actors use at the level below |
|
1 operational (run-time) system |
system actors |
read and write messages containing data items conformant to data types above |
The attributes and behaviour of real entities at the level below |
|
0 material (run-time) world |
external entities |
do stuff that is describable in messages above |
|
This paper says
· Information is a meaning created or found by an actor in some matter or energy that acts as a signal.
·
Data
is a signal or signal element that can be created or perceived by an actor, and
mapped to a general language or ontology.
There is more at http://avancier.website on:
·
the
evolution of description and communication, read A description theory
·
the formalisation of types used to describe things, read A
type theory
·
knowledge, information and data as a hierarchical
structure, read Data, Information and Knowledge.
·
other information theories.
This work may be seen as philosophy based on a scientific view of the world; especially on biological evolution and the science of digital information systems.
For us, information appears in the act of informing – in the communication of knowledge from senders and to receivers.
The elements of information are descriptive or directive qualities (including quantities).
Scientists have developed a broader view of information.
Information Philosophy (I-Phi) <http://www.informationphilosopher.com> is a philosophical method grounded in science.
Its definition of information is any quantity that can be understood mathematically and physically.
That includes information in the structure of any physical object, like a snow crystal, a star, a biological entity, genetic code, cell structure etc.
Any physical structure is thus an "information structure," from which information can be abstracted as meaningful knowledge.
However, a
message that is not actually meaningful to any sending or receiving actor is not
a useful concept in this work.
Social, business and software systems all depend on communication.
Our focus is on information created and found in messages sent or stored with an intended purpose.
Business
systems integration (a focus of enterprise architects) depends on information
flowing in messages and stored in memories.
In today’s “information age”, businesses look to capitalise on information they glean from messages that have been stored in huge quantities.
Our information theory takes a "survival of the fittest" view of how information processing evolved in social systems.
It views business systems as formalised social systems, in which actors communicate to cooperate in activities.
Claude Shannon developed “information theory” about the limits on signal
processing operations such as compression, storage and communication.
Shannon wrote "The
fundamental problem of communication is that of reproducing at one point either
exactly or approximately a message at another point."
His concern was maintaining signal quality in communication; his theory is
about the physical content of a message.
Shannon wrote:
"Frequently the messages have meaning"
But as long ago as
1956, Boulding observed that Shannon did not address the meaning of
communication in social or business systems.
For us, the
fundamental problem of communication is that of knowing what a message means,
at a logical level.
A system can only thrive if actors who receive messages/signals find the same meanings/information that senders encoded in those messages/signals.
And this can only happen if senders and receivers both map elements in messages/signals to elements in a shared language.
To create and
find logical information in messages and memories, actors must map discrete
elements to elements types in a shared language.
This morning
(28/10/2015), I received the email below from a recruitment consultant.
Enterprise
Information Architect: London - £90,000 per annum plus benefits.
My client, a large and iconic London brand is now looking for an innovative and
experienced Enterprise Information Architect.
He/she will focus on developing and leading the exploitation of data and information architecture in conjunction with business area leads, business partners and project delivery teams.
How does the
recruiter expect us to differentiate information from data?
Consider a particular telephone number, or customer address. Is that a data item or an information item?
Consider a message containing customer name, address and telephone number. Is that a data message or an information message?
Does it make any difference who creates the item or message - a human or a computer?
Consider the generalised structures for items and messages – simple data types and complex data types or data structures.
Consider the structure for a telephone number, or an invoice. Is that a data type or an information type?
Does it make any difference whether the type is defined by an international standards body or by a programmer?
Consider the higher level meanings of messages that are created in message by senders and obtained from by receivers.
Does it make any difference who creates the item or message - a human or a computer?
Business
system design usually addresses the plural (sets of things) rather than the
singular (individual thing).
Business processes follow the logical
paths defined in general process types.
Processes monitor and direct business
entities and events that instantiate defined types.
Messages
convey meaningful information about business entities and events by giving
values to their typical qualities.
A business needs senders to encode
meanings in messages, and receivers decode the same meanings from those
messages.
To this end,
the mapping of data types to generalised meanings must be agreed and defined
before messages are sent.
Thus,
business systems formalise what is fuzzy and ambiguous in purely social
communication.
Social system members may share their own language for describing the entities
and events of interest in their domain.
The domain-specific language used in a digitised business system is commonly
defined in what is known as a ontology
A data model is an ontology composed of generalised data types (e.g. Customer, City) instantiated as data items (e.g. Jack Jones, London) in particular messages and records.
This language stands apart from, is independent of, particular actors, particular signals, and particular information.
The proposal here is: Data is a signal or signal element
that can be created or perceived by an actor, and mapped to a general language
or ontology.
Honey bees use the language of dance to signal the locations of pollen sources; situation-specific data is found in particular dance movements.
The bees’ language is generalised; it is independent of particular bees, particular dances and particular pollen sources.
So, why are
the terms “data” (signal) and “information” (meaning) so confused in enterprise
architecture?
Because
systems are built on the assumption that physical data carries the same logical
information to all senders and receivers.
It is assumed
that the meaning actor A gives to data on posting it
will be read with the same meaning by actor B, in a different time and place.
And that data stores record the current state of business entities (and/or the history of business events) in a shared memory all understand the same way.
In practice, the
term data and information are used interchangeably and used so loosely.
You generally
can’t assume either term has any one specific meaning.
Some equate data with information; some equate data with
signals.
(A sister paper called Data, Information and Knowledge addresses the wide variety of ways people distinguish the terms.)
Footnote: Creative Commons
Attribution-No Derivative Works Licence 2.0 1/5/2017
Attribution: You may copy,
distribute and display this copyrighted work only if you clearly credit
“Avancier Limited: http://avancier.website” before the start and include this
footnote at the end.
No Derivative Works: You may copy,
distribute, display only complete and verbatim copies of this page, not
derivative works based upon it.
For more information about the licence, see http://creativecommons.org