A
communication theory
Information exchange in social and software systems
Copyright 2016 Graham Berrisford. One of about 300 articles at
http://avancier.website. Last updated 10/01/2021 15:11
This is a supplement to the chapter on information, description and types here https://lnkd.in/dQNhNbd
It explores the same concepts in more detail.
Contents
An
information and communication theory
The ubiquity
of coding in memories and messages
How memories
and messages employ coding
Information,
data and knowledge
Claude Shannon’s information theory is about
the journey of a signal or message from sender to receiver.
Shannon wrote: "The
fundamental problem of communication is that of reproducing at one point either
exactly or approximately a message at another point."
The
theory addresses limits on signal processing operations such as compression,
storage and communication.
Shannon
also wrote: "Frequently the messages have meaning".
The signal/meaning distinction may be drawn
thus.
·
A signal is matter/energy organized such that
actors can detect structures/variations in it.
·
Signals appear in physical messages and memory
structures.
·
Meanings are created or found by actors in
signals.
However, as long ago as 1956, Boulding
observed Shannon did not address the meaning of messages.
Traditional information theory does not
address what is expressed (descriptions, directions or decisions) in writing or
reading a signal.
So, how to ensure a receiver extracts the same meaning from a signal that the sender intended to convey?
Some approach meaning from the viewpoint of
a linguist – assuming meanings are expressed in verbal ways.
But meanings are found elsewhere - in
non-verbal signs and in the after effects of behaviors.
In fact, meanings arise when and wherever an actor finds any usable information in any structure or behaviour.
This article is not about Shannon’s information theory!
And we don’t begin here by thinking of
information like a linguist or software engineer might do.
Millennia before verbal languages and
computing, animals encoded input signals into memories.
Then decoded memories into the stream of
consciousness when determining responses to events.
And social animals encoded and decoded messages
(e.g. alarm calls) without needing to learn a
language.
Try to imagine how you’d mimic these
incredibly effective processes in software.
Inevitably, you’d find yourself drawn down
paths you understand.
You’d think of data structures - stored in
databases, and transmitted in data flows.
You’d presume the need for formally defined
languages, symbols, syntax and semantics.
But surely, a truly general information
theory cannot start from or depend on how software works?
It ought to start from the evolution of
information created and used by animals
Many animals can not only remember information internally in memories, but also share information messages.
Theorists have defined “information” in a
variety of ways.
In "The Information: A History, a
Theory, a Flood" James Gleick presents a pot pourri of views about information.
The general conclusion is that information is
“anything that could be discerned”.
That is, any structure/variation in
matter/energy.
Similarly, a philosopher has written that
information is “any quantity that can be understood mathematically and
physically.”
That is, any snowflake, star, organism and genetic code is an "information structure".
Here, the interest is in
structures that are intentionally created to represent information.
Information:
a structure or behavior that represents something or phenomenon.
More
scientifically: “A carries information about B if the state of A is correlated
with the state of B.”
Animals
can hold information in memory and communicate it in messages.
In
our triangle, the information at the apex embraces both memories and messages.
|
Intelligent life |
|
Memories
and messages <create and use> <represents> Animals <observe and envisage> Phenomena |
Memories
and messages: holders of information.
In
biology, internal memories and external messages are of different kinds.
Memories
are neural patterns; messages take the form of sounds, smells and gestures.
In
software, the distinction between memories and messages is blurred.
Communication:
the exchange of information between senders and receivers.
Actors
may exchange information directly by sending/receiving messages.
Or
else, indirectly by writing/reading information stored in some memory both can
access.
They
respond to information in messages, often in a way determined by information in
memory.
|
Social communication |
|
Information
in messages <create and use> <represents> Communicators <observe and envisage> Phenomena |
Remember:
“A carries information about B if the state of A is correlated with the state
of B.”
A
message represents a phenomenon if the structure of the message can be
correlated with some features of the phenomenon.
Ashby
was interested in how information is stored and communicated.
In
the processes of creating and using information, rather than its
physical form.
These
processes encode and decode information into and out of physical forms.
Ashby
observed that in the creation and use of information “coding is ubiquitous”.
To
create information is to write or encode a model that represents
something.
To use
information is to read or decode a model, and use it for some purpose.
Nobody
understands how conscious knowledge is encoded in and decoded from our
biochemistry.
But
evidently, those processes exist.
Suppose
you ask me to look at the moon.
·
Your thought is encoded in neural impulses, then
vocal chord movements, then
sound waves.
·
Then from sound waves to my ear drum movements,
to neural impulses.
·
Then to conscious thought in my mind which can
be encoded in my memory.
During
the communication, the idea is expressed or encoded in various forms, public
and private.
Ashby presented this longer example.
“Let us consider, in some detail, the comparatively simple
sequence of events that occurs when a “Gale warning” is broad-cast.
It starts as some patterned process in the nerve cells of
the meteorologist, and then becomes
· a pattern of
muscle-movements as she writes or types it, thereby making it
· a pattern of ink marks
on paper. From here it becomes
· a pattern of light and
dark on the announcer’s retina, then
· a pattern of retinal
excitation, then
· a pattern of nerve
impulses in the optic nerve, and so on through her nervous system. It emerges,
while she is reading the warning, as
· a pattern of lip and
tongue movements, and then travels as
· a pattern of waves in
the air. Reaching the microphone it becomes
· a pattern of variations
of electrical potential, and then goes through further changes as it is amplified,
modulated, and broadcast. Now it is
· a pattern of waves in
the ether, and next
· a pattern in the
receiving set. Back again to
· a pattern of waves in
the air, it then becomes
· a pattern of vibrations
traversing the listener’s ear-drums, ossicles, cochlea, and then becomes
· a pattern of
nerve-impulses moving up the auditory nerve.
… this very brief account mentions no less than sixteen
major transformations
through all of which something [the intention] has been
preserved,
though the superficial appearances have changed almost out
of recognition.” (1956, 8/2)
To send and receive the gale warning message involves a
succession of coding and decoding steps.
The message is passed down and up a communication stack.
After
receiving a message, a listener can verify the accuracy of the warning by
watching the weather.
Representing
a structure
Suppose
A = a map and B = a territory.
And
there are correspondences between the structures of the two entities.
Then
the map carries some information about the territory.
And
the territory carries some information about the map.
Representing
a behavior
Suppose
A = a musical score and B = a musical performance.
And
there are correspondences between the structure of A and the behavior of
B.
Then
the musical score carries information about the process of the performance.
And
the performance carries information about the structure of the musical score.
Remembering
a thing
Suppose
·
A = the state of something in the environment.
·
M = the state of a message conveyed by eyesight
to your brain
·
B = the state of a memory in your brain.
To register
and remember the existence of the thing
·
A is encoded into M
·
M is decoded and encoded into B.
Ultimately,
the meaning of a memory is not found in the memory alone.
It
is found in the process by which B is decoded by retrieval and used.
This
last is the information of most interest to psychology.
Communicating
an idea
Suppose
·
A = the state of a message sender’s brain.
·
M = the state of the message.
·
B = the state of a message receiver’s brain.
To
communicate an idea
·
A is encoded in M
·
M is decoded and encoded into B
The
aim of human-to-human communication is not to draw a biological correspondence
between A and B.
How
far the structures of two brains can be correlated at the biological level is
unclear.
By
contrast, correspondence at the sociological level can be observed and
verified.
In natural language: the terms data,
information and knowledge are often used interchangeably.
To facilitate discussion of social systems,
an informal classification is helpful.
Several WKID
hierarchies have
been proposed and criticised.
The version below seems the best fit to a
system of communicating actors.
|
WKID |
meaning |
|
Wisdom |
the ability to apply knowledge in
new situations. |
|
Knowledge |
information that
is accurate enough to be useful. |
|
Information |
meaning created/encoded or
found/decoded in data by an
actor. |
|
Data |
a structure of matter/energy in which information
has been created/encoded or found/decoded |
Any physical structure of matter or energy
can be used as a data structure or signal.
It becomes a data structure when the
structure encoded to convey information/meaning.
And when it is decoded as conveying
information/meaning.
Data/signal
Any
structure or motion that is variable - has a variety of values – can be used to
store or convey information.
E.g. You
may use
·
The biochemical structures your brain.
·
The shadow on a sundial - to represent the time
of day
·
The state of your office door (open or closed) -
to tell people whether you are open to visitors.
·
Dance movements - to express emotions.
Here
“structure” embraces both data structures and process structures like dance
movements
And
marvellously, humans can form countless structures in the form of words, with
almost no physical effort.
Information/meaning
There is no information or meaning
in a structure on its own.
Data
creators must perform processes to encode/create meanings in structures.
And
data users must perform processes to decode/find meanings in structures.
This
“information” of interest to sociology only exists in those processes.
In
the intentions of data creators and the interpretations of data users.
Why
draw a data/information distinction at all?
This
data/information distinction is a subtle one.
In
business systems, the terms are usually interchangeable.
It
is taken for granted that receivers decode meanings that senders intended to
encode.
Because
messages are transmitted perfectly (where Shannon comes in).
And to write and read messages, senders and receivers use the same language.
The
distinction matters little where a signal/message conveys the same meaning to
all receivers.
A
communication theory has to deal with exceptions
Where
the data in one signal/message conveys different information to different receivers/readers.
And
where where the information potential in a structure or motion can yield legitimately
result in different meanings.
For
example, the movement of the sun across the sky has information potential.
It becomes
actual information when used by a sunflower to turn its face, or a sundial
reader to tell the time.
All communication utilises a structure
The medium for information storage or communication is a matter/energy structure of some kind.
To communicate, animals use sound waves (calls), smells, gestures, etc.
Humans use sound waves, written text, flags, etc.
Computers use electronic signals, radio waves, etc.
Every structure has information potential
There are infinite structures in the matter/energy of the universe.
Some equate structure with information.
Here, we say a structure has information
potential to actors.
There is actual information when actors use some information potential to create or obtain a meaning.
|
There is information potential
in the variable |
There is actual information
when |
|
angle of the sun’s rays |
a human reads the time from the shadow on a sundial. a sunflower perceives the position of the sun and turns to face it |
|
nerve impulses (electrical charges) |
an actor responds by removing its hand from a hot plate |
|
bending of a bi-metal strip |
a thermostat responds by switching a heater on or off. |
|
movements of a honey bee |
honey bees dance to communicate a location of pollen. |
|
open or closed state of an office door |
actors share a vocabulary in which an open door means “you have permission to enter”. |
|
lengths of dots and dashes (in sound, light, braille…) |
actors use Morse code to communicate. |
|
quantity in a number |
an actor says 20 in reply to a request for a fact (say, the speed of a bicycle in miles per hour). |
Information
is meaningful to its sender and/or receiver
Senders encode meanings in structures, and receivers decode meanings from them.
The meanings include descriptions, directions, decisions and requests for them.
Descriptions are usually divided into facts (tasty, tall, scary) about things (say, food, friends and enemies) that actors perceive as discretely identifiable.
Information has at least one sender and/or
receiver
A sender (a voice crying in the wilderness) may create information in a structure that no receiver inspects.
A receiver may find some information in a structure that was not intentionally sent.
E.g. The sun radiates a flow of light towards a rotating earth.
A sunflower finds a direction to turn its face to optimise its energy consumption.
Different
actors can find different information in the same structure
E.g. The sun radiates a flow of light towards a rotating earth.
A sunflower finds a direction to turn its face to optimise its energy consumption.
One man reads the shadow on a sundial as describing the hour of the day.
Another concludes that the sun rotates around the earth; another that the earth spins on its axis.
E.g. the structure in a DNA molecule may be decoded by a biological cell as instructions for making proteins.
And decoded by a human reader of the genome as carrying a gene for some life-shortening condition.
Neither actor can read and act on the structure as the other does.
To communicate requires sharing a structure
and a language
First, the structure of a message must be preserved (a concern of Shannon’s theory).
Second, creators and users must share a language for encoding and decoding that structure.
Two things can go wrong.
First, the structure is distorted between sender and receiver.
E.g. Speaker says: “Send reinforcements we are going to advance.”
Listener hears: “Send three and four pence we are going to a dance.”
The intended signal is distorted at some point between sender and receiver.
Shannon’s information theory is about preserving the integrity of a structure.
Second, creators and users use a different a language to encode and decode a structure.
Or the ambiguity of natural language disables communication.
E.g. Speaker says: “He fed her cat food.”
Listener 1 hears: He fed her cat – food (He fed a woman’s cat some food).
Listener 2 hears: He fed her - cat food (He fed a woman some food that was intended for cats).
Listener 3 hears: He fed - her cat foods (He somehow fed the cat food that a woman owned).
Information is a subjective view of a
structure
The information in a structure depends on senders and/or receivers and the languages they use.
E.g. I leave my office door open.
Case 1: I do it deliberately, to signal that I am open to visitors; you read the door as saying I am open to visitors, and enter my office.
Case 2: I do it by accident, but am open to visitors anyway; you misread the door as saying I am open to visitors, and enter my office.
Case 3: I do it by accident, but am not open to visitors; you misread the door as saying I am open to visitors, and enter my office.
Any
meaning created or found in a message or memory structure is information to
that actor
An actor can change their mind about the information found in a message.
E.g. I say the swimming pool is warm; you hear and act on that information by diving in.
I turns out the swimming pool is cold, and you now recall the information as a lie.
What a sender considers true, a receiver may consider false, and vice versa.
Knowledge is information that is
true enough to be useful.
The accuracy or truth of information is a matter of degree.
Knowledge is information that is true enough to be useful (e.g.
Newton’s laws of motion).
Sometimes what we say can be tested by measurement of meaning against reality.
But all measurement has a degree of accuracy, and even Newton’s laws of motion are approximations.
What sets human society apart from animals?
First, the use of words (and graphical representations of words) to remember and communicate information.
Second, the ability to translation between so many different kinds of description:
· between internal mental models and external descriptions
· between any two kinds of external description
· between descriptions usable by humans and by machines we make.
Business systems evolved over millennia from informal social systems.
They formalising actors’ roles and the activities expected of them.
And formalise messages exchanged between actors playing different roles.
|
A
simple business system |
|
|
Customer |
Supplier |
|
Place
order |
Send
invoice |
|
Send
payment |
Send receipt |
This article looks at how communication works.
Although this article emphasises the importance of language to communication, language itself is the subject of the next article.
Communication
by imitation
People do communicate by making physical symbols that mimic the things described.
· Caveman painted cave walls with images of animals they hunted.
· The Bayeux tapestry depicts events leading to the Norman conquest of England
· Engineers build model airplanes that mimic the shapes of real airplanes.
· Building architects draw plans that visibly resemble physical buildings.
· Mime artists use gestures to outline the forms of things.
· Cartographers draw maps that mimic features on the surface of the earth.
However, the interest here is in communication acts that encode information more abstractly than by imitation.
The encoding and decoding processes use a language.
A language has at least a vocabulary of abstract symbols, and perhaps also a grammar for logically organising those symbols to express information.
Communication via a transient message
Any matter or energy flow can be used by one actor to send information to another.
Evolution gave us humans a unique and dramatically well-developed communication tool.
We can create and use an infinite variety of sounds to symbolise things and types of things.
We translate internal mental models into and out of external oral descriptions.
We give voice to and hear verbal messages ranging from short and simple to long and complex.
Our messages contain descriptions, decisions and directions
Oral communication was a huge step forward for mankind, and is essential to most peoples’ lives today.
Communication
via a persistent shared memory structure
We have a second huge advantage when it comes to sharing mental models.
We have shared memory spaces that far exceed those other animals can use - in scope, complexity and value.
We can record oral descriptions, decisions and directions using that triumph of human invention - the written record.
Thus, we can translate internal mental models into documented models for posterity, for agreement and for testing.
Written communication is so important to modern society that schools prioritise the teaching of reading and writing over other subjects.
However, any structure accessible by communicating actors has the potential to be used as a shared memory space.
Consider the practice of storing meaningful information by setting or changing the status of a door.
· Door closed - means you must not bother me.
· Door open - means you are welcome to come in.
The door position is persistent data, but on its own, it is not meaningful information.
It is only meaningful to actors who know the code.
Meaningful information appears in the processes of opening/closing the door and looking at it.
If parties inside and outside the office do not use the same code, then miscommunication may occur.
All social systems depend on actors sharing information/meanings.
You might think of information in any of these ways:
· data at the point of creation or use by a sending or receiving actor.
· meaning attributed to a signal or data by the intention of its creator/sender or the perception of a perceiver/receiver/consumer.
· a process of intentionally encoding meanings in data, or decoding meanings from data.
The initial presumptions are these:
· Data is matter/energy organized such that actors can detect structures/variations in it.
· Data is found in physical messages and memory structures.
· Information is meanings created or found by actors in data
· Senders encode information in data; receivers decode information from data.
A communication should successfully exchange information provided that:
· The sender uses language X to encode their intended meaning in data in a message or shared memory structure
· The data remains unchanged until it is received or found by a receiver
· The receiver uses language X to decode the meaning of that data.
A point-to-point communication process runs like this.
1. A sender needs to send some logical information (description, decision, direction) to a receiver.
2. The sender encodes that logical information in a physical form using a chosen data format
3. The data structure either travels in a message, or is stored in a shared memory structure for inspection.
4. The receiver either receives the message or finds the shared memory structure.
5. The receiver decodes the information from the data in the message or shared memory structure.
6. The receiver acts in response to the information, as they determine.
The process applies to all sending and receiving actors – be they human or computer.
· A postman finds meaning in the address on an envelope, and uses that information to put the letter in the right letter box.
· An email server finds meaning in the TO line of an email, and uses that information to send the email to the right email inbox.
This table lists how human couriers and digital mail servers read address data, interpret it, then act on that information.
|
The general communication process |
Information use by a human actor |
Information use by a computer actor |
|
1. A sender needs to send some logical information. |
You need to give a destination address to a courier |
You need to give a destination address to a mail server |
|
2. The sender encodes that logical information |
You hand write the address on an envelope |
You type a “To…” address in an email header |
|
3. The data travels in a message or is stored in a memory structure. |
The address is stored on the envelope for inspection |
The address travels in a message to a mail server |
|
4. The receiver receives the message or finds the memory structure. |
The courier picks up the envelope |
The mail server receives the message |
|
5. The receiver decodes the information from the data |
The courier interprets the address as a location in reality |
The mail server interprets the address as a data store location |
|
6. The receiver acts in response to the information |
The courier delivers the envelope to that location |
The mail server places the email in that data store location |
The simple theory above is a little naïve.
To begin with, there are countless data formats: pictures, sound, print, morse code, binary digits.
Actors can translate data between formats (though the verbal description of a painting might be huge, difficult to read and unsatisfying).
Moreover, much communication involves a recursive data/information stack.
Communication stacks
Formats can be arranged in a communication stack such that:
· the bottom level format is more directly and easily converted to/from physical matter/energy and
· every higher level format is more directly and easily encoded/decoded by senders/receivers at that level.
Actors can translate from higher formats to lower formats and vice-versa.
Here, information is any meaning created or found by an actor in a matter, energy or data structure.
In a communication stack, actors at level N encode/decode information useful to them, into/from data/signals at level N-1.
(Perhaps, inside the human brain there is a stack from chemistry to consciousness – that being the thread of control in the top level process?)
Communication networks
What if senders and receivers are not in sight or speaking distance, or cannot access the same shared memory structure?
Data can be forwarded across the nodes of a network, by intermediate actors with no interest in its meaning.
Communication through a human network – by
sound waves
Data can be passed by word of mouth - alternately encoded in speech and in human memory.
Each person translates words they hear into memory, then translates back from their memory into words they speak.
This example dates (I believe) from the first world war.
|
The communication process |
From intention to response |
|
A general needs to send some information to a brigadier |
Intends to send a direction, with an explanation |
|
The general encodes that information in spoken words |
“Send reinforcements we’re going to advance” |
|
The message passes by word of mouth (encoded briefly in each soldier’s memory) |
|
|
The brigadier hears the message and decodes information from it |
“Send three and four pence we’re going to a dance” |
|
The brigadier acts in response to the information |
Sends money by return post |
Here, the communication failed to convey the intended meaning.
Along the route, data was misheard, misremembered or misspoken.
But even if the data arrives intact, the receiver may decode it wrongly, as in an “exception” case below.
Communication through a computer network –
by physical cable or radio waves
Actors at the top level of the stack communicate with each other.
Actors in the middle levels ensure communications arrive intact at intended receiver(s).
Actors in the “logical” or “data link” level translate logical 0s and 1s into and out of physical matter/energy.
At the bottom “physical” level there are no determined actors, only patterns in physical matter/energy.
Suppose data is stored and transmitted without loss or distortion, so a signal arriving at a receiver exactly matches the signal sent.
The normal case is that the receiver extracts the meaning intended by the sender and responds appropriately.
However, a communication theory has to address exceptions to that case.
Language mismatches
When we define the meaning of a word, we define the meaning we expect communicating parties to share.
But the word doesn't have that meaning on its own; it only has that meaning to parties who agree our definition.
E.g. To a traffic light designer, the color amber means “stop”.
Many drivers interpret the color amber to mean “accelerate”.
E.g. Ask for “a scotch” in a bar in the south of England; you’ll be given Scottish whisky.
Do the same in the north east; and you may be surprised to be given a pint of scotch beer.
Your data transmission was perfect, but your message has been decoded using a different vocabulary.
How can we minimise loss or distortion of the meaning in a message?
If sender and receiver use two different words with one meaning, then we can insert an intermediary (broker) to translate the first word into the second.
If sender and receiver use one word with two meanings, the sender can send the word’s definition in the message alongside the word, asking the receiver to use that definition.
Request-response mismatches
Ask for directions to the nearest hotel, you may be directed the wrong way
The data transmission is perfect, but the response is contrary to your wishes.
A system designer can determine what a mechanical receiver does.
But a human receiver may choose the language they use to decode data, and how to respond.
This table shows three possible cases.
|
The receiver decodes the data |
The receiver responds |
System designer’s view |
|
|
Case 1 |
using the sender’s language |
in accord with the sender’s intentions |
ideal - what a system is designed to do |
|
Case 2 |
using the sender’s language |
contrary to the sender’s intentions (do something else or nothing) |
accommodate - design exception handling processes |
|
Case 3 |
using a different language or dialect |
unpredictably |
prevent - change the language or introduce translators |
Other kinds of exception arise in loose, ambiguous, subtle, natural language communication between humans.
Senders are presumptuous, lazy, incompetent or emotional when encoding their meanings in messages sent.
Receivers are presumptuous, lazy, incompetent or emotional when decoding what the sender meant or “really meant”.
Of course, misunderstandings arise, and have to be resolved by additional to and fro between human actors.
But managing and working around human frailties is beyond the scope of this work.
The communication theory here addresses how actors exchange meaningful information, and differentiates information from data.
It proposes that all models of reality (be they mental, spoken, written and digital) are information encoded in physical data forms.
A model of one kind - in a message or a memory - can be translated into a model or data form of a different kind..
All
free-to-read materials on the http://avancier,web site are paid for out of
income from Avancier’s training courses and methods
licences.
If you find
them helpful, please spread the word and link to the site in whichever social
media you use.