50 years of digitisation and EA
Copyright Graham Berrisford. One of more than 300 papers at http://avancier.website. Last updated 15/06/2019 12:07
This paper is distilled version of EA history over the last 50 years.
It concludes with a characterisation of the digital enterprise.
Click here for the references that are referred to below by date.
Read Enterprise and solution architecture for a view of mainstream EA today.
Enterprise Architecture (EA) is a product of the Information Age.
People in social systems have always exchanged messages and remembered information gathered from messages.
As business systems evolved, people formalised the messages and memories of social systems.
They formalised (for example) how they exchanged order, invoice and payment data
Business roles and processes have always required technologies for information storage and communication.
People started using clay tablets, then papyrus and vellum, and then paper.
They created and moved business data using pens, typewriters and snail-mail postal organisations.
They recorded data for future reference in filing cabinets and card indexes.
Since dawn of the Information Age c1970, information technologies have increasingly been “digital”.
Digital information technologies have supported and enabled an ever wider range of business activities.
They increased the ability of a business to capture, store, process and analyse business data.
They strengthened the ability of a business to monitor and direct activities - and inform managers about them.
“Technology has had a prominent role within the Banking landscape for at least the last 50 years.” (2017 reference)
Today, technologies enable a business to exchange information with customers using social media, and with machines using the Internet of Things.
The Information Age brought new opportunities, but also new difficulties.
Solution architects worked to support/enable, speed/scale up, optimise/extend particular business systems.
They acted independently of each other – under the influence of competing information technology vendors.
The result was an estate of systems that were not standardised, not integrated and did not share common services.
Some systems stored different version of the same data, some did not interoperate, and many were tied to particular technologies.
This led to problems in coordinating the business roles and processes that relied on these information systems.
For example, sales and billing systems, or social and health care systems, might be inconsistent and uncoordinated.
By the 1980s, the cost and quality issues caused by “silo systems” led many to call for higher-level “enterprise architecture”.
Enterprise architects don’t design buildings or other concrete things.
Their concern is processes that create and use data that represents things.
The data represents entities and events of importance to the business.
Enterprise architects don’t recruit, motivate and manage individual human actors.
Their concern is the roles those actors play in creating and using data.
The focus is on roles and processes that can – to some extent - be digitised using information technology.
Business directors are responsible for business planning.
They respond to business drivers by declaring strategic directions and top-level goals/objectives.
Business planning may involve predicting demand.
It addresses changes that have been envisioned to one or more of an enterprises’
· Constitution: mergers, acquisitions and divestments, opening/closing outlets
· Market: industry domain/sector/segment, customers and suppliers
· Products: products and services, sales and service channels, prices
· Relationships: partners, in-sourcing and out-sourcing of operations
· Resources: people, wages, machines, locations/buildings and other physical asset types.
· Management structure: sacking or appointing CxOs and restructuring the organisation.
Business system planning is concerned with regular business operations that create and use business data.
It is given direction by business planning, and also informs business planning.
At the same time, it aims to constrain the wastefulness and issues caused by point solution proliferation.
1970s: Duane Walkers Business System Planning (BSP) method
The origins of EA can be traced back to IBM’s BSP, which Duane Walker reputedly started c1970.
Walker introduced techniques for cross-organisational analysis of business processes and data (more on this in below).
Something of these ideas can be seen in John Zachman’s IS Architecture Framework and James Martin’s Information Engineering
1982: Zachman mentioned enterprise-level architecture in his paper proposing an IS Architecture Framework.
The first two columns of Zachman’s matrix for Information System Architecture documentation were inherited from BSP.
He admitted his debt to “Dewey” Walker in an interview with Roger Sessions in “Perspectives of the International Association of Software Architects” in 2007.
He did not copy BSP, he transformed it incrementally during the 1980s.
He said business information systems analysis should be raised to the “Enterprise-level” (Zachman, 1982 reference).
And later, “EA is concerned with “system implementations, extending in scope and complexity to encompass an entire enterprise.”
1980s: Information Engineering
At least once during meetings in the 1980s, Clive Finklestein suggest Zachman use the term EA, but he didn’t do it for 10 years.
However I think James Martin did speak of EA, in connection with his version of Information Engineering.
It may be seen a descendant of BSP and was more widely influential than Zachman’s framework.
Some programming technologies were based on Information Engineering principles.
1986: The PRISM report on the primary architecture domains was published.
The report divide architecture into four domains: business, data, application and technology, which must be integrated. (PRISM report, 1986 reference).
1989: Following the 5th Information Management Directions workshop, the NIST EA model was published.
The NIST EA model put a focus is on considering business and information before technology (NIST EA model, 1989 reference).
It was recommended in US government as:
“a management tool that illustrates the interrelationship of enterprise business, information, and technology environments.
The five-layered model allows for organizing, planning, and building an integrated set of information and information technology architectures. (FEAF v1.1, 1999)
It appears that the NIST EA model was the first publication to use the term “EA” formally.
But it would be misleading to suggest EA started in 1989, or is to be found in one source or standard.
As the digitisation of business processes increased, many EA models and frameworks were established.
1993: Stephen Spewak published a process for EA planning
“Architecture planning is the process… to improve data quality, access to data, adaptability…, data interoperability and sharing, and cost containment.” (Spewak, 1993 reference)
1996: The Open Group Architecture Framework (TOGAF)
TOGAF started as an IT Architecture Framework, based the US armed forces TAFIM, was not for EA!
From 1996 to 2001, TOG members developed TOGAF versions 1 to 7 as a framework for IT (infrastructure) architecture.
1999: Federal Enterprise Architecture Framework (FEAF)
“An EA establishes the Agency-wide roadmap to achieve an Agency’s mission through optimal performance of its core business processes within an efficient IT environment.” (FEAF, 1999 reference).
For decades, government agencies have been striving to systematise and digitise their core business processes.
2002: TOGAF version 8 – the “enterprise edition”
This substantial revision, which borrowed much from FEAF v1.1 (1999), emphasised business and information system architecture over IT architecture.
"Today’s CEOs know that the effective management and exploitation of information… is a key factor to business success.” (TOGAF 9.1, 2009 reference)
2006: “EA as Strategy” Ross, Weill and Robertson
The higher purpose of EA has always been to optimise and improve business processes.
"Companies excel because they've [decided] which processes they must execute well, and have implemented the IT systems to digitise those processes." (Ross, Weill and Robertson, 2006 reference).
The book says EA is about “digitizing core business processes”.
That means describing business operations in terms of discrete entities and events.
· Classifying business entities and events into types such that all instances can be treated the same way.
· Defining the information/data variables the business must capture and store.
· Considering how technologies are best used to support and enable the business systems so described.
Nowadays digitisation of business activities is considered important in a wide range of enterprises.
2007: The Tele Management Forum (TMF)
What do government agencies, financial service providers and telecommunications and have in common?
They are enterprises that are built around data processing.
The TMF is an association of service providers in the telecommunications and related industries.
Their work includes an EA model (not including the IT platform) divided into three domains.
· business functional decomposition and process flow models (eTOM) (2007 reference)
· shared information/data model (SID)
· application portfolio map (TAM).
2011: TOGAF version 9
“Today’s CEOs know that the effective management and exploitation of information through IT is a key factor to business success.” (2011 reference to TOGAF).
“The domain of information-intensive organisations [is] the main focus [of EA models].” (2013 reference)
“An Enterprise Architecture is typically developed because key people have concerns that need to be addressed by the business and IT systems within an organization.” (2016 reference).
EA has always been centered on the use of information, rather than technologies.
The focus has been supporting and enabling regular business processes through optimal use of information systems
The solution architect’s vision usually has relatively short time frame and narrow scope.
The EA vision is broader, to standardise, integrate and coordinate business information and processes across an enterprise.
Enterprise architects respond to business plans, align system changes with those plans, and may influence them.
They take a cross-organisational and strategic perspective of the architecture domains in the table below.
Business mission, vision, drivers, strategies, goals and top-level business plans.
Mergers, acquisitions, divestments, internal organisation restructurings and rationalisations.
(Business Systems Planning)
Business functions/capabilities, roles, processes/value streams and their interrelationships.
The services they provide to each other and to entities in the business environment.
Applications and their interrelationships
The services applications provide to each other and to business activities.
Data stores and data flows, the data structures they contain and the qualities of that data.
Platform technologies and their interrelationships
The services technologies provide to each other and to business applications.
EA is challenging politically as well as technically.
Architects need not only appropriate knowledge, skills and resources, but also cross-organisational authority.
Some enterprises have never set up an EA practice; some have done so and struggled to make an impact.
Read Enterprise and solution architecture for a view of mainstream EA today.
Modern EA frameworks have to help architects work in an Information Age in which trends include:
· Distribution of business operations between locations and between enterprises.
· Integration of business operations between locations and between enterprises
· Semantic interoperability between businesses that communicate over the web
· Digital transformation
Recently Norway’s DNB bank commented that its future is to become a “technology company with a banking license”.
Last month Barclays’ Jes Staley commented that a “modern bank today is a technology company with a balance sheet and with regulators”.
Technology has had a prominent role within the Banking landscape for at least the last 50 years.
With the emergence of new challengers, FinTechs etc, technology is getting more of a mention as a differentiator as part of investor relations updates
Whether it is driving digitalisation, improving colleague/client experience, or driving efficiency.” (2017 reference)
Businesses always want to make money, save money, increase efficiency, effectiveness and customer satisfaction.
EA has always looked to help by “digital transformations” of one kind or another.
What does a digitally transformed enterprise look like today?
1. Business roles and processes are monitored and guided by digitised information, leaving an audit trail.
2. Business actors (customers, suppliers, employees etc.) interact using digital channels, leaving an audit trail.
3. Business products, services and the customer experience are updated quickly in response to feedback
4. Business systems use social media to find and interact with customers.
5. Business data is properly confidential, consistent, conformant to rules, correct and controlled.
6. Business processes and rules can be changed quickly.
7. Business information systems are available when users need them – and graceful when not.
8. Business information systems can be scaled up and down easily according to demand.
9. Business systems are minimally dependent on single-source or single-location technologies.
10. Big data captured from sensors, machines and the Internet of Things is used to good effect.
11. The business looks out for opportunities to use wearable computing, visual web, virtual reality and Artificial Intelligence.
The list above is intended to be broad; some take a specifically customer or employee oriented view of digitisation.
So you may want to add, remove or change any of the points in the list.
This source suggests five priorities:
· Customer Experience: Build customer experiences well. Ground it in ehaviour-based customer data that is collected and used data ethically.
· Culture & Leadership: Employee engagement is low despite social networks and collaboration tools. A leaders worried abut giving control to employees? Your culture may need to shift.
· Content Strategy: You need good content, accessible via touchpoints in any customer journey. Look beyond marketing and comms for expertise in customer service, sales, product development, and HR.
· Digital Ecosystem Rationalization: Who “owns” digital strategy? Marketing, Sales, and Service may have different platforms, data, content, and metrics. Engage HR.
· Paid Social: Social ad spending has doubled over the past two years as channels like Facebook, Twitter, and LinkedIn become increasingly effective.
Enterprise Architecture (EA) emerged soon after the dawn of the Information Age.
It became apparent that, in digitising business processes, there is short versus long term planning trade off.
Short-term planning created silo systems, duplicated data storage, and overlapping systems that could not be integrated.
The need was for a long-term approach to the analysis and re-architecting of business processes and data.
“The origin of this approach can be traced to back to the works of P. Duane Walker in the late 1960s.
As the director of architecture at IBM, Walker had established an enterprise analysis-oriented planning tool called Business System Planning (BSP), concerned with this [short versus long term planning] trade-off.” (Ref. 1 and 2)
BSP was a method for analyzing, defining and designing the information architecture of organizations.
It provided executives with direction, a developmental blueprint for information systems and a decision-making framework for IT expenditures.
It featured analyses that map Processes, Data and Applications (and other architectural entities) to each other.
At the heart of the analysis was the Data entity/Process matrix.
The process BSP has been described thus:
“Its focus on data and processes was new way to view the firm.
The basic building blocks are: Data Classes… Business Processes...
1 Obtain authorization for the study.
2 Assemble the study team.
3 Define the data classes.
4 Define the business processes.
5 Using these data classes and business processes, define the information architecture. etc." (1970 reference.)
In his 1982 study of Walker’s BSP Zachman wrote:
“Enterprise-level architecture is required to constrain the [system] design and development activity.”
“enterprise-wide “architectures,” or structures, through BSP, BICS, or some other methodology can be expected to be increasingly prevalent”.
“[BSP] develops an enterprise-level architecture (albeit rather rudimentary).”
“BSP focuses primarily on the process [create and use] data relationship… several other relationships… support intermediate and/or secondary conclusions.”
“BSP chooses the process/data class structure because it :
· it is trying to identify the data problem,
· it is seeking to develop a stable foundation for architectural use, and last,
· it is attempting to highlight the long-term, short-term trade-off decisions that must be made by the management of the business.” (1982 reference)
EA had been discussed for about 20 years before The Open Group Architecture Framework (TOGAF) was published.
Yet to begin with, TOGAF owed little or nothing to that EA history.
The vision of The Open Group (TOG) was a world in which information flows are “boundaryless”.
TOG looked to realise that vision through the development of IT standards that make information systems portable and interoperable.
At the same time, the US “IT Management Reform Act” demanded that federal government CIOs maintain a “sound and integrated IT architecture repository”.
From 1996 to 2001, TOG members developed TOGAF versions 1 to 7 as a framework for IT architecture.
When it finally joined the EA party in 2002, TOGAF 8 (the "Enterprise Edition") borrowed a great deal from FEAF v1.1 (1999)
The US government’s Federal EA Framework drew from three mainstream EA sources: Zachman, Spewak and the NIST EA standard.
It declared its aims as: “common data and business processes”, “information sharing”, “new and improved processes”
· “Organize Federal information including common data and business processes on a Federalwide (enterprisewide) scale
· Promote information sharing throughout the Federal Enterprise and within segments
· Help the Federal Enterprise develop architecture descriptions
· Help the Federal Enterprise move more quickly toward developing new and improved processes”
Service-orientation and business architecture
TOGAF 8 extended the service-oriented specification approach used in TOGAF 1 to 7.
"TOGAF Version 8… uses the same underlying method for developing IT architectures that was evolved [in] TOGAF up to and including Version 7.
That method was to define a system or component, an architecture domain or layer, logically, by the services it provides.
Version 8 applies that… method to the other domains of an overall Enterprise Architecture - the Business Architecture, Data Architecture, and Application Architecture, as well”
TOGAF 9 introduced a content framework and meta model based on this service-oriented approach.
TOGAF 9.2 introduced more focus on business architecture – and has lost almost everything in TOGAF 7.
Mainstream EA is what it has been since Walker (reputedly) coined the term in the 1960s and Zachman followed his lead.
It is about the optimisation and extension of business roles and processes that create and use business data - and the technologies required.
Some use “EA” as a label for all manner of business management consulting and socio-cultural systems thinking.
This leads to confusion.
Read EA as applied system theory for discussion.
Mainstream EA is about the optimisation and extension of business roles and processes that create and use business data - and the technologies required.
It was designed with reference to "information-intensive” businesses such as government, banking, finance and retail.
Their business transactions continually consume, process and produce business data.
They rely heavily on technologies that “manufacture” the information needed by customers, suppliers and employees.
Six Sigma and Lean were designed with reference to processes in physical product manufacturing.
Six Sigma features two approaches to a project.
· DMAIC (Define, Measure, Analyze, Improve and Control) used to improve existing processes.
· DMEDI (Define, Measure, Explore, Develop, Integrate and Validate) used to develop new products and processes.
Six Sigma emphasises data capture and employs concepts like:
· Voice of Customer, Critical to Quality Requirements, Measurement System Analysis, root cause analysis methods.
· Hypothesis Testing, ANOVA, Design of Experiments, Risk and Change management tools, Control Charts.
Lean is a set of principles derived from work on the Toyota Production System.
It is primarily concerned with the elimination of waste, which is classified into eight types:
· Inventory, Unused Creativity, Waiting, Excess Motion
· Transportation, Overprocessing, Defects, Overproduction.
One strand of EA is to apply Lean principles to information-intensive roles, processes, data, and enabling technologies.
Further, Lean and Six Sigma can be useful in examination of whatever business scenarios/processes are relevant to the "request for architecture work".