50 years of digital transformation and EA

One of about 300 papers at http://avancier.website. Copyright Graham Berrisford. Last updated 14/03/2017 10:25

 

This paper interweaves the stories of digital transformation and EA.

It includes a list of references from 1970 onwards, and a characterisation of the digital enterprise.

Contents

50 years of EA thinking. 1

The first 20 years. 1

The next 20 years. 3

Mainstream EA today. 4

The digital enterprise today. 5

References in chronological order. 6

 

50 years of digitisation and EA thinking

Business roles and processes have always required technologies for information storage and communication.

Since the modern Information Age dawned, the technologies have increasingly been “digital”.

Incrementally, human activities have been replaced, supported and enabled by computer activities.

Digitisation helps an enterprise to standardise and integrate its business data and processes.

Mobile applications, social media, Big Data and the Internet of Things are enabling new business roles and processes.

 

It appears that the NIST EA model of 1989 was the first publication to use the term “EA” formally.

“ [recommended in US government as] a management tool that illustrates the interrelationship of enterprise business, information, and technology environments.

The five-layered model allows for organizing, planning, and building an integrated set of information and information technology architectures.  (FEAF v1.1, 1999)

 

But it would be misleading to suggest EA started with the NIST EA model 1989, or is to be found in one source or standard.

The EA vision was evident long before 1989 in Walker’s BSP, the Zachman Framework, Information Engineering and the PRISM report.

And has been pursued since then by Spewak, Ross et al, and in architecture frameworks like FEAF, IAF, EAF and TOGAF (see references below).

 

Most EA sources share the vision of optimisation by taking an enterprise-wide view of business data and processes.

The solution architecture vision is to support/enable, speed/scale up, optimise/extend roles and processes in a narrow scope.

The EA vision is broader, to standardise, integrate and coordinate business data and processes across an enterprise.

What may be called “mainstream EA” includes countless authors and publications that have shared that vision over the last 50 years.

 

Today, mainstream EA attends to technologies used by business activities to create and use digital data.

And looks out for business opportunities presented by new digital technologies.

However, the architecture of the IT platform is increasingly outsourced.

Mainstream EA also attends to human roles that create and use digital data.

But socio-cultural "systems thinking" has never been a focus of mainstream EA.

Changes to an organisation’s management structure and roles are usually managed by a parallel "business change" team.

The first 20 years

People in social systems have always exchanged messages and remembered information gathered from messages.

As business systems evolved, people formalised how they exchanged and stored business information.

They exchanged order, invoice and payment data - perhaps first on clay tablets, and for many centuries on paper.

They created, processed and moved this data using pens, typewriters and snail-mail postal organisations.

They recorded data for future reference in filing cabinets and card indexes.

 

Digitisation of roles and processes in the Information Age

Since the 1960s, business systems have increasingly depended on digital computing technologies (rather than clay tablets, pen and paper).

The digitisation of business information using computers led us into the modern Information Age.

It isn’t just that data structures evolved from human memory, through written records, to bits stored in computer memory and persistent data storage.

Digital information technologies have supported and enabled an ever wider range of business activities.

They increased the ability of a business to capture, store, process and analyse business data – and benefit from that.

They strengthened the ability of a business to monitor and direct activities - and inform managers about them.

Now, they enable a business to exchange information with people using social media, and with machines using the Internet of Things.

 

Duane Walker

Enterprise Architecture (EA) emerged soon after the dawn of the Information Age.

True, people had started to presume that digital information technologies would be used to capture and provide data.

But the springboard for EA was not thinking about technologies.

 

Consultants wanted to engage business people with how to improve, standardise and integrate their business processes.

Short-term (silo system) planning had created difficulties; a stable basis for wider and longer-term business system planning was wanted.

The need was for a long-term (enterprise-wide) approach to the analysis and re-architecting of business processes and data.

 

“The origin of this [architecture framework] approach can be traced to back to the works of P. Duane Walker in the late 1960s.

As the director of architecture at IBM, Walker had established an enterprise analysis-oriented planning tool called Business System Planning (BSP), concerned with this [short versus long term planning] trade-off.” (Ref. 1 and 2)

 

BSP featured analyses of Processes, Data and Applications.

These architecture entities were mapped to each other (and other architectural entities) in relationships.

At the heart of the analysis was the Data entity/Process matrix.

    Business process

Register customer

Place order

Complete sale

Launch product

Recall product

Data entity

Customer

x

x

 

 

x

Sale

 

x

x

 

x

Sale item

 

x

 

 

x

Product type

 

 

x

x

 

Viewing an enterprise in terms of business data created and used by business processes was an innovation in BSP.

“Its focus on data and processes was new way to view the firm. The basic building blocks are: Data Classes… Business Processes...

[The first five of ten steps are] 1 Obtain authorization for the study. 2 Assemble the study team. 3 Define the data classes. 4 Define the business processes.

5 Using these data classes and business processes, define the information architecture..." (1970 reference.)

 

John A Zachman

In his 1982 study of Walker’s BSP Zachman wrote:

“Enterprise-level architecture is required to constrain the [system] design and development activity.”

enterprise-wide “architectures,” or structures, through BSP, BICS, or some other methodology can be expected to be increasingly prevalent”.

“[BSP] develops an enterprise-level architecture (albeit rather rudimentary).”

“BSP focuses primarily on the process [create and use] data relationship  several other relationships… support intermediate and/or secondary conclusions.”

“BSP chooses the process/data class structure because it :

·         it is trying to identify the data problem,

·         it is seeking to develop a stable foundation for architectural use, and last,

·         it is attempting to highlight the long-term, short-term trade-off decisions that must be made by the management of the business.” (1982 reference)

 

The first two columns of Zachman’s matrix were inherited from BSP.

Zachman admitted his debt to “Dewey” Walker in an interview with Roger Sessions in “Perspectives of the International Association of Software Architects” in 2007.

Zachman did not copy BSP, he transformed it incrementally during the 1980s.

“In the late 1980s, several other architecture frameworks were developed, which were significantly based on Zachman’s prior work.” Ref .2

Other influences on EA included Information Engineering, PRISM report (1986 reference) and the NIST EA standard (1989 reference).

The next 20 years

EA grew in the Information Age alongside increasing digitisation of business processes that use business information.

EA models and frameworks were established as the Information Age progressed, and people recognised the need to address:

·         coordination of business information systems (1982 reference).

·         integration of business, data, application and platform technology views (1986 reference).

·         business and information/data architecture before technology architecture (1989 reference).

·         improving data quality and exploiting business data (1993 reference)

 

Mainstream EA has always been concerned about data quality and its impact on business processes.

Further, EAs have looked to standardise, integrate, optimise and extend business roles and processes that create and use business data.

That is, for example, the theme of "EA as Strategy", (2006 reference).

 

EA is about data and processes first, technologies second, and socio-cultural system thinking barely at all.

E.g. The US government’s Federal EA Framework started with aims to share and improve business data and processes.

FEAF v1.1 (1999) draws from three mainstream EA sources: Zachman, Spewak and the NIST EA standard.

It declares four aims, without mentioning technology:

·         “Organize Federal information including common data and business processes on a Federalwide (enterprisewide) scale

·         Promote information sharing throughout the Federal Enterprise and within segments

·         Help the Federal Enterprise develop architecture descriptions

·         Help the Federal Enterprise move more quickly toward developing new and improved processes”

 

What do government agencies, telecommunications and financial service providers have in common?

They are enterprises that are built around data processing.

“The domain of information-intensive organisations [is] the main focus [of EA models].” (2013 reference)

 

For decades, government agencies have been striving to systematise and digitise their core business processes.

An enterprise architecture establishes the Agency-wide roadmap to achieve an Agency’s mission

through optimal performance of its core business processes within an efficient [IT] environment.” (2001 reference).

 

The TM Forum is an association of service providers in the telecommunications and related industries.

Their work includes an EA model (not including the IT platform) divided into three domains.

·         business functional decomposition and process flow models (eTOM) (2007 reference)

·         shared information/data model (SID)

·         application portfolio map (TAM).

 

More recently, the following advertisement from a bank shows the importance of information/data to modern banking processes.

“Leading bank is urgently seeking a proven EA to engage and lead IT projects including

the Enterprise Information Architecture, information models and flows, data dictionaries, data standards, data quality standards and processes…

develop and maintain the logical Enterprise Information Architecture that enables seamless information interoperability of all Bank systems for efficiency and cost-effectiveness.” (anon.)

 

Nowadays digitisation of business activities is considered important in a wide range of enterprises.

"Companies excel because they've [decided] which processes they must execute well, and have implemented the IT systems to digitise those processes." (2006 reference).

Mainstream EA today

Political, economic and sociological forces influence enterprise architecture (EA), and may be influenced by it.

However, mainstream EA remains focused on the design and planning of systems in which roles and processes create and use business data.

It integrates business processes through the exchange of information in messages and the sharing of memories.

It about the why, when, where and how of business processes that create, store and use information.

It is about the design and planning of changes to those business activities, and/or to the capture and provision of that information.

 

“The purpose of EA is to optimize across the enterprise the often fragmented legacy of processes (both manual and automated) into an integrated environment.”

“Today’s CEOs know that the effective management and exploitation of information through IT is a key factor to business success.” (2011 reference).

“An Enterprise Architecture is typically developed because key people have concerns that need to be addressed by the business and IT systems within an organization.” (2016 reference).

 

Today, mainstream EA frameworks and tools include Cap Gemini’s IAF, SAP’s EAF, TOGAF and ArchiMate.

These EA frameworks have to help architects work in an Information Age in which trends include:

·         Distribution of business operations between locations and between enterprises.

·         Integration of business operations between locations and between enterprises

·         Semantic interoperability between businesses that communicate over the web

·         Digital transformation (see below).

·         Delegation of responsibility for EA to solution architects, and governance of them.

 

Moreover, enterprise architecture (rather than solution architecture) is about addressing these matters at a strategic and cross-organisational level.

This does not mean enterprise architects must model or transform their entire enterprise.

Though they do often draw an enterprise-wide functional decomposition or capability map at a very high level.

Read “Enterprise and solution architecture” for how enterprise architects differ from solution architects.

The digital enterprise today

Businesses always want to make money, save money, increase efficiency, effectiveness and customer satisfaction.

EA has always looked to help by “digital transformations” of one kind or another.

What does a digitally transformed enterprise look like today? Would you add, remove or change any of these points?

 

1.      Business roles and processes are monitored and guided by digitised information, leaving an audit trail.

2.      Business actors (customers, suppliers, employees etc.) interact using digital channels, leaving an audit trail.

3.      Business products, services and the customer experience are updated quickly in response to feedback

4.      Business systems use social media to find and interact with customers.

5.      Business data is properly confidential, consistent, conformant to rules, correct and controlled.

6.      Business processes and rules can be changed quickly.

7.      Business information systems are available when users need them – and graceful when not.

8.      Business information systems can be scaled up and down easily according to demand.

9.      Business systems are minimally dependent on single-source or single-location technologies.

10.  Big data captured from sensors, machines and the Internet of Things is used to good effect.

11.  The business looks out for opportunities to use wearable computing, visual web, virtual reality and Artificial Intelligence.

 

The list above is intended to be broad; some take a specifically customer or employee oriented view of digitisation

The source below suggests five priorities:

·         Customer Experience: Build customer experiences well. Ground it in behavior-based customer data that is collected and used data ethically.

·         Culture & Leadership: Employee engagement is low despite social networks and collaboration tools. A leaders worried abut giving control to employees? Your culture may need to shift.

·         Content Strategy: You need good content, accessible via touchpoints in any customer journey. Look beyond marketing and comms for expertise in customer service, sales, product development, and HR.

·         Digital Ecosystem Rationalization: Who “owns” digital strategy? Marketing, Sales, and Service may have different platforms, data, content, and metrics. Engage HR.

·         Paid Social: Social ad spending has doubled over the past two years as channels like Facebook, Twitter, and LinkedIn become increasingly effective.

https://www.prophet.com/thinking/2016/02/the-top-digital-transformation-priorities-for-2016-part-1/

References

Main references in chronological order

The Walker to Zachman architecture framework history is discussed briefly in these two references.

Ref. 1: https://www.zachman.com/resources/ea-articles-reference/321-the-zachman-framework-for-architecture-revisited-by-paul-hermans

Ref. 2: “Complexity Management in Engineering Design – a Primer” By Maik Maurer.

And in a 2007 interview with Roger Sessions.

 

c1970

Business Systems Planning” see the Wikipedia entry, and the web site of Robinson College of Business, Georgia State University.

1981

“Information Engineering” Savant, Martin, J. and C. Finkelstein (consider also “Strategic Information Planning Methodologies”. Prentice Hall, 1989.)

1982

"Business Systems Planning and Business Information Control Study: A comparison" John Zachman, in IBM Systems Journal 21(1). P31-53.

Zachman, a fan of Walker’s BSP approach, spoke of “enterprise level architecture”.

1986

Dispersion and Interconnection: Approaches to Distributed Systems Architecture  the PRISM report, from a consortium including IBM, DEC and others

This divided enterprise-wide architecture into business, data, applications and infrastructure technology domains.

1987

A Framework for Information Systems Architecture” J.A. Zachman, IBM Systems Journal, Volume 26, No. 3, pp. 276–292, 1987.

1989

Enterprise Architecture Model” National Institute of Standards and Technology.

This NIST EA Model had five layers: business, information, applications, stored data and platform technologies.

1992

Extending and Formalizing the Framework for Information Systems Architecture”, J.F. Sowa, J.A. Zachman, IBM Systems Journal, Volume 31, No. 3, pp.590-616, 1992.

This proposed how to describe “the overall information system and how it relates to the enterprise and its surrounding environment.”

1993

EA Planning  by Stephen Spewak (See entry in Wikipedia for references.)

This defined a data-centric planning process “defining architectures for the use of information in support of the business and the plan for implementing those architectures”.

1996

IT Management Reform Act” (Clinger Cohen Act)

This made a federal agency’s CIO responsible for “developing, maintaining and facilitating the implementation of a sound and integrated IT architecture for the executive agency.”

1998-9

Federal EA Framework, Federal CIO Council.

2001

A practical guide to Federal Enterprise Architecture, Chief CIO council.

2003

The Zachman Framework For Enterprise Architecture: Primer for Enterprise Engineering and Manufacturing.” By John A. Zachman. Zachman International.

This listed 10 key points for EA, emphasising that if you have not documented an architecture, you are implementing, not architecting.

2004?

Integrated Architecture Framework” (IAF) Cap Gemini. One of many EA frameworks; this one influenced the development of TOGAF 9.

2006

EA as Strategy” Ross, Weill and Robertson. This reported and interpreted findings from MIT’s Center for Information System Research.

2007

"Service Provisioning-Challenges, Process Alignment and Tool Support." Bergstra, J. und M. Burgess (Herausgeber): Handbook of Network and System Administration. Elsevier

2011

TOGAF 9.1” can be read at www.opengroup.org  This promotes and encourages EA with reference to the operating model concept from “EA as Strategy”.

2013

“ArchiMate 2.1” can be read at www.opengroup.org

2016

“ArchiMate 3.0” can be read at www.opengroup.org

Today

Avancier’s reference model for enterprise and solution architecture is downloadable from http://avancier.website.

FEAF and its references

EA is about data and processes first, technologies second, and socio-cultural system thinking barely at all.

E.g. The US government’s Federal EA Framework started with aims to share and improve business data and processes.

1996: The Clinger Cohen Act required that US federal government agency investments in major information systems be tied to improvements in business processes.

1998: The CIO Council began developing FEAF to promote shared development for common federal processes, interoperability, and sharing of information among the agencies other governmental entities.

FEAF v1.1 (1999) draws from three mainstream EA sources: Zachman, Spewak and the NIST EA standard.

It declares four aims, without mentioning technology:

·         “Organize Federal information including common data and business processes on a Federalwide (enterprisewide) scale

·         Promote information sharing throughout the Federal Enterprise and within segments

·         Help the Federal Enterprise develop architecture descriptions

·         Help the Federal Enterprise move more quickly toward developing new and improved processes”

It lists 37 references, which include the words data and information 35 times, and technology only 5 times. (See below.)

 

1954.

The Practice of Management.

Drucker, P.E.

Harper & Row, Inc.,

1969.

Data Structures Diagram.

Bachman, C.

Communication of the ACM. SIGBDP Database Vol. 1, No. 2,

1970.

Future Shock.

Toffler, A.

Bantam,

1976.

The Entity Relationship Model - Toward a Unified View of Data.

Chen, P.P.

ACM Transactions on Database Systems, Vol. 1, No. 1,

1979.

Structured Analysis and System Specification.

DeMarco, Tom.

Yourdon Press, Englewood Cliffs, NJ,

1981.

Information Engineering.

Martin, J. and C. Finkelstein.

Savant,

1982.

Three-Dimensional Data Model.

Spewak, Steven H.

Data Base Newsletter, Vol. 10, No. 2,

1984.

Logical Database Design Technologies.

Brown, R. G., and the Data Base Design Group.

Data Base Design Group,

1987b

Entity Modeling: Techniques and Applications.

Ross, Ronald G.

Database Research Group, Boston, MA,

1987a

A Framework for Information Systems Architecture.

Zachman, John A.

IBM Publication G321-5298. 914-945-3836. IBM Systems Journal. Vol. 26, No. 3.

1989a

Handbook of Relational Data Base Design.

Flemming, Candice C. and Barbara von Halle.

Addison-Wesley, Reading, MA,

1989b

Strategic Information Planning Methodologies.

Martin, James.

Prentice Hall,

1989c

NIST Special Publication 500-167, Information Management Directions: The Integration Challenge,

Rigdon, Bradford W.

September

1990a

The Fifth Discipline: The Art and Practice of the Leading Organization.

Senge, P. M.

Doubleday,

1990b

Powershift.

Toffler, A.

Bantam,

1992a

Designing Quality Data Bases with IDEF1X.

Bruce, Tom.

Dorset House Publishing Co., Inc.,

1992b

Extending and Formalizing the Framework for Information Systems Architecture.

Sowa, J.F. and J. A. Zachman.

IBM Publication G321-5488. 914-945-3836. IBM Journal, Vol. 31, No. 3,

1992c

Enterprise Architecture Planning, Developing a Blueprint for Data, Applications and Technology.

Spewak, Steven H. with Steven C. Hill.

John Wiley & Sons, Inc.,

1993d

Reengineering the Corporation.

Hammer, M. and J. Champy.

Harper Business,

1993e

Paradigm Shift.

Tapscott, Don and Art Caston.

McGraw Hill, Inc,

1994

A Brief Introduction to the Zachman Framework.

Burgess, Bruce H. and Thomas A. Hokel.

(800) 890-0902 Framework Software, Inc.

1995.

Database Processing.

Kroenke, David M.

Prentice Hall,

1996d.

Building Enterprise Information Architectures, Reengineering Information Systems.

Cook, Melissa.

Prentice Hall PTR, Upper Saddle River, NJ,

1996c

The Model is the Business, Creating Customer Focused Organization.

Dickinson, Brian.

LCI Press, Kings Beach, CA,

1996b.

Funding Information Systems Investments, October 25,

Office of Management and Budget (OMB) Memorandum M-97-02,

requires that Agency investments in major information systems be consistent with Federal, Agency, and Bureau ITAs.

1996a

The Clinger-Cohen Act

 

assigned the CIOs with the responsibility to develop information technology architectures (ITAs).

1997a

C4ISR Architecture Framework Version 2.0,

U.S. Department of Defense.

 

1997b

The Commandments of COTS: Still in Search of the Promised Land.

Carney, David J. and Patricia A. Oberndorf.

Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA, May

1997c

Data Stores, Data Warehousing and the Zachman Framework, Mapping Enterprise Knowledge.

Inmon, W. H., John A. Zachman, and Jonathan G. Geiger.

McGraw Hill, New York, NY,

1997d

Information Technology Architectures, June 18,

OMB Memorandum M-97-16,

 

1997e

The Business Rule Book: Classifying, Defining, and Modeling Rules.

Ross, Ronald G.

Database Research Group, Inc.,

1998a

Software Architecture in Practice.

Bass, Len, Paul Clements, and Rick Kazman.

Addison-Wesley,

1998b

Redundant Systems Development Costs, The High Cost of Low Quality Data Presentation,

English, Larry P.

July

1998c

High Performance Client/Server.

Loosley, Chris and Frank Douglas.

John Wiley & Sons, Inc.,

1998d

Business Rule Concepts; The New Mechanics of Business Information Systems.

Ross, Ronald G.

Business Rule Solutions, Inc.,

1998e

Looking Back and Looking Ahead, Part 1.

Zachman, John A.

604-899-5452. DataToKnowledge Newsletter, Vol. 26, No. 3. May/June,

1998f

Looking Back and Looking Ahead, Part 2.

Zachman, John A.

604-899-5452. DataToKnowledge Newsletter, Vol. 26, No. 4. July/August,

1999a

Constructing Blueprints for Enterprise IT Architectures.

Boar, Bernard H.

John Wiley and Sons, Inc.,

1999b

Improving Data Warehouse and Business Information Quality: Methods for Reducing Costs and Increasing Profits.

English, Larry P.

John Wiley and Sons, Inc.,

1999c

Life is a Series of Trade-Offs and Change is Accelerating! 604-899-5452.

Zachman, John A.

Special Reprint from the Data to Knowledge Newsletter. January/February and March/April

Other references

From “Complexity Management in Engineering Design – a Primer” By Maik Maurer

Enterprise Architecture and Architecture Frameworks

The design and application of architecture frameworks has become one of the major approaches towards managing the complexity of systems.

The origin of this approach can be traced to back to the works of P. Duane Walker in the late 1960s.

As the director of architecture at IBM, Walker had established an enterprise analysis oriented planning tool called Business System Planning (BSP).

Beginning in the early 1980s, Walker’s student John Zachman carried the BSP approach forward and in 1987 published his findings in “A framework for information systems architecture”…

he also provided model associations on a generic level [material, location, function, people, time and motivation].

 

Other references from the Object Management Group

Unified Modeling Language®: Infrastructure, Version 2.4.1 (formal/201-08-05), Object Management Group, August 2011.

Business Process Modeling Notation™ (BPMN™), Version 2.0 (formal/2011-01-03), Object Management Group, 2011.

Business Motivation Model (BMM), Version 1.1 (formal/2010-05-01), Object Management Group, 2010.