50 years of digital transformation and EA
One of about 300 papers at http://avancier.website. Copyright Graham Berrisford. Last updated 23/07/2017 10:52
This paper interweaves the stories of digital transformation and EA.
It includes a list of EA references and a characterisation of the digital enterprise.
Business systems evolved over millennia by formalising the roles and rules, messages and memories, of social systems.
Every business depends on sending and receiving messages, and remembering what has been done to date.
Business roles and processes have always required technologies for information storage and communication.
Since the modern Information Age dawned, those technologies have increasingly been “digital”.
Incrementally, more and more human activities have been “digitised”.
The Information Age brought new opportunities, but also new difficulties.
Solution architects worked to support/enable, speed/scale up, optimise/extend particular business systems.
They acted independently of each other – under the influence of competing information technology vendors.
The result was an estate of systems that were not standardised, not integrated and did not share common services.
Some systems stored different version of the same data, some did not interoperate, and many were tied to particular technologies.
This led to problems in coordinating the business roles and processes that relied on these information systems.
For example, sales and billing systems, or social and health care systems, might be inconsistent and uncoordinated.
The origins of EA can be traced back to IBM’s Business System Planning method, which Duane Walker reputedly started c1970.
By the 1980s, the cost and quality issues caused by “silo systems” led many to call for higher-level “enterprise architecture”.
Significant references include:
· Business information systems analysis should be raised to the “Enterprise-level” (Zachman, 1982 reference).
· “EA is concerned with “system implementations, extending in scope and complexity to encompass an entire enterprise.” (Zachman).
· Architecture is divisible into four domains: business, data, application and technology (PRISM report, 1986 reference).
· The focus is on business and information before technology (NIST EA model, 1989 reference).
As ever more business processes were digitised, models for EA were formalised.
For decades, authors have put the quality and use of business information at the heart of EA.
· “Architecture planning is the process… to improve data quality, access to data, adaptability…, data interoperability and sharing, and cost containment.” (Spewak, 1993 reference)
· “The domain of information-intensive organisations…is the main focus of the [EA modelling] language” (ArchiMate standard v2.1)
· "Today’s CEOs know that the effective management and exploitation of information… is a key factor to business success.” (TOGAF 9.1, 2009 reference)
The higher purpose has always been to optimise and improve business processes.
· “An EA establishes the Agency-wide roadmap to achieve an Agency’s mission through optimal performance of its core business processes within an efficient IT environment.” (FEAF, 1999 reference).
· "Companies excel because they've [decided] which processes they must execute well, and have implemented the IT systems to digitise those processes." (Ross, Weill and Robertson, 2006 reference).
Ross, Weill and Robertson say EA is about “digitizing core business processes”.
That means describing business operations in terms of discrete entities and events.
· Classifying business entities and events into types such that all instances can be treated the same way.
· Defining the information/data variables the business must capture and store.
· Considering how technologies are best used to support and enable the business systems so described.
The solution architect’s vision usually has relatively short time frame and narrow scope.
The EA vision is broader, to standardise, integrate and coordinate business information and processes across an enterprise.
Enterprise architects respond to business plans, align system changes with those plans, and may influence them.
They take a cross-organisational and strategic perspective of the architecture domains in the table below.
Business mission, vision, drivers, strategies, goals and top-level business plans.
Mergers, acquisitions, divestments, internal organisation restructurings and rationalisations.
(Business Systems Planning)
Business functions/capabilities, roles, processes/value streams and their interrelationships.
The services they provide to each other and to entities in the business environment.
Applications and their interrelationships
The services applications provide to each other and to business activities.
Data stores and data flows, the data structures they contain and the qualities of that data.
Platform technologies and their interrelationships
The services technologies provide to each other and to business applications.
EA is challenging politically as well as technically.
Architects need not only appropriate knowledge, skills and resources, but also cross-organisational authority.
Some enterprises have never set up an EA practice; some have done so and struggled to make an impact.
Some enterprise architects work in a team called “strategy and architecture” or the like.
They act a central design authority and risk management function.
They guide and govern lower level solution and technical architects working on specific programmes and projects.
Why must EA attend to technologies?
Digital technologies help an enterprise to standardise and integrate its business information and processes, and exploit data captured.
On the one hand, some management of IT platform technologies is being outsourced via the use of “cloud computing”.
And “TCP/IP everywhere” enables integration of cross-platform (heterogeneous) distributed systems.
On the other hand, mobile applications, social media, Big Data and the Internet of Things are enabling new business roles and processes.
So EA must still attend to the technologies that help people create and use digital data, and look for opportunities created by new digital technologies.
Read the paper below for a more extensive EA history and many references.
Read EA versus SA for how mainstream EA relates to solution architecture.
People in social systems have always exchanged messages and remembered information gathered from messages.
As business systems evolved, people formalised how they exchanged and stored business information.
They exchanged order, invoice and payment data - perhaps first on clay tablets, and for many centuries on paper.
They created, processed and moved this data using pens, typewriters and snail-mail postal organisations.
They recorded data for future reference in filing cabinets and card indexes.
Digitisation of roles and processes in the Information Age
Since the 1960s, business systems have increasingly depended on digital computing technologies (rather than clay tablets, pen and paper).
The digitisation of business information using computers led us into the modern Information Age.
It isn’t just that data structures evolved from human memory, through written records, to bits stored in computer memory and persistent data storage.
Digital information technologies have supported and enabled an ever wider range of business activities.
They increased the ability of a business to capture, store, process and analyse business data – and benefit from that.
They strengthened the ability of a business to monitor and direct activities - and inform managers about them.
Now, they enable a business to exchange information with people using social media, and with machines using the Internet of Things.
Enterprise Architecture (EA) emerged soon after the dawn of the Information Age.
It became apparent that, in digitising business processes, there is short versus long term planning trade off.
Short-term planning created silo systems, duplicated data storage, and overlapping systems that could not be integrated.
The need was for a long-term approach to the analysis and re-architecting of business processes and data.
“The origin of this approach can be traced to back to the works of P. Duane Walker in the late 1960s.
As the director of architecture at IBM, Walker had established an enterprise analysis-oriented planning tool called Business System Planning (BSP), concerned with this [short versus long term planning] trade-off.” (Ref. 1 and 2)
BSP was a method for analyzing, defining and designing the information architecture of organizations.
It provided executives with direction, a developmental blueprint for information systems and a decision-making framework for IT expenditures.
It featured analyses that map Processes, Data and Applications (and other architectural entities) to each other.
At the heart of the analysis was the Data entity/Process matrix.
“Its focus on data and processes was new way to view the firm. The basic building blocks are: Data Classes… Business Processes...
1 Obtain authorization for the study.
2 Assemble the study team.
3 Define the data classes.
4 Define the business processes.
5 Using these data classes and business processes, define the information architecture.
etc." (1970 reference.)
John A Zachman
In his 1982 study of Walker’s BSP Zachman wrote:
“Enterprise-level architecture is required to constrain the [system] design and development activity.”
“enterprise-wide “architectures,” or structures, through BSP, BICS, or some other methodology can be expected to be increasingly prevalent”.
“[BSP] develops an enterprise-level architecture (albeit rather rudimentary).”
“BSP focuses primarily on the process [create and use] data relationship… several other relationships… support intermediate and/or secondary conclusions.”
“BSP chooses the process/data class structure because it :
· it is trying to identify the data problem,
· it is seeking to develop a stable foundation for architectural use, and last,
· it is attempting to highlight the long-term, short-term trade-off decisions that must be made by the management of the business.” (1982 reference)
The first two columns of Zachman’s matrix were inherited from BSP.
Zachman admitted his debt to “Dewey” Walker in an interview with Roger Sessions in “Perspectives of the International Association of Software Architects” in 2007.
Zachman did not copy BSP, he transformed it incrementally during the 1980s.
“In the late 1980s, several other architecture frameworks were developed, which were significantly based on Zachman’s prior work.” Ref .2
Other influences on EA included Information Engineering, PRISM report (1986 reference) and the NIST EA standard (1989 reference).
NIST EA model
It appears that the NIST EA model of 1989 was the first publication to use the term “EA” formally; it was recommended in US government as
“a management tool that illustrates the interrelationship of enterprise business, information, and technology environments.
The five-layered model allows for organizing, planning, and building an integrated set of information and information technology architectures. (FEAF v1.1, 1999)
It would be misleading to suggest EA started in 1989, or is to be found in one source or standard.
As the digitisation of business processes increased, EA models and frameworks were established.
Significant references include:
· “Enterprise-level” coordination of business information systems (Zachman, 1982 reference).
· Integration of four architecture domains: business, data, application and technology (PRISM report, 1986 reference).
· The NIST EA model: business and information/data before technology (NIST, 1989 reference).
· “Enterprise architecture planning” to improve data quality and exploit business data (Spewak, 1993 reference).
· Later architecture frameworks like IAF, EAF and TOGAF.
EA had been discussed for about 20 years before The Open Group Architecture Framework (TOGAF) was published.
Yet to begin with, TOGAF owed little or nothing to that EA history.
The vision of the Open Group (TOG) was a world in which information flows are “boundaryless”.
TOG looked to realise that vision through the development of IT standards that make information systems portable and interoperable.
At the same time, the US “IT Management Reform Act” demanded that federal government CIOs maintain a “sound and integrated IT architecture repository”.
From 1996 to 2001, TOG members developed TOGAF versions 1 to 7 as a framework for IT architecture.
When it finally joined the EA party in 2002, TOGAF 8 (the "Enterprise Edition") borrowed a great deal from FEAF v1.1 (1999)
The US government’s Federal EA Framework drew from three mainstream EA sources: Zachman, Spewak and the NIST EA standard.
It declared its aims as: “common data and business processes”, “information sharing”, “new and improved processes”
· “Organize Federal information including common data and business processes on a Federalwide (enterprisewide) scale
· Promote information sharing throughout the Federal Enterprise and within segments
· Help the Federal Enterprise develop architecture descriptions
· Help the Federal Enterprise move more quickly toward developing new and improved processes”
TOGAF 8 extended the service-oriented specification approach used in TOGAF 1 to 7.
"TOGAF Version 8… uses the same underlying method for developing IT architectures that was evolved [in] TOGAF up to and including Version 7.
That method was to define a system or component, an architecture domain or layer, logically, by the services it provides.
Version 8 applies that… method to the other domains of an overall Enterprise Architecture - the Business Architecture, Data Architecture, and Application Architecture, as well”
TOGAF 9 introduced a content framework and meta model based on this service-oriented approach.
What do government agencies, telecommunications and financial service providers have in common?
They are enterprises that are built around data processing.
“The domain of information-intensive organisations [is] the main focus [of EA models].” (2013 reference)
“Technology has had a prominent role within the Banking landscape for at least the last 50 years.” (2017 reference at the top of the paper)
For decades, government agencies have been striving to systematise and digitise their core business processes.
“An enterprise architecture establishes the Agency-wide roadmap to achieve an Agency’s mission
through optimal performance of its core business processes within an efficient [IT] environment.” (2001 reference).
The TM Forum is an association of service providers in the telecommunications and related industries.
Their work includes an EA model (not including the IT platform) divided into three domains.
· business functional decomposition and process flow models (eTOM) (2007 reference)
· shared information/data model (SID)
· application portfolio map (TAM).
More recently, the following advertisement from a bank shows the importance of information/data to modern banking processes.
“Leading bank is urgently seeking a proven EA to engage and lead IT projects including
the Enterprise Information Architecture, information models and flows, data dictionaries, data standards, data quality standards and processes…
develop and maintain the logical Enterprise Information Architecture that enables seamless information interoperability of all Bank systems for efficiency and cost-effectiveness.” (anon.)
Nowadays digitisation of business activities is considered important in a wide range of enterprises.
“Companies excel because they’ve [decided] which processes they must execute well, and have implemented the IT systems to digitise those processes.” (2006 reference).
EA has always been centered on the use of information, rather than technologies.
The focus has been supporting and enabling regular business processes through optimal use of information systems
EAs have looked to standardise, integrate, and extend business roles and processes that create and use business data.
Mainstream EA remains focused on the design and planning of systems in which roles and processes create and use business data.
It integrates business processes through the exchange of information in messages and the sharing of memories.
It about the why, when, where and how of business processes that create, store and use information.
It is about the design and planning of changes to those business activities, and/or to the capture and provision of that information.
“Today’s CEOs know that the effective management and exploitation of information through IT is a key factor to business success.” (2011 reference).
“An Enterprise Architecture is typically developed because key people have concerns that need to be addressed by the business and IT systems within an organization.” (2016 reference).
Today, mainstream EA frameworks and tools include Cap Gemini’s IAF, SAP’s EAF, TOGAF and ArchiMate.
Modern EA frameworks have to help architects work in an Information Age in which trends include:
· Distribution of business operations between locations and between enterprises.
· Integration of business operations between locations and between enterprises
· Semantic interoperability between businesses that communicate over the web
· Digital transformation (see below).
· Delegation of responsibility for EA to solution architects, and governance of them.
Moreover, enterprise architecture (rather than solution architecture) is about addressing these matters at a strategic and cross-organisational level.
Read “Enterprise and solution architecture” for how enterprise architects differ from solution architects.
Recently Norway’s DNB bank commented that its future is to become a “technology company with a banking license”.
Last month Barclays’ Jes Staley commented that a “modern bank today is a technology company with a balance sheet and with regulators”.
Technology has had a prominent role within the Banking landscape for at least the last 50 years.
With the emergence of new challengers, FinTechs etc, technology is getting more of a mention as a differentiator as part of investor relations updates
Whether it is driving digitalisation, improving colleague/client experience, or driving efficiency.” (2017 reference)
Businesses always want to make money, save money, increase efficiency, effectiveness and customer satisfaction.
EA has always looked to help by “digital transformations” of one kind or another.
What does a digitally transformed enterprise look like today?
1. Business roles and processes are monitored and guided by digitised information, leaving an audit trail.
2. Business actors (customers, suppliers, employees etc.) interact using digital channels, leaving an audit trail.
3. Business products, services and the customer experience are updated quickly in response to feedback
4. Business systems use social media to find and interact with customers.
5. Business data is properly confidential, consistent, conformant to rules, correct and controlled.
6. Business processes and rules can be changed quickly.
7. Business information systems are available when users need them – and graceful when not.
8. Business information systems can be scaled up and down easily according to demand.
9. Business systems are minimally dependent on single-source or single-location technologies.
10. Big data captured from sensors, machines and the Internet of Things is used to good effect.
11. The business looks out for opportunities to use wearable computing, visual web, virtual reality and Artificial Intelligence.
The list above is intended to be broad; some take a specifically customer or employee oriented view of digitisation.
You may want to add, remove or change any of the points.
The source below suggests five priorities:
· Customer Experience: Build customer experiences well. Ground it in ehaviour-based customer data that is collected and used data ethically.
· Culture & Leadership: Employee engagement is low despite social networks and collaboration tools. A leaders worried abut giving control to employees? Your culture may need to shift.
· Content Strategy: You need good content, accessible via touchpoints in any customer journey. Look beyond marketing and comms for expertise in customer service, sales, product development, and HR.
· Digital Ecosystem Rationalization: Who “owns” digital strategy? Marketing, Sales, and Service may have different platforms, data, content, and metrics. Engage HR.
· Paid Social: Social ad spending has doubled over the past two years as channels like Facebook, Twitter, and LinkedIn become increasingly effective.
1990s TOGAF was born to help people tidy up a messy IT platform, using a Technical Reference Model.
2000s TOGAF joined mainstream EA by adding phases to define target business processes and data.
2010s TOGAF became more a solution architecture project management method with side bar references to EA.
Many scale down and tailor the ADM for use as a management framework for a solution delivery projects.
That is OK, but calling it “EA” obscures the meaning of the term as defined above.
Read EA and TOGAF for discussion of the difference.
Mainstream EA is what it has been since Walker (reputedly) coined the term in the 1960s and Zachman followed his lead.
It is about the optimisation and extension of business roles and processes that create and use business data - and the technologies required.
Some use “EA” as a label for all manner of business management consulting and socio-cultural systems thinking.
This leads to confusion.
Read EA as applied system theory for discussion.
Mainstream EA is about the optimisation and extension of business roles and processes that create and use business data - and the technologies required.
It was designed with reference to "information-intensive” businesses such as government, banking, finance and retail.
Their business transactions continually consume, process and produce business data.
They rely heavily on technologies that “manufacture” the information needed by customers, suppliers and employees.
Six Sigma and Lean were designed with reference to processes in physical product manufacturing.
Six Sigma features two approaches to a project.
· DMAIC (Define, Measure, Analyze, Improve and Control) used to improve existing processes.
· DMEDI (Define, Measure, Explore, Develop, Integrate and Validate) used to develop new products and processes.
Six Sigma emphasises data capture and employs concepts like:
· Voice of Customer, Critical to Quality Requirements, Measurement System Analysis, root cause analysis methods.
· Hypothesis Testing, ANOVA, Design of Experiments, Risk and Change management tools, Control Charts.
Lean is a set of principles derived from work on the Toyota Production System.
It is primarily concerned with the elimination of waste, which is classified into eight types:
· Inventory, Unused Creativity, Waiting, Excess Motion
· Transportation, Overprocessing, Defects, Overproduction.
One strand of EA is to apply Lean principles to information-intensive roles, processes, data, and enabling technologies.
Further, Lean and Six Sigma can be useful in examination of whatever business scenarios/processes are relevant to the "request for architecture work".