Use and abuse of REST (REpresentational State Transfer)

This page is published under the terms of the licence summarized in the footnote.

 

Abstract

An earlier paper outlined a history of modular design and integration, covering OOD, DO, CBD, SOA, WS, EAI, REST, WOA and EDA.

This paper discusses REST in more depth. Is it compatible with SOA?

Contents

Preface (reprise) 1

Five principles of REST. 2

Use and abuse of REST. 7

Remarks. 10

 

Preface (reprise)

"The beginning of wisdom for a computer programmer is to recognise the difference between getting a program to work and getting it right" M.A. Jackson (1975).

 

What makes an architecture good or right? Here, it means meeting requirements more elegantly and economically than alternative designs.

A module should be encapsulated, that is, defined primarily by its input/output interface, by the discrete events it can process and services it can offer.

But the general principle of encapsulation doesn’t get us very far. The architect has to make a series of modular design decisions.

·         What is the right size and scope of a module?

·         How to avoid or minimise duplication between modules?

·         How to separate or distribute modules?

·         How integrate modules?

 

For 40 years, the IT industry has continually revisited modular design and integration concepts and principles.

 

Initially, Service-Oriented Architecture was a reaction against distributed objects using object request brokers like DCOM.

SOA evangelists advocated a more loosely-coupled kind of modular design and integration style, with opposing features.

 

Earliest OO design presumptions

Modern SOA design presumptions

Object identifiers

Internet domain names or URIs

One name space

Several name spaces

Stateful objects

Stateless modules

Reuse by inheritance

Reuse by delegation

Intelligent domain objects

Intelligent process controllers

Request-reply invocations

Message passing

Blocking servers

Non-blocking servers

 

REST adopts the features listed under SOA in the table above, and adds some more.

The term REST was defined by Roy T. Fielding; formerly a designer of Web protocols including HTTP and URIs.

Roy formalized ideas behind these protocols in his Ph.D. thesis.

REST helps software designers make best use of what the Web is good at, especially hyperlinking.

 

Roy talked about architectural styles: abstract patterns that express the core ideas behind an architectural approach. e.g.

·         the null style (which has no constrains at all)

·         pipe and filter,

·         client/server,

·         distributed objects, and

·         REST.

 

Each architectural style comes with a set of constraints that define it.

REST is an abstract style that can be implemented with many different technologies and different ways.

But most popular implementation the REST style employs the Web, HTTP, and URIs – as discussed below.

 

Five principles of REST

Much of this section has been edited, distilled and adapted from the first article in InfoQ magazine - REST / eMag Issue 12 - April 2014.

 

REST is a set of principles for using Web standards to write software applications. 

The term “elementary module” is used here instead of the formally correct “resource”.

 

A so-called RESTful architecture is based on two ideas.

Every elementary module of a software application (Web Service, web site, HTML page, XML document, printer, other physical device, etc.) is named as a distinct web resource.

A client module can call a server module using only the operations of standard protocol - usually HTTP.

This decouples distributed modules; it means a client module needs minimal information about the server module.

 

A so-called REST-compliant architecture adds a third idea.

Every server module is limited to offering only four standard services – as in HTTP.

Given there are fewer operations (verbs) per module, there must be more modules (nouns).

 

Five key principles of REST are introduced and discussed below.

Name each elementary module using a URI

Tim Berners-Lee’s Axiom 0 for Web Design is "All resources on the Web must be uniquely identified with a URI."

URIs are universal identifiers, cheap to create and use.

This globally-unified naming scheme helps both your personal use of the Web (via your browser) and communication between software modules.

 

Using URIs to identify your elementary modules means that each gets a unique identifier in a global namespace.

You don’t have to come up with your own naming scheme.

URIs works on global scale, and are understood by most people.

Human-readable URIs are not a pre-requisite for a RESTful design – but they help.

 

Here are some URIs which identify individual items of interest:

·         http://example.com/customers/1234

·         http://example.com/orders/2007/10/776654

·         http://example.com/products/4554

·         http://example.com/processes/salary-increase-234.

 

Elementary modules that merit a URI are often aggregate objects (containing more than a row in a database table).

For example, an order includes several order lines, an address, and many other attributes.

 

You can name persistent collections of interest as well as individual items.

·         http://example.com/orders

·         http://example.com/products.

 

You can name enquiry results as well as persistent lists.

·         http://example.com/orders/2007/11 (identifies all orders submitted in November 2007)

·         http://example.com/products?color=green (identifies the set of green products).

 

The idea is to uniquely identify everything of interest in a software application.

REST tends to create modules you usually don’t see in a typical application design: a process, a process step, a sale, a negotiation, a request for a quote.

This, in turn, can lead to the creation of more persistent entities than in a non-RESTful design.

 

All the resources that your application uses are given URIs, be they individual items or collections, virtual or physical objects, or computation results.

Link elementary modules using URIs

This principle is more formally known as: “Hypermedia as the engine of application state” (HATEOAS).

This fancy name means that an application follows hyperlinks as it progresses through states.

 

Suppose a server returns the data structure defined in this XML fragment:

<order self=’http://example.com/ customers/1234’ >

<amount>23</amount>

<product ref=’http://example.com/ products/4554’ />

<customer ref=’http://example.com/ customers/1234’ />

</order>

 

An application can follow the product and customer links in this document to retrieve more information.

The beauty of using URIs is that links can point to resources provided by a different application, server, or company on another continent.

In this way, all the resources that make up the Web can be linked to each other.

Use only the operations in a standard web protocol/interface

When you enter a URI into your browser’s address field and hit return — how does your browser know what to do with the URI?

It knows because every server-side resource is accessed via a standard web protocol/interface – such as HTTP.

HTTP verbs include GET, POST, PUT, DELETE, HEAD, and OPTIONS.

The meaning of these operations is defined in the HTTP standard, along with some guarantees about their behavior.

 

A client sending an HTTP message is limited to using the standard HTTP verbs.

Further, in “REST-compliant” scenario, the server-wide resources are modularised so that each module offers only four operations.

What the module does depends on whether it is a collection or an item.

 

Resource

Collection URI, such as http://example.com/resourcelist

Element URI, such as http://example.com/resources/item17

GET (read)

List the URIs and perhaps other details of the collection's members.

Retrieve a representation of the addressed member of the collection, expressed in an appropriate Internet media type.

PUT (update/create)

Replace the entire collection with another collection.

Replace the addressed member of the collection, or if it doesn't exist, create it.

POST (create)

Create a new entry in the collection.

The new entry's URI is assigned automatically and is usually returned by the operation.

Treat the addressed member as a collection in its own right and create a new entry in it.

Not generally used – because it is not idempotent.

DELETE (delete)

Delete the entire collection.

Delete the addressed member of the collection.

 

Idempotence

HTTP is based on asynchronous message passing.

There is no promise a response will be returned to a request; this means you want the protocol to work idempotently.

Idempotence means that sending a repeat message does no harm, which is guaranteed for these HTTP operations:

·         If you issue a GET request and get no reply, you can harmlessly issue the request again.

·         If you issue a PUT request it will create resource at the given URI with the given data or, if it is there already, update the resource with the same data.

·         If you issue a DELETE then repeat it, trying to deleting something already deleted is harmless.

 

Since POST can mean “create a new resource” or invoke other arbitrary processing; it is neither safe nor idempotent.

 

So, a client can always retrieve a representation (a rendering of it) using GET.

And because GET’s semantics are defined in the standard as “safe”, a client has no obligations when using it.

And since GET supports efficient and sophisticated caching, in many cases, a request does not have to reach the ultimate server.

 

If you’re used to a different design approach, these principles and restrictions may seem problematic, but consider this example.

 

Old style: many operations, and few module instances

Consider the following interface:

 

Asset Manager Component

listUsers()

addUser()

getUser()

updateUser()

removeUser()

findUser()

listLocations()

getLocation()

findLocation()

addLocation()

removeLocation()

updateLocation()

addAsset ()

assignAsset ()

 

If a client wants to consume these services, it needs to be coded against this particular interface.

The interface defines the component’s application-specific protocol.

 

REST style: few operations, but many modules upon which to invoke those operations.

In a RESTful style, have to get by with the generic interface that makes up the HTTP application protocol.

This means creating a whole universe of new resources thus:

 

>> example to be inserted

 

Notice that the specific operations of the old component have been mapped to the standard HTTP operations on several new components.

A GET on a URI that identifies a User resource is just as meaningful as a getUser operation on the old component.

Flexibility of representation (data type)

How does a client know how to deal with the data it retrieves as a result of a GET or POST request?

The response contains metadata about the type of data in the HTTP content-type header

 

Thus, HTTP allows a separation of concerns between invoking operations and handling the data returned

So, a client that knows how to handle a particular data format can interact with all resources that can provide a representation in this format.

 

Using HTTP content negotiation, a client can ask for a representation in a particular format.

The result might be some company-specific XML format that represents customer information.

 

GET /customers/1234 HTTP/1.1

Host: example.com

Accept: application/vnd.mycompany. customer+xml

 

Say the client sends a different request, for a customer address in vCard format.

 

GET /customers/1234 HTTP/1.1

Host: example.com

Accept: text/x-vcard

 

By the way, API design is often driven by the idea that everything that doable via a human UI should also be doable via the API.

And aligning Web UI and Web API can help to produce a better Web interface for both humans and other applications.

Communicate statelessly

A server should not have to hold state for any of the clients it communicates with - beyond a single request.

REST mandates that state be either held on the client or turned into a resource state.

The reasons are:

·         scalability: it is difficult to scale out server that maintain client state.

·         loose-coupling: the client is not dependent on talking to the same server in two consecutive requests

 

Use and abuse of REST

A RESTful HTTP approach to exposing operations is different from RPC, distributed objects and Web services; it takes some mind shift to really understand this difference.

This section includes discussion of the five principles:

·         Name each elementary module using a URI

·         Link elementary modules using URIs

·         Use only the operations in a standard web protocol/interface

·         Flexibility of representation (data type)

·         Communicate statelessly.

 

Ugo Corda: I don't have any problem with people defining their own architectural principles.

What I object to is people hijacking the term REST for other purposes.

 

Graham: We could say the same about other terms:

“EA” is abused as a label for every conceivable design and/or management consulting activity.

“OO” and “SOA” often mean little beyond encapsulation of modules behind interfaces.

The meanings of "loose-coupling", "asynchronous", even "component", "service" and "Interface" are all disputable.

All these terms can be dangerous in a conversation between architects until they have spent 30 minutes discussing what they don't mean.

 

Ugo: How true! I have been fighting the SOA battle too.

But in the case of REST there is one advantage: it was defined by a single person, Roy Fielding.

There is a well-defined description (Roy's thesis). Roy’s many blogs clarify the intent of his thesis.

And you can always count on Roy to say "shut up, you don't know what you are talking about" as soon as you stray from the true REST path.

Name each elementary module using a URI

Graham: A module, or resource, could be a Web Service, a web site, an HTML page, an XML document, a physical device.

The idea is that each of the many elementary modules is identified by a URI.

So we don’t need any special infrastructure platform to allocate object identifiers; the DNS does that.

 

Ugo: Further, you don't have to know about all those resources ahead of time.

You need to know just one single resource to get the application started.

The rest you will discover along the way, given URIs returned by the first module.

 

Graham: So, a client doesn’t have to know anything about the overall structure in which components cooperate.

The name space for a RESTful application is (potentially) the whole internet.

Use only the operations in a standard web protocol/interface

Graham: I gather that in REST, client and server components are loosely-coupled in a very particular sense.

They use only the universally-recognised operation names of the standard protocol via which they communicate.

A client of a web resource/module can use only the few operations named in an internet-friendly protocol.

 

Ugo: Once you choose a protocol and its operations, that's it; you cannot keep changing that.

REST is not limited to HTTP and its four standard operations. But in practice, that is what is used today.

 

Graham: REST not only restricts a client component to using nothing but the four HTTP operation names to invoke a server component.

In a properly REST-compliant design, it also means a server component can offer only four services (using the four HTTP verbs).

And this means we need many small components (many nouns).

 

Ugo: Yes, the restriction on the number of possible operations is compensated by a proliferation in the resources.

You might or might not like the large number of resources involved. But if you use proper HATEOAS (see below) that is probably not an issue.

 

Graham: But suppose somebody wants to design a server component that offers more than four operation types?

They can add an operation type attribute to the invocation parameters, and so force clients to name their required operation.

Pretty soon, they have an anti-RESTful design, featuring a few big components (nouns) offering lots of services (verbs).

 

Ugo: Unfortunately that is what is happening in practice because so many people have no idea of what REST really is.

A client should use HTTP operations only in the way they were designed.

Many pseudo-REST users have narrowed down to using just one operation: say POST for everything, even where that subverts the HTTP operation semantics.

That is not REST, and is not SOAP either, so you get the worst of both worlds.

 

Graham: So why are they using REST?  Is it simply to avoid the platform infrastructure and overheads required for SOAP, CORBA whatever?

 

Ugo: Yes, in a lot of cases, plus the myth that REST is easier to do than SOAP or CORBA.

In reality, I think that REST done right can be more complex and less intuitive to regular programmers than SOAP.

But those are not the only reasons.

For more meaningful reasons for doing pseudo-REST (not true REST), take a look at HTTP://nordsc.com/ext/classification_of_HTTP_based_apis.html .

Link elementary modules using URIs (HATEOAS - hypermedia as the engine of application state)

Graham: I have a background in enterprise applications that process structured data.

We expect to use transaction management middleware for request-reply invocations, and to use reliable message delivery middleware for fire-and-forget invocations.

Many advocates of REST are more interested in web applications that process documents or media files.

 

Ugo: I think of a REST application developing according to the HATEOAS mechanism.

I cannot really tell (either in advance or after the facts) how many services are involved, what their contracts are and who is responsible for them.

Let alone find them in a “design-time service catalogue under the governance of one or more architects".

Pretty much all I have for a REST application is the root entry point plus a bunch of media types that are possibly going to be used at runtime (and the hope that no additional media types are going to get involved).

That is pretty much the opposite of the SOA "command and control" approach of having well defined service contracts and well defined business processes known at design time.

Interestingly enough, I have started seeing REST advocates say that the concept of "service" is actually harmful to REST (see http://tech.groups.yahoo.com/group/rest-discuss/message/15134 and following conversations).

Flexibility of representation (data type)

Ugo: Clients (unless they are human) do have to know something in advance.

They have to know the syntax and semantics of the media types returned by the RESTful servers.

 

Graham: Where you say media types, I’d say data types.

 

Ugo: In the case of HTTP, "media type" is exactly as specified in the HTTP 1.1 specification.

A good example of media types are the ones associated with ATOM - see the standard at en.wikipedia.org/wiki.

These include the HTTP media type "application/atom +xml".
The client receives a resource representation from the REST server with a particular declared media type.

The client then can decide, based on that representation, based on its knowledge of that media type and based on its own application goals, what to do next.

Typically, the client will parse the representation document guided by the media type syntax.

Inside the document it will find the hyperlinks that, to its judgment, are relevant to its application goal, and it will then issue one of the 4 operations against one of those hyperlinks.

It will get another resource representation in a particular media type as a response.

And so on and so forth until the whole application goal of the client is satisfied.

 

Please note this could be a major obstacle in practical implementations.

The set of media types used could be open ended (in which case you are almost sure the interaction will fail in cases of unknown media types).

The client complexity will also be very high.

Think of a machine client trying to emulate a human browsing and interacting with the Web.

That is only possible if the client can rely on predefined media types, but even then, it is not a walk in the park.

Communicate statelessly

Chris Britton: A key point in REST is that the server is stateless like a web browser.

We can create a seemingly stateful application by hiding state in the web page or in a cookie.

It needs fast network technology to make this a realistic way of building applications.

When you invoke a service in a fire-and-forget style, you need integration between the message passing middleware and transactions to ensure that one message and only one message is sent and received.

If the sending server aborts the transaction, in most circumstances you want the message to be aborted as well.

Remarks

For me, REST is more building regulations than architecture.

Dion Hinchcliffe says that WOA (he mostly means REST) enables systems to be built from the bottom-up, which it does of course.

Some beautiful systems may emerge from the bottom-up – by evolutionary development - but you can’t call them architected if there is no architect, no architectural blueprint.

 

Some align REST with SOA.

They suggest a SOA be built using using REST as the building regulations, instead of using of enterprise middleware and/or instead of use Web Services and SOAP.

I recently spent several hours talking to a team leader on a huge (400 person) software project.

They are using a mix of technologies, including middleware in some places and REST in other places.

She didn't talk about technology choices (aside from saying Web Methods is not really an ESB.).

Her concern is getting the modularity right, using Demeter's law among other ideas.

She reported they are finding small modules, each with four CRUD operations, seems about right.

She says they are implementing SOA using REST in some places.

 

 

Footnote: Creative Commons Attribution-No Derivative Works Licence 2.0   27/09/2014 21:49

Attribution: You may copy, distribute and display this copyrighted work only if you clearly credit “Avancier Limited: http://avancier.co.uk” before the start and include this footnote at the end.

No Derivative Works: You may copy, distribute, display only complete and verbatim copies of this page, not derivative works based upon it.

For more information about the licence, see  http://creativecommons.org