Another Design Pattern: Layers

Though I usually talk with software as the underlying example, I call this a general design pattern since it works in many fields.

Layers or layering is a way of separating a large goal into smaller pieces. It is not necessarily by sequence of execution or design but by amount of detail of domain expertise. That is, the

The 'layers' pattern is really a pattern using another pattern, the interface. When the terms 'vertical application' or 'horizontal design are used', horizontal means a general purpose set of tools, vertical means using a ladder of tools, one depending on the next to solve a niche problem.

horizontal vs vertical scaling

The canonical example is the OSI model for networking (computer communication) popularized by Tanenbaum (fig 1-20, Tanenbaum, Wetherall, Computer Networks (2011)):

Note - level 7:mailActual communication over a cable is through the physical layer. There are rules and restrictions on the physical layer that the layer above it conforms to in order to get certain behavior but is easier to think in in the higher abstraction.

It is a stacking of individual components, a higher one depending on a lower one. The higher layer is dependent on the lower one, and is specified in the language of the lower one.

Layers of science - mathematics, physics, chemistry, biology, psychology. Here one may notice that the interactions of the layers is not necessarily one of interfaces: though understanding of physics ostensibly underlies chemistry, understanding the former doesn't guarantee an understanding of the latter. Physics may give insight but won't necessarily determine how chemistry works. This just shows that layers is a general strategy for dealing with a large domain (here science is about as large as it gets).

Purity from xkcd

The benefits of the 'layers' pattern is that it cuts up a larger monolithic area into many smaller manageable pieces. Instead of trying to understand the meaning of life, these 'smaller' areas are more manageable. Also, each individual piece can be worked on with a limited vocabulary (that of the lower piece). You don't have to understand all the complexities of the lower layer.

This is another pattern that encourages modularity: the higher layer (the abstract layer) has its own language and ways of doing things implemented in the interface given by the lower layer. The lower layer (and so also layers below it) could be entirely replaced by a different implementation.  Or one layer could be replaced given that it respects the interface of its higher and lower layers.

Another example from computers is architecture: electronics, microprogramming, machine language, assembly language, higher level programming language, specification language. A new layers in design can come from either direction: higher level programming languages like C or Fortran came from wanting to abstract away common patterns in assembly language (like subroutines), and microprogramming came from trying to implement machine language more easily in the electronics.

A higher layer is often an abstraction from a lower one. The common patterns can be set in (parameterized) stone in lower ones, with enough freedom to connect those things in the higher layer to get applications done. That way, the higher layer can worry about its problems without having to worry about some implementation details, and the lower layer can worry about its own problems. This is a way of explaining the benefits of encapsulation/modularity/data hiding: reduce the knowledge requirements at each level while maintaining functionality.

One difficulty with layering is making sure the knowledge of the layers is sufficiently compartmentalized, that the user/designer in one layer needs to know as little as possible of other layers. A leaky abstraction occurs when too much is required of those outside a layer.

Without these layers, there can easily be efficiencies that are possible by thinking fast, by the designer of the monolith using fine local information to get benefits in the larger architecture. But this can lead to spaghetti code (spaghetti design), where small changes in a small but common/overused element can cause large changes (or rather problems/bugs/disasters).

The layers may not be a deliberate separation of affairs, but reached organically. One layer s formed naturally, and then others see that the items at that layer can be used to create a layer on top. Or the implementation of a bottom layer may be rethought in order to create a more useful layer below.

A monolith is a great big ball of cleverness. Every small action depends on efficiencies gained by side effects of other small actions far away. This often leads to very brittle designs, but think of the efficiency gain! Using layers may remove such efficiencies at the expense of ease in modification.

Or it may turn out that a hidden layer implements things in such a way that the patterns of a higher layer can take advantage of the much lower layer if only that information was exposed, either to use directly (dancing links which reuse presumably freed pointers more efficiently) or implicitly.