The principles of complexity

I regularly hear how complex our environment is and how many different systems, platforms, methods and names exist within the Sabre enterprise. Internet standards, specifically those around Service Oriented Architecture (SOA) are an effort to drive a simpler approach to systems and specifically the connectivity of systems. I read an interesting interview a little while back that referenced Coase Theorem regarding the interconnectivity of systems. Ronald Coase won a Nobel Prize in 1991 for the Theorem for its impact in economic modeling and in the interview it was mentioned in the context of connecting systems and that got me thinking. The original notes and thoughts had nothing to do with computer systems but the parallels are very interesting.

A simplified (grossly; My apologies to Mr. Coase) view of the theorem is that when the cost of integrating processes is high then it makes more sense to do things yourself. As the cost of integration is lowered the economics change. If the transaction costs can be modeled then you can run simulations and determine the best possible solution for your own approach. This means that as network communication costs get cheaper and cheaper and the cost of information is driven down that new models and approaches suddenly become viable that may not have previously been possible. Take the well known example of amazon.com. Initially, all Amazon did was take free catalog information, used it to build an online catalog and used the internet to make sales at a discount since they don't have the traditional overhead of a brick and mortar store.

Even following Coase Theorem there are three main factors that inhibit efficient allocation.

• Imperfect information - If you don't know the costs of the transaction or it is uncertain or unstable it is difficult if not impossible to model
• High transaction cost - High transaction costs is the core of the theorem and driving down these costs becomes paramount to flexibility within the model. If any one item within the formula is large it can dominate the costs of the system.
• High negotiation costs - If the costs to negotiate the transaction are high it quickly erodes the value of the external connection. This could be due to too many parties in the negotiation or the overhead of the conversation to establish the connections.
• A fourth item, not core to the theorem that is used to argue against the ability to effectively model the costs is in cases where the actors in the system have irreconcilable different valuations on the transaction.

SOA is intended to be an enabler to drive down the cost of connections of systems. By making integration and calls between systems as simple and cheap as possible it enables a whole new way of building systems. More focus can be placed on the Business Logic layer and the orchestration of the services to unlock business value. As services are built, focus can be made to make the individual services as efficient, cheap and scalable as possible. The cost of building new systems goes down as well enabling new and more creative businesses to be explored.

In order for this to work though there is a few big simplifying assumptions. One, that the cost of integration really is lower. This doesn't just cover the development costs that are immediate and visible but also the long term costs and benefits. Maintenance should be cheaper if services are specialized and localized. For Maintenance to be cheaper services need to be stable. This speaks to the fourth item of irreconcilable different valuations. The best services (cheapest in terms of this current discussion) are the ones that can be taken for granted. Perfect availability is the goal, just like with phone systems you don't pick up your handset hoping that there is a dial tone, you expect it. For computer systems that doesn't mean that you shouldn't harden your own system to protect itself from harm because you expect other services to be there. Quite the opposite is true. The phone company gets their "always on dial tone" through A LOT of redundancy, automatic system corrections and advanced error detection and response frameworks. ( Mary Poppendieck discusses in her Lean conversations how testing to find defsfects is waste while inspection to prevent defects is essential) These same concepts apply to good software design. It is also important to consider the cost of the negotiations. In the case of
systems that means that getting your connection and utilizing it needs to be cheep as well.

This post is beginning to get a little long so thanks to those of you who rode along with me this far. I will follow up in subsequent posts on the Tenets of good SOA design and what we can do to approach our own level of dial tone.