Drag Racing vs Rally Racing

Some of you that know me outside of my Uptime Blog posts know that I am a fan of Rally Racing. In part it is because I drive a Subaru so I was drawn to watch. (Yes, a blue STi with the handle on the back so that God can reach down and shake me when he thinks I am doing something dumb is mine.) But also I like Rally racing because it takes a real world car (granted, they gut the interior and up the strength a bit on the components) and drive it like fiends through corners, over dirt roads, through snow and over jumps. It is a real test of driver abilities and a lot of fun to watch or even play on a game console.

So what does this have to do with technology? It has to do with optimization and how optimized you can make a system, either people or computing. If you take the example of a Rally car they need to have horsepower yes, but that is not enough. You need to have torque, suspension flexibility, handling adjustability the ability to turn and to stop is just as important as the ability to go. In one stage you may be racing on a mostly strait asphalt road, in the next over a switch-back 180 degree turn infested gravel road through a forest. The ability to handle that change is very important to build into the car. Conversely if you look at a drag race car the need for handling, turning, even suspension decrease. What you need is horsepower and pure strait line speed (and hopefully the ability to stop, though if you look at some of these cars a parachute is what is used). How boring.

I maintain that business is much more like a Rally race than a Drag race. You never know what curve a competitor is going to throw at you next. The ability to handle those curves, stop on a dime, turn in the other direction and then go full bore ahead again is critical to business survival.

Sometimes though we seem to optimize our ability to go fast and go strait. We focus only on the immediate goal in front of us. Cut out items that are not critical to our achieving that strait ahead goal. We optimize our "people load", "trim the fat", etc. to keep everyone as utilized as possible moving towards the strait line goal. Then a change comes up... and we discover we can't turn as fast as when we started and we are surprised to find we even have a problem changing our direction simply due to momentum.

This is why things like Non-Functional Requirements are critical to a project and a companies long term success. They maintain the ability to turn and stop and give the slack necessary to allow a shift one step to the right. If we are all pushing so hard to go forward in a strait line even adjusting slightly to the side is difficult. In planning this is Risk Management. It's accepting that planning "if all things go right" is not going to really get you there.

Getting the right strategy means you have to assume your competitors are damn good, or at the very least as good as you are, and that they are moving just as fast or faster. When it comes to peering into the future, you just can't be paranoid enough.

- Jack Welch

A Link, A List and a Quote

Not a big post for today. Just a simple one.

Mark Coulston recently published an article in Fast Company that is worth checking out. It is entitled, Don't Mess Up. IT lists the 10 things that smart leaders do to help them cope with stress that are actually destructive.

1. Avoid confrontation
2. Hire advisors, but don't listen to them
3. Don't accept when they're wrong
4. Avoid dealing with reality
5. Wait too long or not long enough to cut their losses
6. Focus too much on strategy vs. execution
7. Trust analysis over instinct
8. Trust instinct over analysis
9. Practice favoritism
10. Minimize the importance of what they don't understand

Mark also shares a quote and we all know I love quotes so here you go.

"Everybody here has the ability to do anything I do and much beyond. Some of you will and some of you won't. For those who won't, it will be because you get in your own way, not because the world doesn't allow you."

- Warren Buffett speaking the University of Washington

SOA reduces integration costs

Many times business applications typically reflect a software vendor's opinion of how a business process should be performed. Since vendor opinions and real-world processes often do not match, many systems fail to support actual business operations. Easy examples of this can be seen in the big ERP systems that are implemented and "customized" for many millions of dollars only to need more rounds of "customization" with every new release.

One of the goals of Service Oriented Architecture (SOA) is to change the way enterprises deploy applications to support business processes. Using things like Orchestration and Business Process Management technology teams pull together the needed connections to components with a business flow based on how business is performed. Then when business changes (I know, I know.. that never happens...) you simply change the wiring. Of course, that is the goal... in order for it to be a reality some things need to happen. For a list of some of what I think needs to change look at my previous series on SOA changes.

What do you get for all of this effort?

Essentially, you get reduced integration expense. There are of course other benefits, but from a business POV this is what you get. It could be as simple as a reduced integration cost with other systems and future systems, or changing systems. It could be increasing reuse of assets; always a good thing to use something that exists vs build it from scratch each time. Increased customer responsiveness in the ability to change and modify business process more dynamically. You even get a reduction in risk, both business and project level risk, because you are dealing with smaller components that may already exist and allow for localized changes. But, as I started this paragraph, it all boils down to lower integration costs that enable all of these things.

Zapthink recently published an interesting article on the REAL costs of integration as part of their justification for SOA. The most valuable think in the article for me was the picture of integration costs by phase. Go check it out.

The REAL costs of integration - ZapThink

Is complexity justification for itself?

Have you ever had this conversation?

"Why is this so hard?"

"Because our environment is so complex."

"Well, why don't we just use [INDUSTRY STANDARD XYZ]?"

"Because our environment is so complex; in order to support the pieces we have already we need to take [COMPLEX SOLUTION ABC]. The simple approach simply would not work."

"Wouldn't it be better to take the simpler approach and adjust the current complex components?"

"We can't, we have always done it this way."

Etc. Etc. Etc.

I hate that conversation. It get's very old.

Complexity does not justify further complexity. It may be hard... it will be hard to get out of that mode of thinking but it is necessary. While a complex environment may exit it doesn't mean that industry standard approaches won't work. It is my conjecture that complexity has become cultural.

Simplicity is the ultimate sophistication. - Leonardo daVinci

Simplicity is hard. This can be seen if you look at code from an experienced developer versus a new developer. Generally speaking the code will be more readable, more understandable and simpler. (Even though the algorithm may be more complex the elegance of the implementation makes it easier to understand.)

I will end with this quick request. If you hear someone using complexity as justification for further complexity call them on it. Press to see if a simpler approach is possible. It may be a little harder in the short term but in the long term it will pay off greatly.

Is the question more important than the answer?

Right now I have 4 books I am shifting time between, one of them is "How to Think Like Leonardo da Vinci" by Michael J Gelb. One of the items brought by my Gelb is curiosity as one of his seven steps to thinking like da Vinci. In essence this particular step is all about asking questions. It is an interesting thing that as we grow and are educated we are thought to not question but rather to answer.

Gelb states, "In most cases, schooling does not develop curiosity, delight in ambiguity, and question asking skill. Rather the thinking that is rewarded is figuring out the "right answer" - that is, the answer held by the person in authority, the teacher."

In school we learned that the answer was more important than the question. Not just that, but that the answer that is desired by the prevailing powers that be is the answer we want to give. If we don't then we don't get good grades, if we don't get good grades we don't have as many options, if we don't have options we can't make as much money, if we can't make as much money then we can't have as good of a life... or something along those lines. People get caught up in trying to give the answer that their boss want's to hear. Or not asking the questions that they feel should be answered simply because that is how they have been trained, or that is the prevailing culture of the company.

What get's lost in the shuffle is the desire to question. The Wonder of the Universe. We all think of small children and asking why. For those that are not parents kids don't ask the questions you expect. Sure they ask why the sky is blue... but they also ask if everyone sees the same colors, how the sunlight travels if it doesn't have a car, and random things that simply don't make sense to the educated and it would never even have occurred to us to ask.

As I think about Engineering I think that this curiosity and desire to ask questions of what if and how and why are key to a successful engineering approach. In order to build effective systems we need to understand how things operate, what happens when something outside of the expected happens, how the system will respond to it, etc.

From these questions we come up with what we are calling Operational Models that show the threads, the Pressure Points, the potential hot areas of any application. From this we can plan capacity, plan protection and plan impact to downline and upline systems.

So, here is an exercise for you. (Borrowed freely from Gelb) Take a piece of paper (da Vinci. always kept a notebook or note pages with him, now called the Codex Arundel. Bill Gates paid 30 million dollars for some of them, the Codex Leicester... who knows, if you keep one maybe it will be worth millions some day) and write 100 questions on it. They can be about anything. Just let your mind wander. Then look for themes. Then explore those themes.

Sacred Values, Organizational Hypocrisy and Innovation

I recently read a Cutter Edge email (if you are not familiar with Cutter go check them out) that I thought was very interesting. In case you haven't been keeping up with the news Toyota recently passed Ford in the list of World's Biggest Auto Makers. They are also putting focus on GM and trying to get to the number one spot. The Toyota Lexus brand recently took the top spot in 11 of 19 vehicle categories. The reason that Robert Charette wrote the Edge article is not all of these wonderful accomplishments but rather, what they are doing in response to a challenge.

Toyota has recently had some quality problems manifesting in an increase in recalls. The company has a choice with their recent success to continue to press on and work through these issues as time allows or to slow down and put focus on ensuring they are addressed. Given that they have come so far and are so close to their goals it has to be tempting to say that these are just short term problems and to press on as most any successful, large sized company would and even the public markets would expect.

But they are not. Toyota has given an apology for the problems and stated "There can be no growth without quality improvements." They have even assigned executive VP Akio Toyoda, grandson of the company's founder to be responsible for quality improvements. As part of their drive to increase quality Toyota has started several initiatives ranging from adding systems to track repairs to cars even after the warranty expires to hiring more engineers.

You read that right, in this age of continuing layoffs they are hiring more engineers.

As the company grew they added new systems, CAD tools, that would increase engineer productivity and reduce the number of physical prototypes needed. Toyota is recognized as the creator of Lean processes and manufacturing (and it is interesting to note that the problems are not in manufacturing, but in Engineering) However, there are limiters to what tools can do and with the increase in the number of initiatives this meant fewer eyes were cast on each design and "boneheaded mistakes" found their way through the system.

Robert discusses these items in the context of corporate risk management but also mentions Sacred Values and Organizational Hypocrisy. These are when a company says that something is important to it and then does not act like it in reality.

I am encouraged by the fact that companies are having new and creative programs such as Hack Day and other things currently under way in companies. To me this shows that at least some companes take innovation seriously. The question though is, do we take it seriously enough? Many of us are already working well over 40 hours a week on projects and problems that have been funded. Finding the time (and energy) to be transformationally innovative, not just iteratively innovative is hard to do and rarely encouraged.

The real proof will be when something difficult happens. Innovation is, by it's very nature, disruptive. Especially in large companies with business to protect. Read a recent blog post on how "Survival is the Mother of Invention" on the difficulties IBM went through for a very real example.

Engineering Poetry? Limericks? Haiku?

Poetry for Engineers? I know.. it sounds a little out there and I am not just cheating and saying an Engineering degree from the University of Limerick. This was just too good of a follow up to the whole idea of there being other non-technical abilities that make someone a better programmer or Engineer. I encountered a news article on how Georgia Tech is now teaching poetry to their Engineers. By making this part of the curriculum the school is trying to teach students how to leverage their full mind and potential. One of the analogies that was used when describing good poetry could also be applied to good code.

"If something is good, it will stick around. That's just a fact of history. If something is not good, it will dry up ... "

Long, long pause for dramatic effect.

"And it will blow away."

Nicely said professor Lux.

The other aspect of this that I think is interesting is along the same lines as the recent flurry of music posts. Poetry is historically a right brain activity with some left brain rules applied for form. When done well it is a blending of the whole brain to get an answer. As pointed out by Dan Pink in his excellent book Whole New Mind the ability to use both halves of our brain is now required to be successful. If you are a rule and "left brain" only type of person... eventually your job will be automated or offshored. But if you can utilize your whole brain there will ALWAYS be work for you.

Consider this the next time you have to clean your toilet. (I know that seems like a sudden context shift, but humor me for a moment) Your toilet brush is no longer just a toilet brush. That Target toilet brush is not just a toilet brush, it is a designer toilet brush, selected from over 5 different hues and tones from shiny red to stainless steel to be uniquely you. That is how much of a world of abundance we now live in... we now include design in our toilet brushes. (No I did not enter a contest to see how many times I could put toilet brush in a post... but once I got started I just couldn't stop) Our parents and certainly our grand parents were thrilled if they could own one home and one car. Now... many of us have multiple cars.

Consider these things as you come up with your next personal development plan. Are you just looking at adding another language? Are there other things that you could take on to grow into a more well rounded and whole brained person?

You have to feed a lion to know how hungry it is

I am reading a book (Competitive Engineering by Tom Gilb) which has a lot of information on requirements as well as systems engineering. Although the book itself is a bit of a Tome (read as difficult to get through in spots as it pontificates on things I (and most of the world) don't care about) it has some bits that are wonderfully insightful and keep me reading. I thought I would share a few of what I thought were fun quotes and then expand on them a little.

You must feed a lion to know how hungry it is. In the software world this can be stated as unknowable complexity, often described with the phrase "we don't know what we don't know". In essence what this is saying is that you can't have perfect knowledge up front about exactly what requirements are going to be needed and how exactly things should work. In Agile we address this by defining the set of stories then prioritizing and delivering early and often. Do the valuable things first. Many projects get their funding cut due to one thing or another. Projects that are delivering value, regularly not only fight the complexity problem by addressing things in bits and hitting the "goodies" first but also pay as they go allowing business to see value along the way thus reducing their chance of a cut. This statement also applies to the use of Spike solutions. If you don't understand something well enough to give an estimate... don't. Do a spike solution, learn about the problem, reduce the risk and make an estimate with more knowledge.

Even gourmet decays. Another way that I have described this is with the statement regarding requirements that "it's not that I lied, it's merely that the facts have changed since last I told the truth." Requirements change. Any requirement, as soon as it is completed begins to become less and less valid as the business environment and the world the requirement was created for changes. Again, in Agile we address this by delivering value early and often and only going into the nth degree of depth when it is needed. By working with customers on the most valuable items in the backlog for each iteration we are able to keep our requirements as fresh as possible, work on the "right things" and able to more quickly adapt by picking and choosing the stories that have the most value to us at that time. Recent research has shown that only 20% of the functionality in software systems gets used often or always... lets focus on getting that part done first.

Stop the world, I want to get off. We could no sooner say that we are tired of the world spinning and stop it than we could freeze all requirements up front and hope to deliver something that will truly make our customers happy. In Agile this is again dealt with by working on the most important stories first and prioritizing and selecting the new "most important" stories at the beginning of each iteration. By attempting to freeze all of our requirements and not updating them we would cause far more problems (not to mention churn or analysis paralysis) than if we just updated our requirements.

Fantasy Sports

I have now officially fallen into the psychotic reams of people who are participating in Fantasy Sports. It started innocently enough with a begged request from a friend at Church who needed "just one more person to have an even number". As I started exploring just what I have signed myself up for I ran into an interesting example of what I posted about just the other day.

Hockey has always been my favorite sport. In part because the whole idea of Hockey has always been about the team to me. You can have one guy who is super awesome and he will be fun to watch but it takes a dedicated team to get the wins. As Ken Hitchcock used to say "Hard work beats talent when talent doesn't work hard."

And now... here I am looking at stats as the sole indicator of how good of a player someone is. Where I have always been an unabashed homer who cajoles the call guys for being biased if they say anything negative about my team... now I find myself looking at who is playing who and rooting for individuals so my "team" can win. Before my rules were clear... if they were wearing the wrong sweater they were evil and obviously a cheating fake, now things are more complicated. I used to be able to know they were evil by the Red Wing logo on their chest (until they left as a free agent and signed with our team at which point their sins were forgiven) but now I need to look and see if they are on my roster.

Ain't life fun? Sorry for this non-technical ramble of a post but I just found the whole thing particularly humorous given how quickly I find myself living contrary to my own post.

Are we a world of individuals who don't understand teams?

The USA lost to Europe in the Ryder Cup this weekend. According to the news here in the states between this and the loss in World Basketball competition we have too much focus in the US on individual success over team success and this is hurting the U.S. in international sports. This has even been observed in the UK press. (By the way, congratulations European team, well done)

Why do I bring this up? Especially when you consider I don't watch golf (I enjoy golf but my game is one that makes other's feel better about their game) and I have an actual spoken distaste for basketball. (They just mess up my ice no matter how amusing Cuban is.) It's not because of a sudden care about international sports. It's because I thought it brought up some interesting points about teams and teaming.

As a race of humans we have not always been about the individual. Michael Gelb (yes, another mind mapping guy) point out in an article he wrote for Create Magazine that "Prior to the Renaissance, the notion of individuality didn’t exist, because the concept of individuality, as we now understand it, didn’t exist. Paintings, for example, remained unsigned, and painters, anonymous, because the individual was considered unimportant. All creative power was vested Above."

We all have a desire to do our own thing. Be our own person. Contribute on our own merits. Make our will known and get the recognition we deserve. etc. etc. etc. I don't think that in and of themselves these are bad things. They are probably good things. But teamwork is also something that is critically important.

Teamwork is the process of taking a group of individuals and making them capable of more than they would be capable of by themselves. Some of the most valuable individuals in any group are the ones who make that synergistic bonding possible. On sports teams these are the leaders of the team, the ones who inspire everyone when things are going poorly. In business they are the people who others watch and take lead from. The positive (and sadly sometimes negative) influence that raise everyone's mood, performance and vision.

It takes a bit of self-sacrifice to put the team ahead of the individual. This stems from recognition and a desire to personally excel. In an authoritative "Boss, Rules, Controls" culture individuals are actually disincented to promote the team. That is why we all need to encourage and "be the change we wish to see" (Mahatma Gandhi) when it comes to teamwork.

Do you know someone who helps other people around them become their best selves? Talk to them today. Say Thanks.

Tenets of good SOA programming

Today's post is a follow up to my post on complexity being a math problem. SOA programming just like Object Oriented programming has a select set of steps and principles that need to be followed in order to ensure the most effective use of these technologies.

Most programmers can spit out the rules of good Object Oriented design in their sleep. These same principles are very similar in my opinion to the tenets for good SOA design. OO Tenets are Abstraction, Encapsulation, Inheritance and Polymorphism.

• Abstraction means ignoring irrelevant features, properties, or functions and emphasizing the relevant ones… relevant to the given project. This also covers Data abstraction simplifying.
• Encapsulation means that all data members (fields) of a class are declared private. Local use methods may be private too. The class interacts with other classes (called the clients of this class) only through the class’s constructors and public methods.
• Inheritance states that a class can extend another class, inheriting all its data members and methods while redefining some of them and/or adding its own. Inheritance represents the is a relationship between data types. (e.g. a FemaleDancer is a Dancer)
• Polymorphism ensures that the appropriate method is called for an object of a specific type when the object is disguised as a more general type. (e.g. Square being used like a shape)

To keep things simple and play off the magic number of four tenets these are

• Boundaries are explicit
• Services are autonomous
• Services share schema and contract, not class
• Compatibility is based upon policy

Boundaries Are Explicit basically means that a caller of a service need only (and for that matter, should only) understand the interface to a service. It does not need to know what is going on behind the scenes nor should it care. This enables the implementation to evolve over time without requiring changes in its consumers. This abstracts complexity and is core to making SOA work.

Services are Autonomous is setting up services so that they are independent and able to function on their own. While a set of services becomes part of an orchestrated whole and thus build a system they should be designed in such as way as to be as self governing as possible.

Services Share Schema and Contract, Not Class accessing systems should care about the data and schema that are received and the data that is required for a service to be used. This should be viewed as a contractual interface between the caller and the service. Callers should not expect a Class file from that service as services are about data and schema not specific implementations. When integrating systems and performing remote calls the expectation and integration at the class level is fine, but it is not a service.

Compatibility is Based on Policy means that functional compatibility needs to be managed. Versioning of services and ensuring that they are backwards and forwards compatible is a wonderful goal but may not be fully achievable. This needs to be accepted and managed through policy.

Looks like there is ample ground for posts on this topic. What do you think? Do these Tenets make sense? Should there be others?

Why Open Source Will Rule the Software World

About five years ago a CIO that I worked for once told me that eventually all "for profit" software companies would be overrun by Open Source software companies. I thought he was absolutely crazy at the time. Sure Apache had made in roads in regard to the web server market share at that point and Tomcat was doing ok as the reference implementation for Sun's Java servlet container but... come on. But I did stop to think about it... This guy was a retired partner from a major consulting firm who was really working because he wanted something to do and had proven on many occasions to be a very insightful and brilliant man. Now, looking at the Open Source software world I am convinced once again that he may have been crazy... but in that "there is a fine line between genius and insanity" kind of way.

It looks like even world beater Cisco may not be safe from the pressures of Open Source. A project called Vyatta is picking up steam and really looks like its is going to bring some disruption to the otherwise boring networking gear market. Why is this simple little product disruptive to a 5 billion dollar industry? Well... its an open source router... with all the features of high end network equipment... at one fifth the cost. This product is currently in beta but due for release soon. In an recent article in Business 2.0 it was explained that this project was started because researcher Atanu Ghosh was studying the future of broadband and knew that to make changes to router software, he had to submit them to Cisco or some other slow moving network leviathan. So he and some colleagues decided to write their own, caught the eye of an early Cisco employee and some venture capitalists (there is an interesting note in and of itself... venture capital firms are now funding open source software companies...) and viola. This will be a fun one to watch.

Bernard Golden had an interesting series in his blog on CIO Magazine's blog site about why open source is not only a safe choice for companies but the smart choice. In the latest installment he takes an example of a company with 32 million in revenue that has chosen JBoss as their application server. They could have chosen BEA but instead of paying 500k they paid 50k. What happened to the other 450k? That 1.5% of their annual revenues would instead be able to go into product development, salaries, or any of the other things a growing company needs.

What has made this shift in perception and belief from open source being cool but for companies who could risk their own support to software that is available and real for everyone? My opinion is that the ecosystems for open source has changed. Companies are seeing both the financial benefits of open source. Really, off the shelf software has always been a strange model with all costs in the construction and after that it essentially being free. (maintenance, support etc obvious exceptions and in custom software where the majority of the costs actually end up) Now you can download the software, see if it works for you, if it does you can get support from third parties if it is needed or even participate in the open source community as a company for the support you need. (Who would you rather have answering your email for support, a call desk person who knows what's in a system from the manual and has a series of steps to obfuscate real help or a developer who develops code in the system as part of their job?)

Symphony separates good developers from great developers

I have often observed that some of the best developers I have met have a background in music. I have done a bit of thinking about why that could be and I have a postulation.

When you learn to play music you learn how one item by itself can make music. Closely after that you learn how adding additional instruments adds a whole new aspect to the music. Then, when you add more instruments, of different types you get yet another aspect. One after another the symphony builds the whole becoming greater than the parts.

I think that it is this appreciation for the connectedness of seemingly unconnected items that makes developers with a music background able to make better code. Of course just as not everyone who has ever picked up a violin is a stellar developer the understanding of symphony is not limited to those who have played an instrument. The interrelation of things is something that anyone can learn and observe.

When you are going through hell keep going

I heard a construction worker make this statement and it struck me as so profound that I thought I should write a post on it. It is often said that the hardest part of any journey is taking the first step. This can be seen in almost every approach to time management.

Step 1 - break the problem into small, easily consumable chunks.
Step 2 - get started.

But into every perfect plan some problems will fall and how we react to these problems defines who we are. John F Kennedy once said "History will never accept difficulties as an excuse." So we need to make sure that we adapt, move forward and keep things going. In business and life you are not successful if you try really hard and have good excuses why you couldn't get what needed to happen to happen.

If you think about it, the people that we all like to work with the most are the people that have positive energy. They know what they want to get done and choose to make the best and most out of every day. They are grounded people who know who they are and are determined to meet their goals and they can be anywhere in an organization..

I am a big believer in the sentiment of if you never break anything you are not doing anything of value. A good chunk of the long term term success of that statement though is that you don't give up. Thus the profoundness of the statement... if you are going through hell you keep going. To me this equates to sticking to your guns and not giving up because things get hard. It doesn't mean that working 120 hours every week is ok. Sustainable pace is necessary and we need to plan the right resources, phases and release plan to make sure things go right.

SOA Day 7 - From Object Oriented to Message Oriented

Day 1 - Connections = Cost to Connections = Value
Day 2 - Function Oriented to Process Oriented
Day 3 - Build to last to Build for change
Day 4 - Prolonged Development to Incremental Deployment
Day 5 - Application Silos to Orchestrated Solutions
Day 6 - Tightly Coupled to Loosely Coupled

This is the last in my series of mental shifts. Hopefully people found some value in them rather than me just preaching via electrons. I truly believe that if we take these simple shifts to heart we will have better overall systems not just a good SOA architecture. That said, on to today's post... shifting our mindset from Object Oriented to Message Oriented. This is not to say the Object Oriented programming is evil but rather SOA is an offshoot from Object Oriented development focused on integration of components and communication between them and as such needs to be approached differently.

As we start to change the other aspects of our mindset we will be required to change our thinking from specifics of application development to how application components will communicate and connect. Since we will have loosely coupled the application components we will need to think about how we will message between them. In order for orchestration to work we need to know what will be sent in the communications between components. Since we will have components that will evolve incrementally we will need to build messages that will evolve gracefully as things change.

By shifting this component we will be able to safely perform actions that before required synchronous communications to those that are asynchronous. With this shift we will be able to take advantage of queues, allowing our application components to consume and compute as fast as they are able. We will be able to maintain protections of components and between components because it will be built into the architecture.

Obviously none of these things are silver bullet's but are again steps towards allowing higher level computation once again. Just as Assembly gave way to C which gave way to C++ then to Java as languages so too will our application communication metaphors continue to evolve.

SOA Day 6 - From Tightly Coupled to Loosely Coupled

Day 1 - Connections = Cost to Connections = Value
Day 2 - Function Oriented to Process Oriented
Day 3 - Build to last to Build for change
Day 4 - Prolonged Development to Incremental Deployment
Day 5 - Application Silos to Orchestrated Solutions

Today's mental shift is one that I have blogged about before in other forms; shifting our mindset from Tightly Coupled to Loosely Coupled. Coupling is another word for dependency, specifically the degree to which a program module depends upon other program modules. A certain degree of dependency is obviously going to occur within programs that actually provide value. The key is to make it no more than absolutely necessary.

This is another point that builds off of Day 3. By loosely coupling code we will be better able to "roll with the punches" or shift along with the requirements as they change. Good architectural design abstracts out the bits that don't need to be specific and keeps the coupling from being overly tight.

With SOA this is even more important because one of the classic issues that is seen with services is versioning. One version changes just a little bit and causes a ripple effect. There are patterns and best practices that mitigate this and... what do you know, they are focused on reducing the coupling and providing loose version ties.

Libraries and functions become services in themselves. Using orchestration to pull the pieces together we are then able to adjust simple properties rather than needing to rewrite and recompile code. By keeping the coupling loose systems can use underlying technologies that work the best for the application instance at rather than forcing separate components to be tied to the exact same approach. Of course, for all of this to work a shared semantic framework is required to ensure that messages have consistent meanings across them.

Day 5 - Application Silos to Orchestrated Solutions

Day 1 - Connections = Cost to Connections = Value
Day 2 - Function Oriented to Process Oriented
Day 3 - Build to last to Build for change
Day 4 - Prolonged Development to Incremental Deployment

My next platform upon which I am pushing for mental shift is fundamental to several areas both organizationally as well as for projects and management. This shift is from Application Silos to Orchestrated Solutions.

It is tempting to think about applications as separate silos of specific functionality focused on particular business value or individual function type. However to really take a step forward we need to shift our thinking. Components that do searching require input and provide output. Components that control inventory take input and provide output.

If we simplify the components to their base functions and then orchestrate these solutions we enable true reuse. Components can be very specialized and it is the hooking together of the various pieces that will enable the overall solutions. Orchestration solutions are everywhere these days. Whether it is an Enterprise Service Bus or just simple wiring together of services (personally I think in many cases an ESB is overkill but as always, it depends upon the situation. Heck, there is even Open Source ESBs out there now such as Open ESB on java.net, Mule and ServiceMix from Apache as well as the zillion dollar solutions.) pulling the pieces together allow for complete solutions to be built.

This type of change is difficult to implement because there are many things that need to be specified, handled and even governed in order for it to operate smoothly. For example, when a standard messaging type or service setup is created in order for these shifts to work consistently everyone must adhere to them. You can't have someone going off and building a different service approach because they don't like the current one or didn't feel they could comply and hit a timeline.

The benefits of this approach though are many. You can really take advantage of reuse with this since the components by definition are reusable. Data that would otherwise drift to slightly different implementations behind different systems is kept consistent. The ability to get things done with a controlled budget is enhanced simply due to the reduced maintenance. Maintenance consumes most of any system budget, ironically with a successful system the longer it has been around, the more the maintenance costs become.

So while this shift requires a lot of effort, if the standards can be created and adhered to the value truly comes forward.

SOA Day 4 - Prolonged Development to Incremental Deployment

Day 1 - Connections = Cost to Connections = Value
Day 2 - Function Oriented to Process Oriented
Day 3 - Build to last to Build for change

Today's mind shift is to shift how we develop and release. Prolonged Development to Incremental Deployment . If you have been listening to the Agile speakers, reading software development papers or news or otherwise been paying attention chances are you have heard this one mentioned. This mental shift is one that is under way. Iterative development has been in place for some time and really has good traction.

My point on this one is we can't stop with just iterative development. The best feedback that we receive is when something is in use. In order for it to be used it needs to be in production. So this means we need to develop things in verticals of functionality that allows us to get them out and in front of our customers. A wealth of ills can be addressed by this. It does require a more deliberate approach and understanding of your overall application architecture. With this though better quality can be released and built.

Just like a fractal grows and grows making the picture more and more complete so does a SOA based architecture. Each bit released enables other bits to be built or released. By releasing these bits in a planned manner as quickly as possible rather than taking months to develop the "complete solution" we can steer our development to what is really needed. Just as I said in Day 3 where we Build to Change we can accept that our requirements are going to shift and be ready to adapt them faster. Focus on business value not the "complete solution". Complete solutions are a fallacy because if there is business value in a developed system more development will be needed.

SOA Day 3 - We have to shift our mindset if we want successful SOA

Day 1 - Connections = Cost to Connections = Value
Day 2 - Function Oriented to Process Oriented

Continuing the series of mental shifts today we will cover how we build things and the development approach needed.

From Build to last to Build for change. This is another one that people struggle to get out of. It is tempting to think that we are building a system or component that will last forever, that it will not need change. To support this we tend to build in "what if's" and architect a solution until we are blue. What we really need to do though is focus on building in abstraction layers and flexibility, accepting that the system and requirements are going to change.

An interesting item in architecture is that those requirements that have the most churn about them are the ones that need the most flexible designs. A good portion of this flexibility can (and should) be built in through abstraction. If it doesn't matter what whatchamadigger you have behind the library because the library is generic then it gives you the ability to change along the way. This goes for both custom/internal code as well as external components.

A good example of this work the various MOM efforts everyone seems to be working on. Currently they are working on building abstraction libraries for several potential message queue systems. Websphere MQ, JBoss Messaging and SeeBeyond are all potential solutions and rather than building to one specific solution a better approach is to build interfaces that abstract out which mechanism is used behind the scenes. In this manner the teams who are doing the integration don't need to worry about the plumbing and can focus on the application logic needed to differentiate applications instead of the things that should "just work". Another major benefit of this approach is that it frees teams to move to new and different technologies with minimal rework and this is where real negotiation power with vendors and true flexibility in technology by avoiding lock in can be found.

SOA Day 2 - SOA Mindshift - Function over form Process over procedure

Day 1 - Connections = Cost to Connections = Value

Day 2 in the series of SOA mindshifts we need to make in order to ensure a successful SOA architecture.

From Function Oriented to Process Oriented. Thinking about functions is usually easier since that is what we break things down to from a development process. The subroutines that pull together a function makes it possible for us to take things in smaller chunks. Functions though are not what provides a business with value. Any paticular function is only valuable if it enables the business processes in which it sits. If we can make ourselves think about the process rather than sub-optimizing on the functions we can drive a better overall solution because we see the bigger picture.

This point hits many different areas. If we consider logging as an example we would rather know that a process path for booking has gone down than that a specific function within that path has gone down. With the information at the business process level we are able to know what other business processes are impacted. As technologists we want to know what piece is breaking but really that is part of the detail of fixing the problem... the business wants to know impact. Other areas are similarly impacted.

Another benefit of this mindset change is that we will naturally start to take a few steps back when modeling out problems. In the "can't see the forrest for the trees" sense it becomes easier to see the forrest because we are looking at the process and how the trees interact rather than studying the bark and how it is brown. Focusing the whole picture is important for many reasons.

Mindsets must shift if we want successful SOA

In my next series of posts I will propose a series of mental shifts that must occur in order to allow effective SOA to become a reality. Each of these will be proposed in a From X to Y manner over the next several posts. Let me know with a comment if you if you think I am crazy or if these make sense.

From Connections = Cost to Connections = Value. For a long time we have had the mentality that additional connections drive cost. In an SOA environment this changes. The more connections we have the more valuable our overall solution becomes. (My informal polls tell me that this is the one that the most people struggle with switching).

This point can be seen in my earlier point on complexity being a math problem. The more and easier it is to connect the more viable business models, execution methods and doors open up. Coase Theorum has been hard for some to accept as meaningful for technology conversations because they struggle to see a connection as anything other than a cost. In the mainframe world where there was a limited number of connections this was obvious. Now that connections are as ubiquitous with the proliferation of the intenet and IP connectivity the equations have changed.

A gap between stimulus and response

Not to long ago I was introduced to the concept of The Singularity. The idea behind this is that eventually computers will improve to the point that they can be smarter than human beings. Essentially this is when computers reach the point that they can improve themselves. The idea of the singularity is that at this point it becomes impossible to tell what will happen next. A similar theory called the "empty planet syndrome" involves nano-technology and a similar sort of improvement to a place that we can not comprehend and with a thought we could wipe ourselves out.

Of course there is debate about this and it's possibility. Movies like I Robot or The Matrix and others of their ilk point to a rather scary future. On the other hand though I think that there are several reasons why this won't come to pass.

My favorite reason (yes, I have favorites in my own head) is that computers are like incredibly powerful left brains but they lack right brain ability. Most fundamentally they lack the ability to truly choose their own actions. This choice is why someone who grows up in a family with a long long history of "specific problem X" can overcome their biological programming and choose to become something else. In the end a computer will only respond to the ones and zeros at the heart of it's algorithms. In AI research it is interesting to watch how once a new and great AI algorithm is created... it's not longer AI... it's just an algorithm... just code. Humans have a gap between stimulus and response and thus can choose their own actions.

Computers handle logic and rules and conditions with amazing ease. This is why Deep Blue was able to beat chess grand master Kasparov. While in the early days Kasparov was quoted as saying he would rip the computer to shreds he inevitably lost again and stated that computers would continue to win. Just as John Henry was beaten out by a machine queuing up the industrial age so to did Deep Blue usher in the information age. All of that wondrous tech aside... computers still can't recognize a face. (Algorithms are being built that key off of facial features, voice tones and other previously right brain lock activities and they will continue to improve but the basic limits are still there.)

The principles of complexity

I regularly hear how complex our environment is and how many different systems, platforms, methods and names exist within the Sabre enterprise. Internet standards, specifically those around Service Oriented Architecture (SOA) are an effort to drive a simpler approach to systems and specifically the connectivity of systems. I read an interesting interview a little while back that referenced Coase Theorem regarding the interconnectivity of systems. Ronald Coase won a Nobel Prize in 1991 for the Theorem for its impact in economic modeling and in the interview it was mentioned in the context of connecting systems and that got me thinking. The original notes and thoughts had nothing to do with computer systems but the parallels are very interesting.

A simplified (grossly; My apologies to Mr. Coase) view of the theorem is that when the cost of integrating processes is high then it makes more sense to do things yourself. As the cost of integration is lowered the economics change. If the transaction costs can be modeled then you can run simulations and determine the best possible solution for your own approach. This means that as network communication costs get cheaper and cheaper and the cost of information is driven down that new models and approaches suddenly become viable that may not have previously been possible. Take the well known example of amazon.com. Initially, all Amazon did was take free catalog information, used it to build an online catalog and used the internet to make sales at a discount since they don't have the traditional overhead of a brick and mortar store.

Even following Coase Theorem there are three main factors that inhibit efficient allocation.

• Imperfect information - If you don't know the costs of the transaction or it is uncertain or unstable it is difficult if not impossible to model
• High transaction cost - High transaction costs is the core of the theorem and driving down these costs becomes paramount to flexibility within the model. If any one item within the formula is large it can dominate the costs of the system.
• High negotiation costs - If the costs to negotiate the transaction are high it quickly erodes the value of the external connection. This could be due to too many parties in the negotiation or the overhead of the conversation to establish the connections.
• A fourth item, not core to the theorem that is used to argue against the ability to effectively model the costs is in cases where the actors in the system have irreconcilable different valuations on the transaction.

SOA is intended to be an enabler to drive down the cost of connections of systems. By making integration and calls between systems as simple and cheap as possible it enables a whole new way of building systems. More focus can be placed on the Business Logic layer and the orchestration of the services to unlock business value. As services are built, focus can be made to make the individual services as efficient, cheap and scalable as possible. The cost of building new systems goes down as well enabling new and more creative businesses to be explored.

In order for this to work though there is a few big simplifying assumptions. One, that the cost of integration really is lower. This doesn't just cover the development costs that are immediate and visible but also the long term costs and benefits. Maintenance should be cheaper if services are specialized and localized. For Maintenance to be cheaper services need to be stable. This speaks to the fourth item of irreconcilable different valuations. The best services (cheapest in terms of this current discussion) are the ones that can be taken for granted. Perfect availability is the goal, just like with phone systems you don't pick up your handset hoping that there is a dial tone, you expect it. For computer systems that doesn't mean that you shouldn't harden your own system to protect itself from harm because you expect other services to be there. Quite the opposite is true. The phone company gets their "always on dial tone" through A LOT of redundancy, automatic system corrections and advanced error detection and response frameworks. ( Mary Poppendieck discusses in her Lean conversations how testing to find defsfects is waste while inspection to prevent defects is essential) These same concepts apply to good software design. It is also important to consider the cost of the negotiations. In the case of
systems that means that getting your connection and utilizing it needs to be cheep as well.

This post is beginning to get a little long so thanks to those of you who rode along with me this far. I will follow up in subsequent posts on the Tenets of good SOA design and what we can do to approach our own level of dial tone.

Vision without execution is daydreaming

For those who haven't been paying attention to the news, Bill Gates has decided to take a step back on his active involvement in Microsoft and increase his focus on his foundation. You can say what you want about Microsoft and Gates' business practices or technologies but one thing that is undebatable is Gates' effect as a Technical Leader and visionary.

In one of my personal favorite leadership definitions Steven Covey defines leadership as communicating to people their worth and potential so clearly that they come to see it in themselves." A great quote, similar in power to today's lead in from Bill Gates that "vision without execution is daydreaming". Bill has been a galvanizing force for Microsoft and the technology industry as a whole for some time and while it is doubtful that his decreasing involvement spells the immediate downfall of Microsoft it certainly indicates change is coming. Ray Ozzie is another great technical leader and visionary, the father of Notes and then Groove Networks he has a definite track record. But when it comes to attracting and retaining talent, even my Mother knows who Bill Gates is.

Technical leadership is an important aspect of any company. There needs to be a Yin and Yang relationship between business and technology. If that relationship gets out of whack in either direction a companies ability to execute suffers. If the business has too strong a voice then things like deadlines and functionality get pushed. If technology has too strong of a voice functions that may not be long term valuable get pushed, beeding edge technologies and increased risk are pushed. In reality there needs to be a set of trade offs between both sides to get the best possible solution.

This active conversation between business and technology is one of the big reasons for agile methods. By going through stories for prioritization and assigning costs for each along with a fixed velocity a team can concentrate on delivering the most reasonable business value in each release. Research from The Standish Group has shown that only 20% of functionality that goes into a system is Always or Often used and the majority (64%) of things that are created are Never or Rarely used.

Wikipedia defines engineering as:

Engineering is the application of scientific and technical knowledge to solve human problems. Engineers use imagination, judgement and reasoning to apply science, technology, mathematics, and practical experience. The result is the design, production, and operation of useful objects or processes.

I think this is a great definition as it doesn't dig into the specifics of implementation but rather focuses on the what, why and desired outcomes. As we look into the engineering discipline we find that a heavy dose of imagination and creativitiy is required to come up with quality solutions to technology problems.

One of the items that this definition brings up is intuitive problem solving. This is the idea that each individual will have a set of experiences that they apply to intuitively understand and resolve a problem. I see this sort of problem solving regularly within our my own teams. As we dig into a solution, individuals raise questions like "what happens when one of the nodes goes down, sure the solution will fail over but can the remaining node handle the reconnect traffic?" What this usually means is that they have successfully broken that paticular item in the past and bear the scars to prove it. Technology Scars = Better Intuitive Problem Solving. So then the question becomes what can we codify to accelerate that experiential learning curve for other people?

One interesting thing I have observed is that many brilliant technologists that I have run into have had a background in music. It's almost as if having played an instrument opened up certain portions of the creative brain and enabled them to think differently about problems. Your brain is like a muscle after all and if you don't excercise it it will atrophy. Creativity in concert with logical problem solving seems to be key to approaching any complex problem. If I can divide a problem in half, determine the side with the issue and then devide the identified half in half and do it again I will eventually find the source. What I think this means is that good engineers need to have a balance of left and right brain.

Welcome to my little corner of the Internet. Where I can conjecture and cajole and care not a wit if anyone else reads, feels or deals well with anything I say. Ain't the Internet cool.

Just to set expectations (other than my own) I will blog about

• technology stuff - I am a geek... I like to think of myself as a high end geek... but that's also sort of like saying I am white trash with money
• Scotch - It could be that the tech sector has driven this desire to drink into me, but either way it's something I enjoy
• Agile stuff - Sort of related to the technology stuff but a bit more on the fluffy side
• Miscellaneous - whatever doesn't fall into the above categories... I refuse to contain myself.. it is my corner of the Internet after all... here I am LORD OF THE LITTLE PEOPLE
• Meaningless information - I am frequently told I am a wealth of meaningless information as I read near constantly, remember random (importance debatable) items (though names elude me) and otherwise chase bunny trails of conversations (I have been accused of maintaining 2-4 streams of thought conversations at time with a single person... apparently not everyone enjoys this :) )

Anyway, stay or run screaming at this point this post is done.