Chuck DeVries - Creative Engineering Blog

2009-05-29T06:52:00.000-07:00

Searching for search and finding Godot

The battle for search eyeballs is heating up again. The model has been shown to work and Google is now Ad Mogul extraordinaire. Microsoft in classic fashion has tried, learned and is trying again. Yahoo has had much publicized struggles. Even new players are coming into the space. The buzz about Wolfram Alpha is everywhere.

What this next level of competition is about is relevance of information. The reason Google initially made inroads with search was their optimization routines and ability to give people search results that were relevant to what they wanted. Recent additions such as the ability to promote and bury links based on your own preference are continued iterations of this optimization. Similarly iGoogle and personalized history of search with functions like returning the link you clicked on last time you searched for something are again continued iterations.

The next generation of search is taking more of that sea of information that is out there and turning it into something useful. This is both good and bad. It's good/great in terms of making things digestible and understandable, especially when you are researching a particular topic. But it is less so when what is important is being determined for you and you loose some of the outliers. Outliers are where innovation happens. This is not to say that results don't get buried with a raw information return because they certainly do but just as with code and layers of abstraction, with great power comes great responsibility and the need to know what is going on behind the scenes. In many cases this behind the scenes is proprietary information and algorithms that companies (rightly) don't want to share as their secret sauce. What a dilemma.

Wolfram Alpha is the new kid on the block and have gotten a lot of Internet love with an effective launch that you could watch in video live. It's cool for checking out what happened on the day you were born, doing research on specific items and generally getting a quick easy view of relevant data to concrete topics.

Microsoft Live didn't really live up to what was desired by Microsoft but they are back and ready for round two with their new search engine Bing. (I thought that was the sound the Southwest ads used for new deals?) Bing is in pre-launch and is supposed to do a lot of what Wolfram does and use other proprietary correlation tools to give better and deeper results. ComputerWorld has a nice Visual Tour of the functionality to give a good idea of what is coming.

That leaves Google. Google's new exciting new news is the pre-pumping of Google Wave. If buzz on the Internet is any indicator there is a lot of excitement and potential here as well. If it wasn't clear why there was rumored interest in Twitter from Google it is now. Wave is intended to be a merge of... well... everything Just In Time.

So, it looks like the observation that the next generation of systems are about visualization and the presentation of information, not just finding it. Boy won't this be fun to watch.

2009-05-11T17:24:00.000-07:00

The Hangout and the Bazaar

I have been more active of late in both the Twittersphere and Facebook universes and have a few observations to share.

Facebook is cool and is a great place to connect with friends. New and old friends all in one spot. The feel is more like a social club or hangout than an internet site. This means that people say things (as many have pointed out) that they might not say in front of their boss, mom, or who knows who else. This has led to the two-face-book approach where people have their friends profile and their work profile separate. This way when you comment on the silly photo of your friend who has had too much to drink it doesn't show up on your bosses page and so on. After all, Facebook is now for old fogies. (I suppose that's me as over the course of a few weeks I had numerous friends ping and prod me to get online as things like school reunions become the talk of the day.) It has the feel of being a semi-private club where what you say is sort of private though you know that it is subject to repeat.

Twitter on the other hand is like a big bazaar on the internet. Everything is there and nothing is really private. There is an ability to keep your updates to only people you approve but it is not regularly done. There is a social etiquette followed regarding followers and following, numbers and post types, frequency etc. This gives a basic structure though there is the constant pressure of those attempting to use things for their own marketing (and annoying) messages. Just search on something current like #Startrek (Hashing is a way to mark tweets for easy search and trending)(The new Star Trek is a fabulous movie by the way, both Campy and High Tech and Throw Back and Modern all at once. GO SEE IT.)

So I am using both. For different reasons. I love connecting with old friends and reconnecting with people I have lost track of on Facebook. I also love the raw feed feel of twitter though. It feels like the early days of Bulletin Boards when you could chat all day with people you didn't know connected only by your interest in computers and ability to type.

2009-04-28T12:33:00.000-07:00

Making sense of nonstructured nonsense - Trend Spotting in the Over-Information Age

As we look out over the web and remember that all of this information at our finger tips didn't used to be there it's a little hard to believe. The current Generation Y has a reason to be asking as this information has always been there for them. They are used to being perpetually connected and living their lives online. Tools like Twitter and Facebook make connectivity via digital methods a default. Connectivity from access devices such as iPhones or something as strait-forward as SMS provide snippet updates and easy push of information. Even Microsoft is getting into the game with their recent Vine announcement. All the while "normal" information continues flowing out. Papers are published and go to the web, news releases feed directly out, blog posts go out every second. Lots and lots of unstructured data every single day.

This brings about the challenge of the next generation of systems. Simple links to information without context aren't necessarily useful any more. When the search engines first came out the ability to point you in the right direction was enough. It no longer is. If a search engine returns a particularly pithy 140 character tweet on my topic I probably won't be thrilled. If it returns a mesh of links based on that tweet connecting with others, blogs and research topics in a neat visualized package... well that would be cool.

For example, I am obsessively (those that know me probably are nodding their heads right now... I would ask you to stop please) interested in ants, papers on ants and recent research about ants. I believe that how they work, organize and where collective intelligence emerges provides us with many great clues on building scalable systems out of unstable parts. But in order to keep up on this I need to keep running searches and queries against various engines, look to known research outlets, receive email pointers from friends and build my own views and filters to what I think is worthwhile.

Another example, twitter and other outlets can give indicators of where traffic problems are. People get stuck in traffic and tweet to their friends they are stuck. What if that information were automatically gathered, from the problem areas themselves, with geocoded info and placed on a google map mashup. Talk about useful. (I will of course conveniently forget to mention the downsides of texting, tweeting, facebooking and driving. Don't do it.)

Visibility to all of this disorganized and free flow data is the next great mountain to climb. Terms such as Crowdsourcing, Trend Spotting, Data Visualization are the buzzwords of the day. For a quick and inspiring note on what you can do with this type of information check out Erik Hersman's Ted Talk. Knowing what is getting buzzed on Twitter and where the collective consciousness is headed is good and in many cases ahead of "standard" news outlets. How to create tools to view this information in a meaningful way is the challenge we need designers and engineers to be working on.

In the meantime if you see good ant stuff let me know.

2009-04-25T09:06:00.000-07:00

I watched an interesting Ted Talk that got me thinking along a couple of avenues. Margaret Wertheim presented The beautiful math that links coral, crochet and hyperbolic space. Go ahead and watch if you have 16ish minutes. I'll wait.

I have mused before that we know less about the universe than we like to think. For simple examples look at things like Dark Matter and Dark Energy... wonderful hypothetical constructs that we don't really understand that make up only 80ish percent of the universe...

When you look at things like science fiction and thin

gs like space travel and teleportation in particular a good amount of suspension of disbelief is usually required. Well, when you look at some of the items that actually occur in nature maybe not.

Margaret does a great overview of Euclidean Space (Planar), Spherical Space and finally Hyperbolic Space. For those that are not visual or auditory learners though here is a quick overview of getting from destination A to destination B in each space.

Euclidean Space (discovered by Euclid (go figure)) is essentially planar space. There is a whole lot of math involved in a full explanation but essentially it is the rule that given a strait line and a point the only line that can be drawn through the point and not go through the line is parallel to the original line and there is only ONE of those.
Spherical Space is a similar math run but can be simplified to say that when you have a line drawn in spherical space and you have a point there are ZERO ways to draw a line that doesn't intersect from that point and the line.
Hyperbolic space has even more math and if you go back just a little bit in time much of that math is proving that it is impossible. However, nobody ever told choral that so it uses it all the time. Following the line and point theory again this basically is the statement that given a line and a point there is an infinite number of lines that can cross the point without crossing the original. Margaret does a great tactile example of this with a demonstration of a simple crochet piece.

So, that was a lot of words and explanation to say we see this type of space in multiple ways. It is easy to jump to an explanation of "multi-dimensional" space being the answer but think about the web. You can get anywhere from anywhere with one simple button push. There may be a mess of routers and switches that do math in between but to you using your browser it's click and away nearly instantly. As the math for this type of space matures and we understand it better things like "folding space" and traversing large "distances" instantly may become a reality.

What fun to think about. All from something as simple as a piece of choral made out of crochet. Next time you watch grandma spin that yarn just think, she could be paving the way to space travel.

2009-03-18T07:33:00.000-07:00

IBM in talks to buy SUN

Well not two days after I say how their cloud innovations may be too late in coming for SUN... now IBM is in talks to purchase them. How very interesting.

This potential deal is interesting on multiple fronts.

RISC (Not Risk as in risky but RISC as in Reduced Instruction Set Computer) This potential merger creates a big home for the remaining non-Intel chip guys. While HP does have their own this combination of fanatical Sun worshipers and the IBM "tried and true"ers is a potentially strong combo. Very different audiences and users though with one driven by tech innovation in Sun users and the other in the "never been fired for picking IBM" mentality.
Java - Not a lot of the press on the topic brought this up. I guess because it is a shared standard and all that rigmarole. The reality though is that Sun still drives Java in a lot of ways. IBM on the other hand has been a big Java pusher on multiple fronts for development tools, open source etc. This could be a big deal. Of course there is always the gotcha side of the new languages such as SCALA coming up in popularity for the many-core problem.
Open Source - Sun was big big in the .com era and went down when the .com boom went up. Since then they pushed into Open Source fairly successfully and actually have several software products that they are responsible for that have decent success. Grid, MySQL, even Solaris (a religion among some by itself) are big software assets. Of course the hard part is that IBM has their own efforts in these areas.
People Logistics - If Sun has an office somewhere, so does IBM. The overhead portion of this could be a big savings opportunity. Not geeky though so I will stop there.
Competition - Some of the press has stated this is a response to Cisco's entry into the data center space for servers. I doubt that but it does narrow down the big data center players. HP, IBM... ... well...

All told I think this could be good for Sun and IBM. If done right. (isn't that always the trick though... if it works everyone is a genius if it doesn't they were doomed from the get-go.) It gives me some hope for these products and companies for the future and in any case it will be fun to watch.

2009-03-16T09:01:00.000-07:00

More cloud updates

Since my last post I have been sent a few and read a few other articles that I thought might be worthy of note related to what clouds are and why it is important.

Sun's CTO Greg Papadopoulos did an interview with Informationweek and discussed some of the efforts that SUN has going on. One of the biggest items of note in that article is the new Drizzle product coming out of the MySQL work. There are many articles and blog posts on the topic as well but the gist is that it is an effort to Cloud-smarten MySQL. Seeing how "MySQL gets used, people only use a subset of the relational capabilities because it has a horizontal scale and there are all these concerns that go into large-scale deployments. So Drizzle sort of strips back down to a small core and then builds up the distributed capabilities." I have made no secret of my feeling that Data is no longer relational. So I think this is encouraging news though as seems to be the case lately with SUN it may be too late for MySQL.

(Aside... Greg Papadopoulos and David Douglas recently co-authored Citizen Engineer: A handbook for socially responsible engineering. Though it is not yet available you can check out an interview with David Douglas on the book and engineering at the Mercury News. Personally I find this to be a refreshing take on the responsibility of Engineering as a profession and all of the personal and corporate responsibility that should go along with it.)

As if they noticed Microsoft pushing cloud tech Amazon announced a few new items in their own cloud initiatives. The ability to reserve capacity for known needs. They also expanded the ability of Windows based services in another Amazon zone. All of this from a company that sells stuff, proof that necessity is the mother of invention.

2009-03-11T12:12:00.000-07:00

Even Microsoft realizes it’s all about the cloud. (though opinions still differ on what a cloud exactly is… I personally like the blind men describing the elephant/cloud analogy)

Wired did an interview with Ray Ozzie - Long article but worth reading even if you are an ABM coalition member. (I have always thought highly of Ray Ozzie, even if I did hate Notes itself as an administrator and user, it was visionary in its setup and ahead of its time. Heck… Ray even saw P2P coming before the rest of the industry started to catch up.)

Further proof of the gap between the over-served and the under-served. The question is if Microsoft is late enough to the cloud innovation game that their delay will actually cause them harm in market share. They have rallied before and come in with overwhelming response. (e.g. I.E.) The truth is the services such as PPT, DOC etc is so ubiquitous in the work environment it will be difficult to displace them with anything else. The question though becomes more of what will happen to those types of documents and will they be service disassembled themselves enabling other innovation plugs to go in. Also, who knows maybe Microsoft will have the bones to push further change into media distribution and synch. Though Apple has a pretty strong hold and the media providers can’t seem to understand that technology can help versus hurt their business (self destroying DVRs anyone? Eventually a multi-billion dollar DVD business… sigh)

What a great time to be a technologist.

2008-08-12T07:25:00.000-07:00

The Cathedral and the Bazaar and the Ignorance of Crowds

CIO.com recently did an interview with Linus Torvalds and the rules he lays out for quality code development are very good. It goes well with my theory of each development team needing a Benevolent Dictator role.

Some time ago I was forwarded a great article on Innovation as it relates to Open Source. Several of the key observations were saved for the last page. I thought it would have been good to draw some suggested next steps so I thought I took a crack at it and the items below are the result.

The article by itself has a good deal of value. But for complete understanding you probably need to have the previous context (inferred from the article is probably sufficient for basic understanding) of The Cathedral and the Bazaar. Wikipedia has a good simple summary of the paper. At it's simplest though it is a paper written by Eric Raymond about how "given enough eyeballs, all bugs are shallow." This means that it becomes easier to drill to quality because more people with disparate backgrounds means better test coverage and bug finding. (And a lot more but that's not what this post is about.)

My actual observations from the Strategy and Business article: I have historically referred to the needed role to make any product development work, whether Bazaar or Cathedral, as the Benevolent Dictator role. It is someone with the vision and strength of personality to push or pull those around them into a unified functional reality of a product. Without that role you get a lack of direction and vision and problems ensue. We have seen this need prove out over and over with the hardening work that we have done. Those teams that had a Benevolent Dictator that laid down tenets for development, surrounded themselves with a “Wizard Enclave” of smart folks for enforcement and saw it through generally had much better systems. This showed in test coverage, uptime, ability to adapt and change and many other areas.

I am a big fan of Open Source developed products and have been for some time. It has certainly proven to be the case that those that are successful have this Wizard/Leader/Benevolent Dictator that surrounds themselves with the Wizards that get there via meritocracy. From this “Brain” squad comes unification and initial innovation. Ongoing innovation can be pulled from the Chaos of the Crowd but to become real will need to be driven by the Benevolent Dictator.
When it comes to our own efforts, Internal Open Source and Product Development need to be set up with this type of role and if we can’t identify someone to fill this role then we can’t honestly expect to move that product forward.

2008-05-30T14:09:00.000-07:00

Creating a customer obsessive culture

I read a post from Harvard Business recently that was so head spinning I have to share. I have subscribed to the HBR Ideacasts (easy to add to your iPod via iTunes) for a while and listen to them on my commute to work. They are both insightful and quick and easy to understand. I have access to their published components through work and the electronic lead ins provide a great way to focus on the stuff that actually matters.

An example of that is a post titled Why Zappos Pays New Employees to Quit - And You Should To. The idea of giving people a $1000 bonus to quit sounds absolutely insane when you first look at the surface of it. Especially when you consider the high cost of talent acquisition, nut then when you think a little deeper it starts to make sense.

The people who would take such a bonus and leave are people who are asking themselves the question of "what have I gotten myself into". They may be thinking, "this is all well and good but is this really for me?" In most situations when you start a new job you are so focused on pleasing everyone you hardly stop to think if the fit you thought was there really is. As an employer you are also focused on trying to make sure someone fits and gets all the opportunity to succeed they may need. The perception of someone leaving is so very negative that no one wants it.

By taking this approach Zappos short circuits all of those negative items. It becomes not just ok, but encouraged to consider your fit. It becomes not anathema but perfectly acceptable to call it quits when there is not a fit.

It may not be something that every company goes out and does right away (or potentially even should ever do) but I certainly applaud the Zappos team for thinking differently and realizing how to get to their real goal of excellent customer service with fanatical employees.

Brilliant.

2008-05-16T13:25:00.000-07:00

Data is no longer relational

Ok, so the headline may be a little of an over statement for effect. Perhaps I could have said that data that people are interested in is no longer relational but then it wouldn't have been nearly so pithy.

With much respect to Edgar Codd and his invention of the relational model for database storage I think it is time to move forward. Relational databases are great for things like financial models, personnel data and other Enterprise Systems as well as many other standard, repetitive data. It gave a solid reference point to learn data structures and modeling to several generations of budding Computer Scientists.

When I say it is time to move forward I am not saying we should immediately move all data systems to Object Oriented Databases or otherwise induce data chaos. What I do want to push is the idea of unstructured data. Computers are great with rules, with structure and with fundamentally binary relationships. Algorithms are starting to mature around unstructured data (for an example go search google.) but it is still not widespread or well understood.

New exciting algorithms such as Amazon's Dynamo (Werner Vogels is one of my favorite speakers and bloggers on distributed tech, if you are not familiar with him in this space you should be) database are showing in real world situations that distributed systems and distributed data are a reality.

In a lot of system designs because people are so familiar with relational data structures and systems we find Object Models that look like a relational database design. When asked why it looks like this the answers are fairly consistently things like "this is how the database stores it, for speed we need to do the same" or "it just made sense when we pulled the DBA in to help us with the model."

Objects are not relational! They are objects. Then when you get into full structures of objects or object trees there are relationships but it is not the same as a relational database. Especially as we start to build and mature distributed system algorithms it doesn't make sense to use a centralized data store. If the data can be broken up, distributed and stored with the algorithms that will use it performance will improve.

In fact I would argue that the elusive SLA of a system response can begin to be discussed if we can tie the data to the processing. Granted there are new complexities in this model for synchronization, segmentation and consistency but there are ways to solve them. Similarly consistent access to the same servers is also possible.

What other great examples of distributed computing and distributed data storage have you seen?

2008-05-15T09:38:00.001-07:00

A time management observation

Have you ever noticed how critical decision meetings have a tendency to be scheduled at the end of the day? I have noticed this as a trend recently in several of the streams that I am working on.

My first thought was that this was due to the need to work through the day to get the information together in preparation for that meeting. Anecdotally (meaning stuff I have seen, done or messed up myself but may not be statistically significant to the world at large) this seems to be the case in 20-40% of the high criticality meetings that get scheduled. This leaves a gap of 60-80% that could happen at other times in the day.

At the end of the day you are tired. At least (anecdotally) I am since the day has been spent making decisions, digging into problems and hopefully doing real work. So my next thought was why do we push these important meetings to late in the day if we are actually going to be less well equipped to handle them?

The answer (or at least my version of it) came to me as I thought about the tasks list and task list management that I do. Each day I follow the process below.

Review a list of tasks. I assign each one an A, B, or C. This roughly amounts to A - MUST DO, B - Good to do, C - No chance I am getting to this, I could schedule it forward now but then that would take time from getting to the A and B stuff.
Assign a number to each task starting at 1 for each letter category. (Yes, I never even bother with C)

Now when I do this I can say A1 is more important and I should work on it first. Great! Only my human nature kicks in (chances are if you are reading this you have one of those as well. We tend to keep them in closets and they can get dusty but they do peak out at the worst of times) and suddenly I find myself drifting down the list to item A4 or (say it isn't so) a B1... because it's easier. My little human nature side lives for that little endorphin rush of the check mark so... hmmm... off I go.

With meetings I think we do the same thing. We schedule some simple meetings up front. Easy discussions, standing team meetings etc etc. Then we postpone some of those really important, hard meetings and conversations to later in the day.

So I am going to try to move my important meetings earlier. When I find myself setting up one of these meetings I will catch myself and ask the question "would this be better in the AM? Or now?"

Am I nuts? Is this just how business is supposed to work? Have you seen this? In a global economy with companies getting more global in audience as well as employee base these types of behaviors become not only self-pain inflicting but they don't even make sense given global time zones and the fact that the sunlight moves.

2008-02-27T19:57:00.001-08:00

Creation of new is playing with a purpose

The creation of something new is not accomplished by the intellect but by the play instinct acting from inner necessity. The creative mind plays with the objects it loves.
- Carl Jung

I ran into this quote today and thought that it was simply excellent. When the "zone" hits and you are creating anything new, be it a program, a system, art, music or a new business model it feels like playing. When you watch someone who is really good at it, they almost seem to be having too much fun... it can't possibly be work that way right? I have always found so much enjoyment in these types of activities myself that to describe it as anything else is pretense. When your brain is in the right state, you are relaxed, the deep parts of your mind, where the real horsepower is, can bubble up thoughts and put things together that your conscious mind simply can't.

This phenomena was described to me recently as operating with the power in the frontal cortex (the default conscious section) is roughly equivalent to using the change in your pocket. When you are in "the zone" and your deeper mind is able to work it is more like the national product of the United States of America. That's one heck of a difference.

If you never play with your inner child they atrophy. Or potentially worse, get bored and go off in random directions. Kind of like any muscle in your body when you don't use it your mind gets tired too and you can't be as readily be creative... plus personally I think it just makes you prematurely old. If you ever want a dose of reality on your brain aging check out the Brain Age games on the Nintendo DS.

The moral of this post... play every day.

2008-02-22T07:10:00.001-08:00

YAY Input Innovation

There few things in technology that really get me hopping excited any more but every now and then something pops into the tech stream that makes me happy. Today is a happy moment for me.

Input devices and Human Computer Interface design though not something that is part of my core job or direct areas of research has always been one of those items that I have had a lot of interest in. I blame a particularly engaging professor I had in college that did research in this area that helped open my eyes to how silly things are today.

For example, we still use the QWERTY keyboard layout, a model that was built to slow down typing... yes really. On typewriters it was built so that the hammers would hang up less often because commonly used letters were farther apart and people would be slower thus causing less hits. If you are old enough or lucky enough to have had the experience of a typewriter that hung you will understand why this was a good idea at the time. Now critical mass is there and the world is unlikely to change. Just try and tell a Gen-Xer that can type 100+ words per minute on a QWERTY that you are going to change the layout. Of course now Gen-Y is used to a phone and small device without the need for a keyboard at all.

Apple has filed for a patent on a new generation of input device that is focused on touch, write and multi-touch. For more info check out the Apple Insider post for great detail from the patents themselves. One of my favorite features/ideas from the iPhone has always been the multi-touch. In fact part of the reason I expect to be getting an Apple laptop now is that they are adding that ability to those as well.

While we still have not achieved a DNI (Direct Neural Interface) jack yet so all we have to do is think it's only a matter of time. By the way... that really is happening. Controlling your computer with the power of thought is currently focused on assistance for the physically disabled. You don't need to be a dreamer though to realize that the downstream implications are simply fascinating.

2008-02-04T16:17:00.000-08:00

Microsoft bid to buy Yahoo

It has been in the rumor mill for a while now that Microsoft was seeking to buy Yahoo. When there is sufficient smoke there seems to be fire and now the deal is officially proceeding. There are reasons for it and reasons against it. Apparently Live has not pulled in the traction that Microsoft wants and the desire for ad revenue and internet traffic control as a business model is

Google brings up some interesting points on the combination having more webmail and portal traffic than anyone else and Microsofts track record of following the path of Embrace, Extend, Exterminate when it comes to competition. But that input from a competitor is unlikely to bring about any sort of block of the deal. In fact some reports are even suggesting that the deal could be good for competition.

I am not 100% decided on the deal and it's positive or negative effects. Microsoft has a long record of proven execution and when they really set their eyes on something they can make it happen. There is a wide range of incredibly bright people there that when unified to a cause can deliver. The big change that I see though is in release management. Microsoft has a history of classical software development with releases taking years and subsequent patches taking months. In the internet world, and Google is a great example of this, change is constant, the ability to roll out new versions and try new approaches every week is key to success. This will be a big change from the default Microsoft dev approach. Will this Agile development change cross over into Microsoft's standard products? Another one of those much discussed items, SAAS Software As A Service suddenly becomes a lot more of a potential reality. To me this is the real reason for the desired purchase. Ads and Eyeballs are great to monetize and transform what Microsoft has done for a long time.

No matter what finally happens, this aught to be fun to watch.

2008-02-02T15:38:00.000-08:00

You have to pay attention to the periphery

I have long been interested in how the brain works and why we do some of the bizarre things that we do. Add my interest in innovation and viola there is a fun intersection.

Individuals as well as companies have problems spotting, let alone executing innovations. In many cases the innovations seem obvious in retrospect causing people to think "well why didn't ABC co do that? They must be idiots." The truth is that spotting something outside of our current focus is not anything that we are wired to do. In fact we are wired to do the opposite.

For a few cool examples of this check out these YouTube videos. (Normally I would do an embed for your convenience but these are not available for some reason so go check them out.)

Change Blindness; you cannot be aware of everything
Mind's Eye; the Brain relies constantly on Visual Memories

The defense our mind puts in place that keeps us focused and able to complete a task without all of the information available overwhelming our brain is the one that keeps us from noticing things on the periphery. But if the periphery is where the interesting things happen what do we do?

The first step is to be aware of what you are focusing on. The next is to allow yourself time to wander, or in many cases allot yourself time to wander. This doesn't mean daydreaming (remember, vision without execution is daydreaming) but it does mean looking at things outside of the immediate and allotting effort and brain cycles to it occasionally. Probably not at the same rate as your real focus but without carving out at least some you quickly find that your immediate focus is all you have time for.

2008-01-30T19:57:00.000-08:00

Ignorance more frequently begets confidence than does knowledge - Darwin

Have you ever watched on tv or listened on the radio to one of the spots on Stupid Criminals? The premise is those criminals that get caught because the things they do are just plain dumb. For example, here in Texas a man convicted of robbery worked out a deal to pay $9600 in damages rather than serve a prison sentence. For payment, he provided the court a check... a forged check. He got 10 years. If you have time... do an internet search... and laugh.

A research paper pubished in 1999 is very relevant to this topic and I thought I would share. The postulation of the paper is that in cases where one is ignorant, that same ignorance keeps you from realizing your ignorance. The graphs in the paper are very telling regarding people's regard of their own abilities. Even after tests were administered people still felt they were in the quadrants that they were not.

So... why is this relevant in a technically focused blog? I bring this up because if you consider the rate at which technology moves it requires a near constant diligence and research to stay current. So far Moore's law that processor speeds will double and halve in cost every 24 months has tracked fairly close. Software architectures seem to have an average lifespan of 2-4 years. As processors get quicker more and more levels of abstraction are able to be added and new algorithms are enabled. Bill Gates has often been attributed with saying "640k aught to be enough for anybody"? (He denies having actually said that BTW) Now we have services that handle thousands of transactions per second with message payloads of well over 10x that.

The meaning in all of this is that work in technology needs to be one of constant improvement. Both in ourselves as well as our systems. We need to be constantly learning, innovating and expanding. We can't let ourselves individually or as a company become complacent and be ignorant of the fact that we are ignorant. We need to be blissful because we love what we are doing and are growing as part of something larger than ourselves... not because ignorance is bliss.

Assume that your skills need to be refreshed every two years... and act accordingly.

2008-01-12T16:06:00.000-08:00

Wide Area Network --> Home Network --> Personal Network

Everything keeps getting smaller and smaller. A few years ago people wrote off the "crazy people" who started to say that eventually everything would have an IP Address. Well now it seems like EVERYTHING has an IP and certainly everything hits a network somewhere. The internet may not quite be ubiquitous but things get closer and closer every year.

While a personal network used to be the network that you ran at home. Originally for the geeks among us (yes, I have had a network in my house for YEARS), then wireless (as before, first with manual configurations but now available through every retailer under the sun with easy push button setup). Well now it seems like technologies such as Bluetooth and other personal networking technologies have people running their own networks in human form without even thinking about it.

Bluetooth on phones and laptops has been around now long enough that even my mom (well, ok maybe not my mom as she chooses to believe cell phones are evil and voice mail even worse but most moms) can turn on a bluetooth headset and wirelessly interconnect. As other devices hit the market (Watch Gizmodo for a few weeks and be amazed) these personal networks will get larger and more populated. Especially as Apple gets more and more into the general consumer personal device business (this upcoming week's Macworld will be great to watch for just this reason, last year brought us the iPhone... who knows about this year) these things will get even more ubiquitous.

What a great time to be a techie.

2008-01-08T13:39:00.000-08:00

Are you an Egghead?

Having something of a sick interest in the meaning of words, origin of words and their use had me wondering this morning where the word Egghead came from and what the history of it's use would be.

Turning to the Internet source of all random knowledge: Wikipedia, I asked. Egghead has origins starting in the 1950s and is a slang anti-intellectual epithet directed at people considered to be out of touch with reality due to their own intellectual interests. What fun.

It was even used by Richard Nixon in the 1952 presidential campaign to describe his opponent. So it also has a political connotation to it. No real originating source was quoted. The article also mentions a potential start of the description of an Egg shaped head referring to a Bald head. (something near and dear to my heart) Also no mention of Egghead software who changed to newegg some time ago though apparently now egghead.com goes off to Amazon. Not that this is completely relevant but it's my random chain of thought.

Ain't the Internet cool?

2008-01-03T09:06:00.000-08:00

Do you sleep work?

Have you ever gone to bed wondering how you would solve a difficult problem and dreamt up the answer? Apparently you are not alone. A recent survey found that 51% of those surveyed dreamed about work and nearly 70% of those put those dreams into action.

I have always slept with a notebook and pen on my night stand so that I could write down what I thought up while not sleeping or dreamed about. It is interesting to me how many people apparently do the same.

This is of even more interest when you think about it in terms of the "work 40 hours" credo in Extreme Programming. (Really more of a manage your spikes in hours and avoid death marches but I digress). The core of the reason behind that is if you are well rested and fresh you do your best thinking and are best equipped to handle what gets thrown at you.

So next time you have a tricky problem to work out, think about it before bed, write it down as a checklist to complete (leave it on the paper so you can actually sleep) then dream up an answer. (Remember you are still limited by the fact that you won't sprout wings extra heads or anything along those lines so it is likely best to discard those dreams... in fact probably best not to talk about them either.)

2007-12-12T13:54:00.000-08:00

Having forgotten my phone today and not having constant access to email made me remember an article I ran into and had to share. Several key things contribute to burn out but one of the major ones of today's business world is email. So here is a fun little exercise.

Think about the number of emails that you receive in a day...

Now think about your average response time to those emails...

Now think about how many of those emails are really critical, high value things that need immediate responses...

Scary isn't it? Recent findings even shown how the level of access provided by things like Blackberry's and other email devices actually make you dumber by as much as 10 IQ points with close to drug addiction levels.

To address this immediate brain drain one of the things suggested is to set aside specific times to answer emails. Don't let your email be a task list. There are emails that are urgent and there are ones that can wait. Prioritize.

2007-10-12T06:50:00.000-07:00

Friday Link Day

Oracle offers to buy BEA for $6.66 billion. Outside of the immediate 666 references to evil this does have several interesting implications to the software world. It has been no secret that Oracle has been trying to buy RedHat and JBoss and this puts yet another point of pressure in that series of events and the two worlds of Open Source software versus Corporate Sold software.

Airlines agree on check-in by cell phone. The headline alone on this one caught my attention. There are many implications to this possibility. I have long been jealous of the fact that non-US based cell technology is farther ahead and this is yet another item that will drive the laggard gap.

And last links but not least links, here are a few fun links that were shared with me from my JanaRSS feed. Google Smackdown, a query tool to see which phrase or words appears more across the web. Soople - an easy expert search, with a collection of tools in easy format also built on Google tech.

Yay Friday!

2007-10-03T07:28:00.000-07:00

You wouldn't move without a destination for your stuff...

Consider this story. You gets a transfer to a new city. It's a great opportunity, great job, benefits and salary. The new company offers you a relocation package where they will help you with housing, selling your current home and everything. It all just falls into place, your home sells in short order and you are ready to go. You pack up all of your belongings to move out to your new city. Couches, Televisions, Cars, Toys etc. Everything is picked up by the movers put into the 18 wheeler with care and zooms off down the road. You get on a plane and head to your new city, ready to get on with your life. You land and are ready to head to your new home and realize you don't have one. You never set anything up. No new home purchased, no apartment, nothing. Oops. Your belongings have no where to go. You call your company and tell them they need to get you a house and... what do you know... they just laugh.

Ok... so this story is a little strange and something you are probably thinking "I wouldn't do that. What kind of idiot would pack everything, be ready to move and not have a destination?" Not many right...

Actually you would be surprised. While not a house and a couch we get requests all the time for a home for an application. An application that has to be delivered in just two weeks! This is when the inside voice starts to emphatically state "Failure to plan on your part does not constitute and emergency on mine." Of course that is the inside voice and not the outside voice.

What can you do about it? If something is being built, know where it is going to go. Don't just assume, know. Know what the requirements are up front, what the application will run in, on and through. It seems obvious that you would want a planned home for a product but when you get busy coding and building those types of things are easy to loose track of or to assume that "someone" is taking care of it.

2007-10-02T07:20:00.000-07:00

Will there be a claxon when it's time to panic?

When asked what the gist of what we do in Engineering is I generally have to think about it because it differs given different situations. In some cases it's about technology and how it is applied. In some cases it is about ramp up plans and safe rates of growth. In some cases it is about algorithms built for scalability and consistency versus short term function. At the heart of all of these things though is Risk Management.

Risk Management is an art and not a science. If it was easy to tell everything that would go wrong then nothing ever would. So since we don't have perfect information there is a balance that needs to be achieved between safety and progress. We need to look at the risk, its potential cost (PLOP factor) and then weigh all of this with previous experience and make a call to be later judged as a good call or a bad call. Or... if everything does what it is supposed to then it's a decision that just fades into the background.

So when you encounter a risk what do you do with it. It actually boils down into some simple choices. A Cutter article a while ago defined out a basic framework that I wrote on a sticky and refer to now and then as a framework.

Accept it - It's a risk. It's understood. There is not much you can do about it so move on with life and be prepared if it happens.

Avoid it - Sometimes a risk when found can simply be avoided. The ones that I think of in this regard are running a volume test in an overlapping time window with a system change.

Transfer it - This is the get someone else to do it approach. To successfully use this approach the other party needs to be aware that they are getting the risk (no email volleys please). This makes sense when there is someone who is better qualified or has a business to handle the type of thing you are dealing with. It may cost money but mitigates the risk.

Reduce it - This approach is commonly used when it a risk we have to face and work through, but can't directly transfer it or otherwise avoid it. A good example of this is a ramp up plan that is overly optimistic or doesn't account for transition. We mitigate this risk by reducing it and slowing the ramp down in order to make the problems smaller.

While not my own list I have thought that this provided a nice structured way to think through risks and what you need to do. If nothing else it helps in the acknowledgement that there are risks, even if we do choose to not do anything we need that to be a conscious choice.

2007-08-22T14:30:00.000-07:00

The PLOP Measurement

For some time I have somewhat tongue in cheek referred to the way that we prioritize work as the PLOP Measurement method. (Patent Pending). In the unending quest to educate and inform here is a quick definition of the PLOP Measurement Method and how you might apply it to your work.

Contrary to what you might initially think PLOP is not just how big of a splash will something make when it falls into the water. PLOP is much deeper than that. PLOP is short for Prioritization by Level Of Pain.

PLOP has gone by many names, methods and approaches for years. Risk Management, Concern Logging, Caveat List and many more have been used. What they all boil down to though is exposing and managing efforts with an acceptable level of risk.

With the PLOP method issues are judged and prioritized by their potential to cause pain. Pain can be felt in any of a number of ways:

business impact
cost of outage
size of effort
number of systems touched
likelihood of customer impact
potential severity level
inconvenience of time for roll out (tell me you don't look at something that needs to be released in the middle of the night as higher risk than something that can go in during the day)

Some might dismiss this as a subjective measure and make accusations of pessimism and paranoia. While the paranoia point might be correct, I find as a pessimist I am rarely disappointed. It's something that we do whether we acknowledge it or not. It may not be a hard number but that sinking feeling that you get in the pit of your stomach is usually a really good indicator of how bad the PLOP rating should be. The trick is listening to it and taking the right actions rather than slowing everything to a crawl.

2007-07-30T14:23:00.000-07:00

Overtime is a productivity killer

This post may not be one of my most popular with PMs around the world but here it is anyway. Overtime, in the long run, does not help a project. When the numbers are run (and boy do productivity numbers get run...) it turns out that work that is done when mapped out with a scatter diagram across many different projects is essentially the same. [See Slack pg 64, it's cheap in paperback and well worth the read] The work that is done is the same per day... not per hour.

Intuitively this may initially seem strange if not outright wrong. How could a team that consistently works 50 or 60 hours a week end up with the same productivity as a team who works 40 hours each week? The answer lies in the long term effects of overtime, not the short term spike in productivity from a controlled burst. Controlled bursts really can be effective, as long as they are just that, controlled bursts.

Overtime itself is not an evil thing. Many are the stories in corporate lore of the team that pulled an all nighter or pushed through that special effort over the course of the last week to deliver everything on time. Good managers, [project managers, people managers, senior individual
contributors] know when to pull the trigger on overtime to push things over the goal line.

The problem comes because this can be addicting. Both to the manager and the individuals. People like being super stars, like the accolades and recognition of being really dedicated. Believing this to be a great success without evidence to the contrary managers become dependent upon the push to get everything done. Rather than pushing back on scope, push on the people. It is an easier road and you don't have to worry about an unhappy customer. At
least not right away...

With pressure comes a bit more focus. That's a good thing right? Sure, when that pressure leads to cutting out unnecessary steps and trimming requirements that are not needed. But when that pressure becomes an unending stress to deliver no matter what, it starts to become noise.
People stay, because they see others staying. People work, because others work. But the real urgency and productivity goes down. Knowing that they will be there working at night, things get put off, people talk in the hallway, time is spent surfing the web, etc. Why not, they will be there later anyway.

There are of course many other costs to too much overtime, but this is just a blog and not a book, so I will give it a rest now that I have you thinking.

2007-07-26T12:06:00.000-07:00

The principles of Tao Teh Ching in Software Development

You may be asking if I drank a little too much Green Tea to be comparing Software development to the principles of Tao Teh Ching but please bear with me on this one. Honest, it makes sense. (At least I hope it will after I write it down and it's outside of the world inside my head) First, the basic principles. (there is a lot more in the book as its a compilation of a good deal of Chinese Philosophy)

That which remains, is easy to handle.

That which is not yet developed is easy to manage.

That which is weak is easy to control.

That which is still small is easy to direct.

Deal with little troubles before they become big.

Attend to little problems before they get out of hand.

For the largest tree was once a sprout, the tallest tower started with the first brick, and the longest journey started with the first step.

Just as these wise words say, things are easier to handle early on. Once they get rolling or get bigger though many times there is too much momentum to control it. Just like a small snowball at the top of a mountain can become a massive avalanche so too can a small problem ignored early in development become a costly, potentially project killing problem later in the development cycle.

Releaseing early and often during software development makes sense. By focusing our teams on the first things that really matter we can focus our efforts on the things that make a difference. By focusing on the things that we expect to be hard we can ensure we have enough time to manage the risk associated with them. Identifying things early allows us to adapt our plans and make the best set of decisions.

2007-07-25T12:08:00.000-07:00

My trip to the post office

I have been asked before where my inspiration for blog posts comes from. Sometimes its from things I read, other times it's from events that I am a part of. As a little window into why I blog what I blog I thought I would share one of my "triggers". A recent trip to the post office caused the trigger for my recent post on Process Engineering. Now... on to my rant.

First, a bit of passport news. The US Government recently decided that no matter where you go you need a passport to travel and come back. Not really that big of a deal other than that it hadn't been a requirement for Mexico or Canada, popular destinations for many people. This (of course to those who actually think) caused a spike in demand for Passports. (I found it hilarious that the website for passports had initially stated that due to unanticipated demand for passports there was a delay in processing. Now corrected to simply state processing times have increased due to high volumes. If you change the law to state everyone needs one... demand will increase. duh.)

I recently went to the post office to get passports for my family and renew mine since it expired this year. For my own there is a nice improved process where you can mail your existing passport, a form and the money required and they will ship it back to you. No lines, no muss, no fuss. (Although the page states you must apply in person or by mail, if you go in they will tell you to mail the form... what a great way to waste an hour in line.) You can also fill out the forms ahead of time to simply hand them in and make it fast.

However... if you do not have a passport here is the current process.

Go to the post office. There are a limited number of post offices that do this so check first. (This was the least stressful step in the process)
[10 minutes - Post Office is close, easy drive]

Stand in the Passport line. It is important to note that you will need a photo of your head. These can be purchased for cheap at Costco etc or $15 at the post office. (How convenient. Of course photo or not you all stand in the same line.)
[60 minutes - 3 people ahead of my family. 5-7 photos taken, Forms filled out for each person, 1 line, 1 person doing everything]

Talk to the Passport person. One passport handled at a time, stapled Birth Certificate (actual certificate) to form along with check for Passport.
[10 Minutes - 5 minutes per person w/ 2 kids]

Hold up right hand and swear that the form is truthful. (Humans are apparently unable to lie with their right hand up. Practice in the mirror)
[2 Minutes - 1 minute per person w/ 2 kids]

Receive ticket to pay post office for service
[1 minute - Only one ticket for the group, process has been "optimized"]

Stand in Post Office line to pay Passport ticket. This is the same line as people buying stamps, weighing packages and otherwise wishing they were elsewhere. (Most of the articles have long since archived out but if you remember back to early this year some 3700 post offices improved their turn around times on lines by removing the clocks from the walls. Really.)
[2 hours - really it was just 45 minutes but I have a caveat. Based on the rate of line movement for the people who were in the "passport room" ahead of us in the passport line, they waited 2 hours to pay. However, it was nearly closing time and 3 extra people miraculously appeared and chewed through the line so they could go home on time.]

Return proof of payment to the Passport Office. Receipt for payment needed to be shown before the paperwork would be put in the "to be sent" pile. Now, at this point most people are fed up enough that they just walk in the door and show the Passport person the stub and don't wait in the line so I guess you could say this was "efficient".
[5 minutes - Had to wait to get attention for the drop as the Passport person was taking photos]

All totaled this accounts for (10+60+10+2+1+45) 128 minutes of my actual time. Just over 2 hours. Considering that we had everything filled out, ready to hand in and walk away without additional photo time and we had an accellerated exit because the "pay line" wanted to go home this is truly disturbing. The people who were in the Passport office when we got there were there for at least 4 hours. If this was a private entity they would be pushed out of business.

Opportunities for improvement abound.

Have two lines - One for photos and one for completed forms with pictures. This would have cut out at least 60 minutes.
Have a machine take the photos with the cost of credit card freeing the person to just deal with forms. If people can't get it right they can do it again.
Transmit automated digitals taken since the Passport office is printing them anyway just attach that.
Give People the forms to fill out and they don't get in the line until they are done. This keeps people will filled out forms from waiting for people to remember where their mothers were born.
Insist on pre-filled forms. They are available on the internet. A terminal could be set up for filling out and printing.
Pay in one place. Let the Passport office collect all of the fees. This would cut out at least half of the time for most requestors.

If these suggestions were put in place my 2 hour passport ordeal would have been a 5 to 10 minute paperwork drop. I would have nice things to say about the easy nature of the excessive instead of wanting to post about Process Engineering and needs for improvements.

RANT DONE.

2007-07-20T12:35:00.000-07:00

Know where you are so you can go where you want

Aside from the desire to make a pithy title I did want to blog a bit on Process Engineering. A quick Google define: provides

Process engineering is about applying engineering approaches, techniques, and
tools to the construction of Process Models. [Rolland1998]
en.wikipedia.org/wiki/Process_Engineering

Aside from sounding very technical and confusing this definition really does capture the gist of what Process Engineering is about. Looking at how you do things and applying techniques to improve it. This could be any process really, from how you save a few moments by putting your toothbrush in a cup on the right side of the sink to how data is pre-processed for correctness before loading into the system slowing down initial loads but saving time on back-outs.

Why is Process Engineering important? I am glad you asked. It is important because without having an understanding of what we currently do we cant' figure out why we do it and thereafter improve it. For example, I was involved in a process improvement initiative once where we boiled a process that originally contained 30 steps to one that had 5. How? We figure out that each step (there were really only 5 major actions in the process) was supported by it's own set of steps that validated the information received from the previous step. Each team would send a spreadsheet to the next team, that team would run it's own validation, purging and cleaning and boil it to a new spreadsheet because they didn't trust the data then send the new spreadsheet to the next team... etc. By inserting a system where users could check in the spreadsheet to be pulled apart into a database with referential checks all of the separate validation could be done automatically and viola life was good.

What should we do with this? Similar to many of my posts my statement here is that we should question things. Don't be afraid to do something different than it always has been done. But don't just change for the sake of change. Understand what is currently being done and why. Use this data to attack inefficiencies and fix them. Sometimes it requires code or a system but other times even a people process change will improve things. First and foremost though, understand where you are so you can define where you want to go.

2007-07-02T15:17:00.000-07:00

More disruptive tech.. here Wii go

I had another experience with Disruptive Technology this weekend that I thought I would share. First a quick intro story, my wife decided that I spend too much time writing blogs and doing other work at night so she gave me a Nintendo DS to play Pokemon with my son. (We were playing the card version but that is all old school and passe now.) The people at Nintendo are GENIUSES... if you purchase the new Pokemon game for DS you have two options, Diamond or Pearl. The variant between the two is the likelihood that you will run into any particular type of Pokemon. So in order to get the best mix of Pokemon you need to trade with someone else... this is where I come in. I get one type, my son gets the other... GENIUS... now we buy two games.

It get's better though, and this is where the disruptive tech comes in. If you have a Wii (and if you live under a rock and have not heard of this you need to get out more or read) then the new version of Pokemon Battle Revolution for the Wii also allows you to trade back and forth your Pokemon with the Wii version. What a Wii bit of fun. (sorry, I had to, they will kick me out of the blogger's union if I don't hit my pun quota)

This brings me to my point (already). The interface for the Wii is entirely different than the existing interfaces for video games. This has, of course, hit the press and blog pages long ago but I thought it made a great point. The Wii does not support true HD, it's tiny and has no huge sizzle effect like the XBox 360... it's even cheaper and simpler than the other boxes and yet, it is rocking the video game world. Because it changed the interface rules. The Nintendo folk's thought differently and instead of focusing on ubber graphics and interaction view(the standard thought pattern of the day in games) they changed the interaction method. With your Wii wireless controller (it's all Wii small... there I think that's quota) you control standing in the middle of your living room. It's easy enough my 4 year old can play bowling and baseball, fun to watch in and of itself.

What can we learn from this? Don't be afraid to go another direction. Wii is having amazing success and corresponding sales (if you doubt this go try to find one in a store). This causes me to think about basic interfaces and the way our users interact with systems. Not just web versus green screen but form versus map and open search versus controlled UI. This is the gist of Web 2.0 and why the interfaces work. Kids who are growing up now don't think in forms, why would they? Their interfaces are things like iPhones and Wii.

2007-06-27T21:52:00.000-07:00

Disruptive Technologies that make you go hmmmm

Two separate events in the past few days have made me consider disruptive technology and it's impacts. While fun for those standing on the sidelines it has to also be a terrifying event for those who are in the middle of what is being disrupted.

Disruptive Tech 1: The iPhone
Disruptive Tech 2: Google Apps

Disruptive Tech 1: The iPhone. Yes, the often referred to Apple foray into the cell phone business (in some cases referred to as the "Jesus Phone" it's PR is so big) is a great example of a disruptive technology. It is already wrecking havoc with phone manufacturers and not because it is the worlds best phone, but it is slightly off from today's phones. The NY Times put out a great article late Tuesday night as the phone was hitting the general population with the compelling headline "The iPhone matches most of it's hype". Gizmodo posted a good summary of "Finally Confirmed: What the iPhone doesn't have" [Songs as Ringtones • Games • Any flash support • Instant Messaging • Picture messages (MMS) • Video recording • Voice recognition or voice dialing • Wireless Bluetooth Stereo Streaming (A2DP) • One-size-fits-all headset jack (May have to buy an adapter for certain headphones) • 3G (EV-DO/HSDPA) • GPS • A real keyboard • Removable battery • Expandable Storage • Direct iTunes Music Store Access (Over Wi-Fi or EDGE)]

That's quite a list for something that is still going to change the phone world. It has a controlled UI and OS. You can't load things on it and Jobs has made no bones about confirming that will remain the case. But it has a paradigm changing interface, consolidates devices for those with iPods and Phones and PDAs, even smart phones. As eloquently stated by David Pogue in the times article "It’s substance; it’s style. It does things no phone has ever done before; it lacks features found even on the most basic phones."

Personally I am going to hold off getting one. I recently purchased an 8525 (YAY I get to skip my Blackberry across a lake) and I have been quite happy with it. (heck I have already customized the snot out of it and have my Yodeling cow ringer going) But the geek in me covets parts of the iPhone and I may end up with one when I next need a phone... when it uses the faster Edge network and apps to get to business email are actually available. In any case, the reactions from the "old school" cell phone makers is going to be a blast to watch.

Disruptive Tech 2: Google Applications. I have to admit that when Google first rolled these out I thought to myself "ok... that's nice... why do people even try to beat Microsoft in this space?" Well a quiet little post on the Google blog made a light bulb go off for me. Google isn't trying to hit Microsoft at the desktop (well not right away) they are hitting them squarely in the admin.In the blog post Google indicates how they are going to give free access to advanced features of the full Google office to schools. The standard package for individuals is free already but a Premier Edition, targeted at companies is also available for $50/user. Now the Education Edition also includes the ability to integrate with existing facilities, conference and resource scheduling and other premium services.

Educators want to educate. They don't want to run systems for email etc etc. Now, they can get email, integrated chat and other niceties for free. This is where giving it away starts to make sense. It's not just about giving it away to the individual consumer in hopes of pushing ads. It's about pushing it into corporations... wow, now that has to be a scary thought. As with the iPhone, it may not truly change the world, but the potential is there and it will be fun to watch. (By the way, although corporate email isn't available via the iPhone... if you use gmail you are good to go... hmmmm)

2007-06-26T08:45:00.000-07:00

Every integration point created WILL fail and Scalability

I have two things for today's post.

1. I received a note this AM from my "FYI- Jana RSS feed" (A good friend that works with me that sends me email) that I had to pass on to everyone. Read the article because the quote below is SOOOOOOO true.

“One of my recurring themes in "Release It"
is that every call to another system, without exception, will someday try to
kill your application. It usually comes from behavior outside the specification.
When that happens, you must be able to peel back the layers of abstraction, tear
apart the constructed fictions of "concurrent users", "sessions", and even
"connections", and get at what's really happening…”
Agile, Architecture and
the 5am Production Problem - by Michael
Nygard

2. This weekend I attended the Seattle Conference on Scalability. It was sponsored by Google and it was GREAT. Over the next few weeks I will digest what I heard and saw and report it back via blog. At the moment though I have to admit my dueling day jobs are getting the better of me so I haven't had time to couch my ideas into pithy post prose. I promise it is coming though. I will however let you know that Google recorded the presentations and they should be available by the end of the week on YouTube.

2007-06-25T12:02:00.001-07:00

Enterprise Architecture is not technology selection

Now that I have extolled the virtue of standards like muscle building and listed our current set I wanted to post a bit on why I believe having a consistent Enterprise Architecture is important.

First and foremost on the list of why an Enterprise Architecture is important is that for a company to be successful it needs must build IT Solutions, not IT Capabilities. What is the difference you ask... (or even if you didn't I am going to tell you anyway) Capabilities can survive in non-integrated verticals but Solutions require integration. An End to End view of paths and functions is required to get a solution. An IT Capability is something like storing a customer information record in a database. A Solution provides the customer with value because it can use that information and to customize a trip.

One of the fundamental rules of development is breaking things up into smaller, solvable pieces. At the individual system level this makes perfect sense. However, when you start to look at things at the enterprise level, things implemented in pieces build piecemeal solutions. Then, these piecemeal solutions need to be integrated. This is what I have referred to in previous conversations as the fiefdom effect. Essentially, this is where individual systems or areas have their own fiefdoms and protect their system boundaries and "control their own destiny". While this may be fine a micro level, the independent pieces work fine after all, at the macro level this increases the integration burden and decrease overall company agility.

This is where we get to what Enterprise Architecture is all about... Enterprise Architecture's (and Engineering, YAY Engineering!!) job is to take a long term view so that teams can build capabilities not just fulfill the short term order of the day. If we have a solid base upon which to build (and this is an art in itself, not building the Taj Mahal or better yet the Winchester House when an outhouse will do the job) we enable everyone to move much faster and make integration between components that much simpler.

Yet another argument for a seamless SOA infrastructure. Or at least that's what the vendors are touting as the latest Technology Software solution for this problem... but we will save that post for another day.

2007-06-19T12:22:00.000-07:00

Monkeys down a fire pole

How many monkeys can slide down a fire pole in 5 minutes?

Ok, so this question doesn't seem to have a lot to do with technology or engineering when you fist consider it. But give me a little bit of patience and let's see where we go.

I have recently begun to undertake a personal quest for education and education provision to others about threading and how to do it well and efficiently. With the introduction of multi-core processors and processor companies no longer creating faster and faster single core CPUs with clocking, over-clocking and over-over-clocking, it has become more important to actually understand threads.

Even as recently as when Java first came out the only places that you really ran into threads in open systems was in Solaris (or other Unix) systems with big multi-CPU backplanes. These were usually reserved for big 'ol database boxes though so you could optimize your system to really what amounted to one big fire pole. If we imagine each processor as a fire pole and each unit of work to run as a monkey, we can start to use my fun little analogy.

Everything you did was about queuing up the monkeys efficiently then sending them down the pole as quickly as possible for each individual Monkey thereby maximizing your monkey throughput. Great schemes for keeping the Monkeys in the right order, keeping the pole clean during non-monkey sliding moments (Garbage collection in Java-land) and other similar optimizations were the wave of the day.

Now the fire poll manufacturers (Intel and AMD) discovered that they just couldn't build a faster fire pole without violating the laws of physics (and it's not that they wouldn't break those laws, more that they couldn't figure out how) and therefore decided the best way to move forward was to build a multi-fire pole firehouse in the same amount of space. It was a great innovation... but the monkeys were still optimized for a single fire pole. So in many cases (not all, but many) the fire pole (CPU) utilizations actually was very low even though other areas were slammed. Monkey lines (memory) and even Monkey storage (I/O) became bottlenecks with a low 25% pole usage.

The time has come to build education on efficient fire pole usage. We need more effective ways to slide those little monkeys down. There are some basic practices that seem to make sense. Sun recently explained how they are building better threading into Java and things are rolling out from there. That's a great start... but that is just a start. To really use the power we are now putting into machines we need to figure out better ways to feed the cores.

Help me save the monkeys... do you have any good tips, observations, sites or do you have any bad monkey practices that you have seen that should be avoided? This practice is, of course, not limited to Java but C, C++, Ruby and others. I would like to pull these together to be shared with everyone. If you have a good monkey tail please drop a comment. I would love to hear from you.

2007-06-18T09:08:00.000-07:00

WARNING... FOR YOUR SAFETY...

Most home invasions come in through Windows...

Saw this come accross the wire on the Ubuntu site and laughed out loud after watching it. I thought it ranked up there with the Ruby Commercials. Of course it's a little simplistic since we all know there are vulnerabilities in Linux as well as Windows. Though the frequent target is certainly Windows due to the ABM (Anything But Microsoft) cooalition's distaste for the "evil empire".

2007-06-16T10:10:00.000-07:00

Build your Corporate Muscle Memory

I have posted before about the need for standards to allow us to focus on differentiating functionality and I have been thinking about this again since I have been working on a project in which standards are a big part.

Have you ever considered how as you learn something it become easier with practice, eventually become second nature? Watch a small child learn to walk and it is something that they have to really focus on at first. Then as we get older it is something that we simply take for granted. Does this make us expert walkers? Or do we just establish walking as a "dial tone" activity that we no longer need to think about?

Think about a chef. When they are first learning to do things they need to focus on every cut of a vegetable. But as they become an expert they no longer even measure a teaspoonful of salt. (BAM! Just watch an episode of Emeril Live and see how closely he holds to a written down recipe for an example.) A chef can do things without any concentration at all that many of us would require many times the time to do.

A runner or athlete is another good example. Runners develop muscles that work in very distinct ways. Their develop something called muscle memory that causes the muscles to introduce an actual change to how the muscles work. This change increases the level of accuracy and flexibility for running by optimizing the muscles for the repeated action. There is a trade off however, the muscles become very good at a certain set of items and less flexible for others. Because a runner is training to be a runner, not a gymnast this is a fine trade off.

Standards are similar in that we want to establish a Corporate Muscle Memory. It will require focus and concentration at first to do the new standard processes. It will also reduce flexibility to a certain extent in that there will be specific micro-processes that could become harder but the specialties become faster. But with the goal being to move as quickly as possible with agility towards the companies direct goals this is a trade off worth making.

Focusing our efforts on learning and optimizing according to standards allows is to build our Corporate Muscle Memory. We can then go faster and focus on differentiating functionality rather than dial tone. Just as with human ability companies ability needs to evolve over time and standards are no different. When the time comes to change a standard there will be a period of concentration required for us to develop that new Muscle Memory and for it to become second nature. But as we do whole new levels of speed and capability are possible.

2007-06-15T09:13:00.000-07:00

Developers are Artists

In a meeting a little while ago I heard a comment, at the time said in a little bit of a flip way, that developers are artists. While not necessarily meant in a way that would suggest that Developers are right brain oriented, creative, canvas painting snobs the statement did get me thinking.

We were discussing standards. A topic that can be scary for people who want to forge their own way... artistically creating the next generation of award winning software... etc. (blah blah blah)

So off my brain went on a metaphor dive and it actually started to make sense to me. While artists want to be able to choose their own tools to ensure they get the right tools for the job there is a lot to be said for simplicity. Having the right brush size is critical... but it's still a brush. Certainly, there is a difference between the 10 for a dollar brush and the $20 brush but if you compare a $19 and a $22... is that really something an artist wants to waste their time doing? I mean really... there are people who are paid to do that... shouldn't the artist be creating... painting... utilizing their skills to make a difference.

Our approach to standards needs to be the same way. We will have people who specialize in brushes. Or in this case Database Platforms, Application Server platforms, Middleware solutions etc. We have a set of brushes sized appropriately for the job. Or in this case standard containers such as J2SE and J2EE containers both basic and fully enterprise capable... Tomcat, JBoss, JBoss with Clustering, etc. We want our artists to focus on painting the canvas with differentiating impactful imagery. Or in this case business logic that implements the vision the company has for functionality.

I realize some of you may be wondering just how deep my analogy rabbit hole is going to go but I will make a few more points and abuse it just a little more and then let you get on with your life. Just like brushes need to be replaced and a new canvas is occasionally needed so to do application and development standards need to be refreshed. Standards are just a representation of the best knowledge at a point in time. They are meant to be challenged, updated, and tuned. Without that effort and focus they become dated and slow things down rather than speed them up.

So as we set and roll out new standards as part of our efforts to accelerate development and increase stability and quality keep that in mind. Let's make it so that our artists can paint... but let's also challenge the status quo.

2007-06-13T09:31:00.000-07:00

A case for the Designated Scapegoat

Have you ever considered why it is that when we consider systems we assign it attributes such as he or she or "misbehaving" or otherwise letting us down? I have. Why you ask.... darned if I know, I have a lot of random things float through my head and some of them are actually are entertaining. Hopefully this is one of those.

It seems that we as humans have a fundamental psychological need to explain things. So in order to explain them we put attributes either internal or external to explain why things happen. Maybe it is because from the age of "really small" we ask why? (and those of us who have children know the number of times we answer that question becomes numbing.) In psychology this is referred to Attribution Theory.

External attribution is things like "the devil possessed the machine and crashed it" or the ever popular "the data center is on a burial ground of some kind and it causes the servers to crash". It may not be true, but it helps us to deal. Or certainly to laugh at our misfortune.

Internal attribution is, in essence, blaming yourself. The easy example is things like "I am a sinner, please forgive me." We see this a little less in technology though it does pop up as well with the occasional person who perpetually places blame on themselves.

This fundamental psychological need is why I am suggesting the role of Designated Scapegoat on all projects. If a designated scapegoat is assigned at the beginning of any project we can simply move on to the fixing of problems since don't have to waste time in meetings deciding who or what is at fault. All of the posturing, political planning, set up, stonewalling, denial etc can cease and we can simply move forward. All blame can preemptively be assigned to the Designated Scapegoat and productive work can begin immediately. My experience with this role suggests that your Designated Scapegoat should be someone who is generally a good natured, understanding and who everyone knows in their hearts is beyond reproach.

Imagine how instead of a two hour meeting where people discuss why it is not their problem you start the meeting off with "Gee Chris, that was a mess up, I can't believe that you crashed the servers across the globe all at the same time." Chris then responds "Yeah, you know in Project Manager school we learned that the best way to crash a system is to push the button really hard, right in the middle." Then blame and attribution discussions are done and conversation can move to actual observed behaviors and problem resolution. What a time saver!!

So as a productivity aid for your next project assign a Designated Scapegoat up front, save yourself from Attribution Theory and all those unnecessary meetings. Focus on fixing problems, root cause analysis and long term fixes. Jump right past the blame game with this easy step.

2007-06-01T07:52:00.000-07:00

Step 1... round up the lawyers

We have all heard the jokes and everyone has read the phrase before. The first step to improvement is round up the lawyers and send them to the bottom of the ocean.

If you have had your head under a rock or otherwise not seen any tech news lately you may have missed some of the latest fun with lawyers and the grand FUD they can churn up.

First Microsoft came out with a statements about how Open Source has violated 235 of their patents. Of course no details were provided on the patents themselves just a bit of churn and great press. Using their recent deal with Novell for patent sharing (Read my post on how when you wrestle a pig you both get dirty but the pig likes it for more info) as an example of "good" behavior they encouraged others to strike similar deals. Part of the real fun in all of this is how at the same time Microsoft was (out of another side of it's mouth some might say) calling for Patent reform due to suits from others against their own patent infringements.

Then the Linux foundation crafted a rebuttal. As would be expected they called out Microsoft on what the patents were. They clearly called out the FUD aspect and took a more offensive approach and begun to fund patent claims among other actions. Spinning from IBM, Sun and even Nokia releasing patents for Open Source use this approach seems to be gaining momentum.

Now the new GPLv3 is even including language that would cause Microsoft to loose rights to sue if the new Linux license terms in GPLv3 go forward. Further complicating this is how the deal with Novell may actually be what causes Microsoft to loose enforceability on the patents. Now even Novell is exploring patent reform. Even going so far as to fund "Patent Busting" activities.

Wow... I guess if you are a lawyer this would be fun or at least profitable. But as a techy I have to just sit back, pop some popcorn and watch the show.

2007-05-27T18:28:00.000-07:00

Observations from my Romp with Ruby

My experience with Ruby on Rails was a positive one, or to quote the Rails site, it was "Web development that doesn't hurt". I am still by NO stretch a ruby or rails expert but I did learn a lot and I was impressed with the abilities in such a relatively young language. I still believe that no language will ever solve all of the worlds programming ills. But they will continue to get to higher and higher levels of abstraction and developer productivity.

Ruby is essentially an object oriented scripting language. Rails is a separate library set that is focused on productivity enhancements and tools to really accellerate web development. Rails uses the Model View Controller pattern for development and a strong implementation of the Domain pattern for data persistence. You could very easily build a fairly complex app all without any knowledge of the database. (Now there are obviously other concerns with that but we won't dig into that in this post.)

CSS and stylesheets can be automatically generated along with full basic UIs with a tool called scaffolding.. (It's not that pretty but gives something upon which to build and actually works great for validating that you have the right model). For many of the common advanced UI actions there are simple sets of commands as well with built in Javascript AJAX - scriptaculous commands for visual effects, drag and drop, dynamic lists and more. Everyone knows you have to have AJAX to be cool now adays.

If you want to poke at Ruby and learn a bit check out RadRails for development (it lacks intillisense type function at the moment which made me sad but everything else you could want works right out of the box) and InstantRails for a working environment. It's a great combo to get you going in just a few short downloads.

Every language will have some weak spots and Ruby is no exception. But it does work hard to answer many of the common problems in today's frameworks. My favorite (have to give a plug for the Engineering side of things) is things like fixtures and test automation built-ins to encourage test driven development. My biggest question is still, how does it scale with very high volumes?

2007-05-23T19:48:00.000-07:00

Solaris vs. Linux

As mentioned in my post Sun gets a new spark I recently had the opportunity to visit with some of the folks from SUN and get some visibility into the efforts they have under way. One of the things that this visit caused me to ponder was Solaris 10. When Sun open sourced Solaris I wrote it off as the death throws of a previously strong giant and it made me sad. I originally cut my Unix teeth on Solaris and did a lot of work with it as the stable foundation on which to build open systems. However, as Linux came into its own I have also been one of those folks who has systematically been replacing Solaris in the data center with Linux. For the most part going with the masses to Red Hat but with occasional forays into others as well.

Some developers that I know have stuck with Solaris for their development environments even through the Linux craze due to tools that they liked. Originally I had gone the other way, moving development environments to Linux and deployment environments to Solaris. Solaris in prod for stability and Linux for tools and speed in development.

With the new innovation going into Solaris, things like ZFS, zones and Grid it gives pause. The difficulty now is that everyone and their kid sister has run Linux, known Linux etc. There is a wealth of Linux talent out there and the Solaris admins are much harder to find. Of course Red Hat has been getting more proud of their products and pricing their support accordingly which also adds to the mix. With all of the recent silliness from Microsoft on their FUD around Open Source and indemnity clauses things get even more interesting. (I personally am of the belief that if they thought they could win they would already be in court... but I am sure the FUD is the real goal)

So given your choice... which would you pick?

2007-05-22T18:16:00.000-07:00

Pay attention!

A while ago I read a post by Seth Godin on how to be a great audience. I thought it was a great post and provided some very good tips on what to do and how to listen and be (as the post says) a great audience. The context was a presentation to a group of eighth graders and how he could see who would be the good audience and who would not. The students who leaned forward, engaged and asked questions and drew out a better presentation.

Over the following months I have seen this over and over again in meetings. It's amazing the number of people who don't pay attention in a meeting. They drift off to their blackberry (assuming that the network is still working) while someone else is making a point. People bring laptops to meetings and proceed to answer email, read news (and if it's not an RSS feed of my blog that's just rude) or otherwise ignore why they are there in the first place.

Now I will admit that many meetings that get called seem to be meetings for the sake of meetings rather than meetings with a mission. This can be addressed separately. Be direct in meetings and don't be shy about making sure that there is a focused purpose and when that purpose has been achieved... don't feel that the meeting needs to go on just because it has been scheduled for an hour and fifteen minutes of discussion handled it. Give people the gift of time.

So, in your next meeting, make an effort to pay attention. Listen, lean in, ask questions, be a good audience and for goodness sakes PAY ATTENTION!!

Rails Envy: Hi, I'm Ruby on Rails...

2007-05-15T09:03:00.000-07:00

Rails Envy: Hi, I'm Ruby on Rails...

Click here to view on YouTube

I laughed until I stopped.

2007-05-04T07:10:00.000-07:00

Friday Link Day - Microsoft to buy Yahoo!?

News has started to come out that Microsoft is looking to resume talks to buy Yahoo! I think someone has Google envy.

Along the lines of the silicon related post from earlier this week... IBM announced that they have devised a way to create vacuum spaces in chips that will allow them to use less power and run faster. The chips even "assemble themselves" to a certain degree.

Getting tired of the laws of physics holding them back with silicon Intel is working on Laser chips now. NEC is saying that laser chips could power petaflop-class chips. Optical chips may still be a few years away but the idea sure is fun.

2007-05-01T09:56:00.000-07:00

SUN gets a new spark

I recently had the opportunity to go out to California and visit SUN. It was just a short period of time but it was an information packed couple days. We discussed topics ranging from Sun GridEngine to strategic direction, from Developer Tools to hardware threading.

I was particularly happy to see that SUN appears to have obtained a dose of humility. Where their previous corporate strategy seemed for a while to be "ABM" (Anything But Microsoft) they are now focusing on understanding their spot in commodity markets and differentiating based on service. The previous approach and Scott McNeely's top 10 lists was certainly entertaining it didn't do much to further the companies goals.

SUN has now taken a page out of the RedHat handbook on making money with Open Source and are trying to make money with service differentiation. I had written off the initiative to Open Source Solaris as the dieing throws of a declining giant. But listening to the vision that they have and what innovation Sun is now building into Solaris 10 with things such as ZFS and Virtualization such as Zones.

Their approaches with Silicon and drive to innovate there was also interesting. They now sell Sun boxes with AMD and Intel processors. Talk about Cats and Dogs living together. We even discussed a current effort sun is working on with Microsoft for a large system provider where they are serving real time video off of Sun hardware running Windows. ... Let that sink in for a minute...

Another interesting point that was brought up was how Moore's Law and fitting more onto the same bit of silicon applies to RISC chips as well as CISC chips. Since RISC is smaller (Reduced Instruction Set) they can then fit more processors on the wafer. So they are legitimately talking 32 and 64 core processors. Talk about the need to thread your application. Initiatives such as Niagara and Victoria Falls also provide low energy approaches.

Just read some of the press of the days for a glimpse of some of what they have been up to.

2007-04-27T07:55:00.000-07:00

Friday Link Day 2007-04-27

My favorite link of the week is the discovery of a new "potentially inhabitable" planet. It is believed to have water! If you start now by the time that your children's, children's... heck a lot of them are born, its just shy of 21 light years away, you can have an address that ends in Planet 581c!

Software Engineers can go to space too. I wonder if they ran any parts of flight control and operations on the Excel or Word... and if he knew they were if he still would have gone.

Do you hate putting your phone or iPod on the ground. Then you need the Electronics Hammock... yes, it's really a product.

If you hate your BlackBerry (like me) but enjoy the abuse and want to put the interface on a Windows Mobile 6.0 phone BlackBerry would like to help you out. They are apparently releasing BlackBerry functionality on Windows Mobile with ATT first and moving from there.

2007-04-23T12:21:00.000-07:00

How do you get work life balance?

I was reading through some blog posts and stumbled onto a presentation I wanted to share. Stuart Levine, author of "Cut to the Chase," provides a downloadable PDF manifesto entitled Reclaim Your Life: A Two-Week Challenge to Help You Regain Time It has 11 great tips on how to get to the point, and get the time you need to really make a difference. He starts with a quick review of work life balance and how it's easy to say and not so easy to do. Check it out... make the time.

2007-04-22T21:56:00.000-07:00

Nothing gets done till nothing gets done - Woehlke's Law

What a provocative thought... nothing gets done till nothing gets done. This law is really more of a postulate or a theory then an actual scientific law as I don't believe any mathematical proof has ever been done. That said, it is one that has a lot of circumstantial and experiential evidence piled up towards it being the truth.

We have all been on projects where we know going in that the right [Resources, Timeline, Staffing, Education, ...] is not in place for the project to be successful. But we march on and because we all want to do a great job we try our hardest and keep the worst from happening for a long time, sometimes succeeding despite the obstacles and sometimes not failing until very late. What Woehlke is stating in this is that in order to get management attention and get the resources that are truly needed to be successful a problem needs to be evident.

A few quick reads on Woehlke's law -

The Nimble PM has a nice article on Woehlkes law starting with the quote "Project managers will not get the staff they need so long as they muddle through with overtime, ulcers, and super-human effort. Only when deadlines are missed will senior management approve the staff who, had they been available at the outset, would have prevented the missed deadlines"
Cutter had an advisor article in 2005 where Donna Fitzgerald made some great points about how to avoid getting caught in the trap and how hard it is for most of us over achievers to really understand what this law means.

What do I think this law means? If you look at a project and know that it can't be successful you need to prove it. Not with whining and bellyaching. If you go to management with a story of how "this will be really hard" you will get a reaction of "well duh, that's why we have you, our gifted team on this project". Rather you need to have "Data in fact". Set up tests to show where performance is and what would be needed to mitigate it. Set up early aggressive iterations to show a realistic rate of development. It's all about risk mitigation... if you can show where the real true risk is then you will get help. If all you have is an intuitive "this is hard" you will be patted on the head and told to go try hard.

2007-04-20T08:25:00.000-07:00

Friday Link Day - 2007-04-20

Does Apple + Google = Love? It is a high interest geek story and makes for attractive rumors. The enemies err... competitors for both are often the same. My personal thoughts are that there will be much collaboration but no marriage. The Forbes article brought to you in pictures certainly is a fun read no matter what the eventual outcome.

Speaking of a marriage... Like the MAC? Or at least the UI? Check out this Blog post with a style sheet that allows you to put a MAC interface on Google Reader. (If you are looking for an RSS Reader I recommend the Google Reader BTW.)

Then to keep the Google theme for the day Google shows us the first tenet of innovation in action. The best way to have a good idea is to have lots of ideas. Apparently Froogle didn't fall in the good category so Froogle is dead... long live Google Product Search.

2007-04-15T12:10:00.000-07:00

A quick snip on testing

I have recently had two different offline topics end up with the same point so I figured I would pass it on here as well. Testing and why it is important to test to failure and not just test. To stick to the intuitive reason, it is because then you know when your system will break and what will happen when it does. It is naive to believe that your system will be 100% problem free, it is a far more realistic expectation to accept that you just don't know when.

One of the fundamental race conditions of the universe is that as soon as you build a better, more idiot-proof system, the world will build a better idiot. Similarly in a system which is successful and experiencing growth, you will eventually hit a point that exposes problems that didn't show themselves under lesser pressure or lesser load. In aircraft they X-Ray and image things like propellers not because they want to ensure that it is perfect, but so that they know what quality it is. Under the right conditions a very tiny bubble can cause a propeller to literally fly apart. Better to know these things up front.

My Jana RSS feed recently (inside joke) recently came across with these two links, one for example, one for humor that are relevant to this topic. Check out this Ultimate Failure test... and fly with a little better feeling in your heart because you know that they care enough to do this. Then check out Monkey Testing at it's best. (I am trying very hard to resist the 1000 monkeys and Shakespeare sonnet reference... darn... I failed.)

In conclusion remember, it's ALWAYS better that you know where things will blow up than for someone else to tell you.

2007-04-09T13:17:00.000-07:00

Go get an MBA?

I recently read a fabulous post by Seth Godin on NOBS, the end of the MBA. In between bouts of laughter I thought immediately of post material and here we find ourselves.

Not that there is no value in an MBA. Certainly there is and depending upon the program, classes and approach to teaching lots of value can be garnered. But it is not the be all and end all that it is often made out to be. For many people a real world education and a desire to learn are what is needed to be successful. Through books, such as those written by Seth Godin are great sources of knowledge, the catch is that you need to make the time and spend the effort to learn from them.

Another important point is to consider what your focus is. MBAs focus on business administration and financials in most cases. If your passion and skills are around technology and development then it may not be the best suited thing for you to change that focus. Focus on what you do well, not what you think others want and you will go further and faster.

Stephen Covey often uses the term circle of influence to describe what you control and what you do not. If you focus on what you influencce you can expand that influence, knowledge and experience. When you focus on things outside of your circle of influence or beyond your control you only end up getting frustrated and end up with less time and less energy to focus on the things that can help you and expand your circle of influence.

You have a limited amount of time and a limited amount of energy and spirit... use them for something that gives you more, not less.

2007-04-08T18:18:00.000-07:00

Innovation in automobiles

Looking at the automotive industry, it seems as though innovation, real innovation hasn't happened in a long time. Sure there has been incremental improvements and there has been many great additions that have made our lives easier or better. (Try and tell a Mom who drives her kids for any distance that you are taking the DVD player out of her vehicle and you are going to face some serious wrath.) But next generation types of things seem to be a long time in coming.

I encountered an interesting one while channel surfing for cool HD content and it got me thinking about what stories are out there so here is my quick list.

We have all heard the stories of the self driving cars using approaches like that from GM is neat idea but one that people who love to turn the twisties view with some amount of horror.
Alternative fuels show promise and may help with future energy problems.
Keyless cars and other innovations that are cool... but differentiating?
Reinventing the wheel? Michelin is trying with the new tweel.
Even cars that can go 235 miles per gallon of gas!

The thing that gone me thinking about the whole topic though was the idea of thin cars. Two people cars that lean to corner faster. One example is the Carver. Another example is the Naro. The thought of twisting and turning but taking up less space and energy could be fun. Of course in my case, my immediate reaction was "will the power to turn the tires be enough?"

2007-04-07T09:20:00.000-07:00

I have posted before about how globalization and other changes to the world business environment have driven a need to develop and use our whole minds. This in turn is also driving what I believe is a need to address school systems to adjust to these needs. I recently read an interesting article from the Wall Street Journal on how Pain Free Trade Spurs Second Thoughts. The article is an interesting piece on how Alan Blinder, economist, Princeton teacher and one time advisor to the Federal Reserve has changed his mind about the impact of globalization and offshoring to American jobs.

It is easy to jump to the protectionist approach (especially for politicians) but really this just delays an inevitability. What, in my opinion, is the more correct response is to adjust our understanding and focus on what will be staying put. High touch areas and things that instantaneous technical bandwidth won't address will always be in demand. As a recent quote from Texas State Representative Rob Eissler, on the importance of arts education states "Left brain is logic, right brain is creativity. We don't want our kids to compete internationally with half of their brains tied behind their backs." (Original post from Dan Pink's blog. Dan is the author of the fabulous book A Whole New Mind that deals with many of these topics.)

2007-04-06T08:03:00.000-07:00

It works on my machine...

It's been a while since I have done a Friday Link Day so I wanted to throw together a quick one and pop a few of my favorites out for this week.

It works on my machine. What technologist has ever existed who has not used these words? Well now you can stamp that application with all of the recognition that a single threaded one time test deserves. By following the criteria outlined in this Blog post you can use the logo below and show the quality of your app.

Next, in the news of the truly cynical at heart, check out this article on Job Cuts for Fun and Profit. Short Circuiting Circuit City decided that 3,400 of it's workers were making just over 50 cents more an hour than their own determined average acceptable salary for a sales associate... so they are laying them off to hire cheaper people. So the next time you are looking for help and wondering why you can't find anyone who knows as much as your 7 year old you can remember how saving 50 cents an hour in employee costs got you an extra 25 cents off a computer. The article, one pulled together as part of the Knowledge at Wharton effort from U Penn Wharton School of Management, actually makes many good points on a number of topics and is worth a read if you get a chance.

2007-04-05T07:30:00.000-07:00

TPF as a SOA platform

Really... I am not kidding. For those of you who read my posts at all regularly you know that the majority of my time is spent in Open Systems. But in reality there is still a lot of data and processing that occurs on the mainframe. I caught the following press post on Information Week and simply had to read it.

IBM Opens Up System Z Mainframe To SOAs

Now, reading into the press release it's much more fluffy than the title sounds. But really, would we expect IBM marketing to not be able to do their jobs? (I think they were the original inventors of F.U.D., right?)

Still, it does make you think about what kind of data source a mainframe could be for any high volume system. The ability to handle large amounts of data and transactions in a stable manner is a consistent throttle that we encounter in the Open Systems world. Now the cost/benefit needs to also be compelling since there are many other ways to solve the problem then simply making the iron bigger.

I had an opportunity to talk to some IBMers lately and they are still big on big boxes, just now they want to slice those boxes into 100s of virtual servers and provision them dynamically. I found it ironic that when you consider the open source revolution pushing hard away from big iron, we are now pushing closer to it. The IBM Z series and the IBM P series even sit in the same "refrigerator boxes" now. (For those who are not Z or P savvy Z=Mainframe, P = Open Systems) So you could walk out on a raised floor and not even know if it's an open system or a mainframe system... now it's just a system.

But then, isn't that how a service consuming customer views it?

2007-04-01T07:48:00.000-07:00

When you wrestle a pig you both get dirty but the pig likes it

The next in the significant Open Source events category is one that occurred with Microsoft and Novell. (Yes, that Microsoft). This deal has to do with Linux as well, but in this case its SUSE that is under discussion. Read the announcement or watch the video footage if you want more detail.

The pundits on this one are even more passionate. (As if we would expect anything else with that Microsoft being involved) Linux Journal likens the deal to Racketeering from Microsoft with FUD sowing from Novell/SUSE. Bernard Golden claims the deal just proves that Open Source is important to your future.

According to the Public Relations statements by both companies the deal is all about Virtualization, Web Services Management and Document Format Compatibility. It is also a five year patent protection agreement where Microsoft agrees not to sue users of SUSE if there is any patent infringement in the SUSE system.

My thoughts around this go immediately to other companies that have made a deal with the devil... err... I mean entered into cooperation agreements with Microsoft. (And no I am not an ABM Coalition card holder. I actually have a fascination and respect for Microsoft, .NET in particular and even SQL Server... I just think that from a business perspective you have to realize what you are dealing with) Take for example Sybase.... Somewhere around Sybase version 4.9.2 Sybase and Oracle were the big mean Relational databases competing tooth and nail with one another. Sybase was, at the time, viewed as slightly more technically advanced while Oracle had better marketing. Sybase then entered into a cooperation agreement with Microsoft to build a version of their database product specially tuned for the Microsoft Environment. A great move, they would get information from Microsoft on the particulars of the platform, build a super tuned engine and everyone would benefit... right? They were very successful, the product created is the forefather of the now nearly everywhere Microsoft SQL Server. As far as what Sybase got out of the deal... they got some short term revenue in the first few years of the agreement before it ended and Microsoft took things over. (Honest, they are still in business now) Other deals that can be pointed to include Borland... Symantec... Corel... need I say more?

Not that this will end up being an end of the world. Just like Microsoft owns the productivity suite on Apple they will likely get the information they need to build and own the productivity suite market on Linux. If .NET components run and integrate well with other platforms but work best on Windows platforms... hasn't Microsoft always been very good at Embrace, Extend, Exterminate?

Another one that will be fun to watch.

2007-03-23T07:22:00.000-07:00

Now that I have posted about Google and praised them in several regards I will point
out how silly that is. The Halo Effect is the general belief that because a company is successful it is successful because it is brilliant, innovative, has a great culture, [insert appropriate praise here]. Conversely when that same company stumbles it is because they have "fallen away
from that brilliant, innovative, great culture, [insert appropriate praise here] that made them great. Therefore management books, methods, papers, research, Blogs (heh) are written on this company and how you can be successful like them by following "these 12 easy steps".

The reality of the situation though is rarely that either one of those items is absolutely true. Good companies can have problems, and bad companies can have success. A recent article in the McKinsey Quarterly even suggests that "Suggesting that companies can follow a blueprint to achieve lasting success may be appealing, but it’s not supported by the evidence." and "The delusion of lasting success is a serious matter because it casts building an enduringly high-performing company as an achievable objective. Yet companies that outperform the market for long periods of time are not just rare but statistical anomalies whose apparent greatness is observable only in retrospect."

Wow.

This effect is referred to in psychology as the Halo Effect. Just we tend to be nearly binary in how we view others attributing positive traits to those we like and negative traits to those that we don't. Rarely viewing the people with a mix of positive and negative. The book The Halo Effect: ... and the Eight Other Business Delusions That Deceive Managers covers this topic in some detail and relates it to businesses

So what does that mean? It means that success is hard... go figure. It means that you need to constantly adjust your strategy and approach to be relevant to the times and business environment in which you find yourself. The approach that got you where you are today simply won't continue to work. It means that if you stand still, you die. From a corporate perspective and a management and directional challenge perspective this is a whopper of a concept.

A great quote on this topic from US Treasury Secretary and Goldman Sachs executive Robert
E. Rubin “Once you’ve internalized the concept that you can’t prove anything in absolute terms, life becomes all the more about odds, chances, and trade-offs. In a world without provable
truths, the only way to refine the probabilities that remain is through greater knowledge and understanding.”

That's almost grounds for a philosophy debate. Up there with "Only the insane have the strength to determine that which is sane."

2007-03-22T07:17:00.000-07:00

What if Google...

How many conversations start off with the phrase "What if Google...". When businesses try to do something different, especially something innovative or disruptive one of the quick lead in questions to that conversation starts with "What if Google were trying to do this?" or "What would we do if Google entered this market?".

It seems that Google is the fairly universally viewed as the thought leader of the current business age. There are even books available such as The Search: How Google and Its Rivals Rewrote the Rules of Business and Transformed Our Culture and others. Google approaches
things differently. They give their engineers one day a week to "do whatever"... just suggesting that as a possiblity in many companies will at a minimum get you a look like you have three heads.

One of the questions that DeMarco asks in Slack is "Is Google in a Hurry?" Certainly, from an outside point of view they don't appear like they are. If they were how could they spare time to let their developers chase ideas for a full 20% of their time? Google rolls out releases on a regular basis sure, but accross a very large product line and they are small. When they do roll out new products or significant releases they roll out a beta first, let people volunteer to move, thus getting a willing guinee pig audience who will tolerate some experimentation. Once the beta is solid they start to migrate people. (With the recent Blog reader full interface change they even let people flip back to the old interface for a period of time as long as they gave feedback on why.)

All of that doesn't amount to being in a rushed state, running at 120%. But, if you ask anyone about productivity they will certainly be mentioned and there is absolutely no question they are successful.

This trend has been seen in other companies as well. Apple for some time has been viewed as a company that runs at full throttle and approaches things differently. Amazon for a long time held the distinction of being the "What if Amazon entered this market?"

2007-02-16T08:25:00.000-08:00

Not dead yet...

Rumors of my death have been greatly exaggerated. Ok. So maybe there weren't really rumors of death. I did hear from enough of my regular readers though that I wanted to pop a quick post out to declare that I am still alive just HORRIBLY busy at the moment. I normally write my blogs at night and post when I get a chance but of late I have been working regular work during that time. Rude isn't it?

Rather than simply declare that I am alive (and whine that I am busy) I will share a bit of scalability learning as well. I was recently sent a link to a wonderful article on the stability journey of MySpace.com (if you don't know what MySpace is you probabl live under an internet rock) in Baseline Magazine. It's a fairly quick read and it discusses many of the standard things you learn with scale. (Partitioning, Virtualization, Simplicity, Segmentation etc.)

2007-01-08T13:16:00.000-08:00

The PM who cried Urgency

Have you ever been working on a project that you have been told over and over is urgent the date that was initially given to the customer has to hold? The urgency from that initial swag date builds from a normal project to a high effort project, on to a difficult project and on to
a death march?

A critical learning that needs to be accepted is that the purpose of a schedule is planning, not goal setting. A common problem is the initial SWAG (or ROM) becomes the cast in stone plan when the only thing that everyone agreed to at the SWAG was that it was wrong.

A general rule of thumb that I have found to hold true when estimating a project with initial information is that it is good to a rough magnitude of +200% to -50%. (and how often it actually goes to -50% is... let's just say infrequent.) This really means that the estimate is really intended for planning purposes to allow budgets to be roughly planned and dates for initial thoughts to be formed for go/no go decisions.

The unfortunate reality though is that in many cases the initial SWAGs are taken as gold. Plans and dates are communicated around them and then, in order to make the dates that have been "communicated to the customer" the development team is forced to become a team of super-heroes running a death march to meet the date on something that may not have real value in the long term. One of the ironies that I recently read about is that in many cases the death march project is one that has little to no value to the business overall and it seems that the only way the project was worth doing was if it met the impossible schedule.

All of this doesn't change the fact that if a plan missed the date, it was a bad plan. In many cases we try to blame the performance of the team, environment, technical difficulties or other things but the fact remains the plan is what was incorrect. It doesn't really matter why the plan was incorrect, it was. This means that we need to run performance checks earlier in the cycle. A post-project review is always a good thing but what if we did what needs to be done
up front?

While reading through various articles for this topic I ran across one on the Best Practice for Voluntary Overtime. It shows the correlation between an increase in pressure and productivity and then the corresponding decline in actual productivity when pressure is increased too high.
People are more productive and more creative and quality is higher when regular schedules are kept, planned and executed.

2007-01-06T14:20:00.000-08:00

My Recent Reading List

Over Christmas as well as the last little while I have read several books that will undoubtedly have an impact on my blogs over the next little while. So in the spirit of sharing (and encouragement of cheating) here is a quick list. If you are interested in reading them and taking
guesses on which posts entered the list of "I should post on that" feel free to send me email and I will confirm or deny. Of course I try to attribute anything that is direct quotes or go with a theme for a week so you will likely see these mentioned again.

Siddhartha - This is a book by Hermann Hesse about the journey of a man through his life. It is one filled with many levels of meaning and one of those books that you read and learn more about yourself than you expected.

Slack - Getting Past Burnout, Busywork, and the Myth of Total Efficiency this book by Tom
Demarco focuses on many of the challenges that face companies and leaders in the post-downsize, post-bubble work world. This book is a very easy read, fit for a two-three hour flight depending upon your reading rate. It tries to not just preach but also provide real world tools to fix concerns.

The 8th Habit - I have mentioned this book before and in fairness I have to say that
I am taking a while to go through it but it is not the fault of the book. The style is easy to read and the content is great. I have just had a lot going on and have taken breaks to read other books, read blogs, and play Fantasy Hockey. Being the latest in the series from Stephen
F. Covey and the gang at Franklin Covey crew you know that it is good stuff. This book
comes from the experience that the team there has had in teaching the The 7 Habits
to companies and individuals across the globe.

Along with these I have been reading lots of Blogs.

2007-01-05T14:17:00.000-08:00

What is quality anyway?

You could practically start a holy war with the number of interpretations of the word Quality and what it means in the context of software development. In some contexts it follows the battle cry of "ZERO DEFECTS". In other cases it is the cry of "no problems in production."
In fewer instances it is the question of "what will thrill our customers?"

In the grand scheme of software development and usage the first two are valuable, of course, but the last one is what keeps customers. In personally would maintain that it is the one that is most important. Of course customers won't be thrilled if the software is buggy, constantly
going down in real world use of inconsistent in operation. But with that basic understanding of a foundation on which to build what really amounts to quality is not pure defect free or 100% incident free systems but ones that meet customer's needs. There are multiple levels of meeting those needs since in business, the rules change very frequently requiring basic functions and a constant ability to adapt.

So what does this mean? To me this means we should set up steps that allow us to focus on quality of features not just quantity of features. Taking advantage of things like Test Driven design allows us to focus on the right features and changing things quickly to adapt to needs.
It means that we shouldn't hold on to every single feature that has ever been dreamed up and try to win a numbers race of total features. It means we should focus on delivering the right FEW features that are really what customers want, make sure that all of those items are fully
functional, high quality and stable in production.

Numerous stories of start up companies in the computing industry can be told regarding how a start including a SIMPLE product that was widely successful because those few features met a customer need. In some cases a need that wasn't even known at the time. Computers on a grand, multi-room scale existed for a while but the personal computing industry itself came when Woz created a simple computer with just a very few functions called the Apple 1. By keeping things as simple as possible a computer costing only $666.66 was created and took off like gang busters. Think about the productivity tools you use today like Word that started with little more than the ability to put in text and save it. (and man were they ugly when they started)

So... drive down the number of features we are pushing in a release. Focus on doing them really well. Make the features differentiating and easily assembled to allow customers to do the work that really needs to be done. Do the IMPORTANT, customer THRILLING things, first and REALLY well

2007-01-02T14:13:00.001-08:00

What I did over my Christmas vacation

A whole lot of nothing. Or at least that was the initial plan. Nothing becomes very time consuming. You can spend a whole day doing nothing and end up very very tired. I guess it's all about the nothing that you choose to do. I read some (expect some book blogs), surfed the web a lot, played with my new Dell laptop and DVD abilities and generally hung out with my family.

Main A/V gift of the season - Slingbox. Very fun little HD Slinging utility that I am just now starting to get going. Plus it's just so darn cute. In combination with the new 5 LNB receiver and box from DirecTV going in today with even more HD the DeVries family hockey cheering channel device will always be available.

Fun gift of the season - Roboraptor. Already found a set of forums with methods to hack them. What fun. One person even has his breathing fire. My wife, understanding her boys, got one for each boy... me and my son.

Best Gift by proxy - My son got a Lego NXT. This is the ultimate Son and Geek-Dad gift. Robotic Legos. With 4 sensor types, 3 servo motors and BlueTooth controllability its the ultimate in building block programming.

Of course there were other fun gifts as well but these were my immediate favorites.

2006-12-26T11:38:00.000-08:00

Drag Racing vs Rally Racing

Some of you that know me outside of my Uptime Blog posts know that I am a fan of Rally Racing. In part it is because I drive a Subaru so I was drawn to watch. (Yes, a blue STi with the handle on the back so that God can reach down and shake me when he thinks I am doing something dumb is mine.) But also I like Rally racing because it takes a real world car (granted, they gut the interior and up the strength a bit on the components) and drive it like fiends through corners, over dirt roads, through snow and over jumps. It is a real test of driver abilities and a lot of fun to watch or even play on a game console.

So what does this have to do with technology? It has to do with optimization and how optimized you can make a system, either people or computing. If you take the example of a Rally car they need to have horsepower yes, but that is not enough. You need to have torque, suspension flexibility, handling adjustability the ability to turn and to stop is just as important as the ability to go. In one stage you may be racing on a mostly strait asphalt road, in the next over a switch-back 180 degree turn infested gravel road through a forest. The ability to handle that change is very important to build into the car. Conversely if you look at a drag race car the need for handling, turning, even suspension decrease. What you need is horsepower and pure strait line speed (and hopefully the ability to stop, though if you look at some of these cars a parachute is what is used). How boring.

I maintain that business is much more like a Rally race than a Drag race. You never know what curve a competitor is going to throw at you next. The ability to handle those curves, stop on a dime, turn in the other direction and then go full bore ahead again is critical to business survival.

Sometimes though we seem to optimize our ability to go fast and go strait. We focus only on the immediate goal in front of us. Cut out items that are not critical to our achieving that strait ahead goal. We optimize our "people load", "trim the fat", etc. to keep everyone as utilized as possible moving towards the strait line goal. Then a change comes up... and we discover we can't turn as fast as when we started and we are surprised to find we even have a problem changing our direction simply due to momentum.

This is why things like Non-Functional Requirements are critical to a project and a companies long term success. They maintain the ability to turn and stop and give the slack necessary to allow a shift one step to the right. If we are all pushing so hard to go forward in a strait line even adjusting slightly to the side is difficult. In planning this is Risk Management. It's accepting that planning "if all things go right" is not going to really get you there.

Getting the right strategy means you have to assume your competitors are damn good, or at the very least as good as you are, and that they are moving just as fast or faster. When it comes to peering into the future, you just can't be paranoid enough.

- Jack Welch

2006-12-06T08:48:00.000-08:00

A Link, A List and a Quote

Not a big post for today. Just a simple one.

Mark Coulston recently published an article in Fast Company that is worth checking out. It is entitled, Don't Mess Up. IT lists the 10 things that smart leaders do to help them cope with stress that are actually destructive.

1. Avoid confrontation
2. Hire advisors, but don't listen to them
3. Don't accept when they're wrong
4. Avoid dealing with reality
5. Wait too long or not long enough to cut their losses
6. Focus too much on strategy vs. execution
7. Trust analysis over instinct
8. Trust instinct over analysis
9. Practice favoritism
10. Minimize the importance of what they don't understand

Mark also shares a quote and we all know I love quotes so here you go.

"Everybody here has the ability to do anything I do and much beyond. Some of you will and some of you won't. For those who won't, it will be because you get in your own way, not because the world doesn't allow you."

- Warren Buffett speaking the University of Washington

2006-12-04T21:15:00.000-08:00

SOA reduces integration costs

Many times business applications typically reflect a software vendor's opinion of how a business process should be performed. Since vendor opinions and real-world processes often do not match, many systems fail to support actual business operations. Easy examples of this can be seen in the big ERP systems that are implemented and "customized" for many millions of dollars only to need more rounds of "customization" with every new release.

One of the goals of Service Oriented Architecture (SOA) is to change the way enterprises deploy applications to support business processes. Using things like Orchestration and Business Process Management technology teams pull together the needed connections to components with a business flow based on how business is performed. Then when business changes (I know, I know.. that never happens...) you simply change the wiring. Of course, that is the goal... in order for it to be a reality some things need to happen. For a list of some of what I think needs to change look at my previous series on SOA changes.

What do you get for all of this effort?

Essentially, you get reduced integration expense. There are of course other benefits, but from a business POV this is what you get. It could be as simple as a reduced integration cost with other systems and future systems, or changing systems. It could be increasing reuse of assets; always a good thing to use something that exists vs build it from scratch each time. Increased customer responsiveness in the ability to change and modify business process more dynamically. You even get a reduction in risk, both business and project level risk, because you are dealing with smaller components that may already exist and allow for localized changes. But, as I started this paragraph, it all boils down to lower integration costs that enable all of these things.

Zapthink recently published an interesting article on the REAL costs of integration as part of their justification for SOA. The most valuable think in the article for me was the picture of integration costs by phase. Go check it out.

The REAL costs of integration - ZapThink

2006-10-21T20:20:00.000-07:00

Is complexity justification for itself?

Have you ever had this conversation?

"Why is this so hard?"

"Because our environment is so complex."

"Well, why don't we just use [INDUSTRY STANDARD XYZ]?"

"Because our environment is so complex; in order to support the pieces we have already we need to take [COMPLEX SOLUTION ABC]. The simple approach simply would not work."

"Wouldn't it be better to take the simpler approach and adjust the current complex components?"

"We can't, we have always done it this way."

Etc. Etc. Etc.

I hate that conversation. It get's very old.

Complexity does not justify further complexity. It may be hard... it will be hard to get out of that mode of thinking but it is necessary. While a complex environment may exit it doesn't mean that industry standard approaches won't work. It is my conjecture that complexity has become cultural.

Simplicity is the ultimate sophistication. - Leonardo daVinci

Simplicity is hard. This can be seen if you look at code from an experienced developer versus a new developer. Generally speaking the code will be more readable, more understandable and simpler. (Even though the algorithm may be more complex the elegance of the implementation makes it easier to understand.)

I will end with this quick request. If you hear someone using complexity as justification for further complexity call them on it. Press to see if a simpler approach is possible. It may be a little harder in the short term but in the long term it will pay off greatly.

2006-10-15T19:35:00.000-07:00

Is the question more important than the answer?

Right now I have 4 books I am shifting time between, one of them is "How to Think Like Leonardo da Vinci" by Michael J Gelb. One of the items brought by my Gelb is curiosity as one of his seven steps to thinking like da Vinci. In essence this particular step is all about asking questions. It is an interesting thing that as we grow and are educated we are thought to not question but rather to answer.

Gelb states, "In most cases, schooling does not develop curiosity, delight in ambiguity, and question asking skill. Rather the thinking that is rewarded is figuring out the "right answer" - that is, the answer held by the person in authority, the teacher."

In school we learned that the answer was more important than the question. Not just that, but that the answer that is desired by the prevailing powers that be is the answer we want to give. If we don't then we don't get good grades, if we don't get good grades we don't have as many options, if we don't have options we can't make as much money, if we can't make as much money then we can't have as good of a life... or something along those lines. People get caught up in trying to give the answer that their boss want's to hear. Or not asking the questions that they feel should be answered simply because that is how they have been trained, or that is the prevailing culture of the company.

What get's lost in the shuffle is the desire to question. The Wonder of the Universe. We all think of small children and asking why. For those that are not parents kids don't ask the questions you expect. Sure they ask why the sky is blue... but they also ask if everyone sees the same colors, how the sunlight travels if it doesn't have a car, and random things that simply don't make sense to the educated and it would never even have occurred to us to ask.

As I think about Engineering I think that this curiosity and desire to ask questions of what if and how and why are key to a successful engineering approach. In order to build effective systems we need to understand how things operate, what happens when something outside of the expected happens, how the system will respond to it, etc.

From these questions we come up with what we are calling Operational Models that show the threads, the Pressure Points, the potential hot areas of any application. From this we can plan capacity, plan protection and plan impact to downline and upline systems.

So, here is an exercise for you. (Borrowed freely from Gelb) Take a piece of paper (da Vinci. always kept a notebook or note pages with him, now called the Codex Arundel. Bill Gates paid 30 million dollars for some of them, the Codex Leicester... who knows, if you keep one maybe it will be worth millions some day) and write 100 questions on it. They can be about anything. Just let your mind wander. Then look for themes. Then explore those themes.

2006-10-08T19:28:00.000-07:00

Sacred Values, Organizational Hypocrisy and Innovation

I recently read a Cutter Edge email (if you are not familiar with Cutter go check them out) that I thought was very interesting. In case you haven't been keeping up with the news Toyota recently passed Ford in the list of World's Biggest Auto Makers. They are also putting focus on GM and trying to get to the number one spot. The Toyota Lexus brand recently took the top spot in 11 of 19 vehicle categories. The reason that Robert Charette wrote the Edge article is not all of these wonderful accomplishments but rather, what they are doing in response to a challenge.

Toyota has recently had some quality problems manifesting in an increase in recalls. The company has a choice with their recent success to continue to press on and work through these issues as time allows or to slow down and put focus on ensuring they are addressed. Given that they have come so far and are so close to their goals it has to be tempting to say that these are just short term problems and to press on as most any successful, large sized company would and even the public markets would expect.

But they are not. Toyota has given an apology for the problems and stated "There can be no growth without quality improvements." They have even assigned executive VP Akio Toyoda, grandson of the company's founder to be responsible for quality improvements. As part of their drive to increase quality Toyota has started several initiatives ranging from adding systems to track repairs to cars even after the warranty expires to hiring more engineers.

You read that right, in this age of continuing layoffs they are hiring more engineers.

As the company grew they added new systems, CAD tools, that would increase engineer productivity and reduce the number of physical prototypes needed. Toyota is recognized as the creator of Lean processes and manufacturing (and it is interesting to note that the problems are not in manufacturing, but in Engineering) However, there are limiters to what tools can do and with the increase in the number of initiatives this meant fewer eyes were cast on each design and "boneheaded mistakes" found their way through the system.

Robert discusses these items in the context of corporate risk management but also mentions Sacred Values and Organizational Hypocrisy. These are when a company says that something is important to it and then does not act like it in reality.

I am encouraged by the fact that companies are having new and creative programs such as Hack Day and other things currently under way in companies. To me this shows that at least some companes take innovation seriously. The question though is, do we take it seriously enough? Many of us are already working well over 40 hours a week on projects and problems that have been funded. Finding the time (and energy) to be transformationally innovative, not just iteratively innovative is hard to do and rarely encouraged.

The real proof will be when something difficult happens. Innovation is, by it's very nature, disruptive. Especially in large companies with business to protect. Read a recent blog post on how "Survival is the Mother of Invention" on the difficulties IBM went through for a very real example.

2006-10-07T08:23:00.000-07:00

Engineering Poetry? Limericks? Haiku?

Poetry for Engineers? I know.. it sounds a little out there and I am not just cheating and saying an Engineering degree from the University of Limerick. This was just too good of a follow up to the whole idea of there being other non-technical abilities that make someone a better programmer or Engineer. I encountered a news article on how Georgia Tech is now teaching poetry to their Engineers. By making this part of the curriculum the school is trying to teach students how to leverage their full mind and potential. One of the analogies that was used when describing good poetry could also be applied to good code.

"If something is good, it will stick around. That's just a fact of history. If something is not good, it will dry up ... "
Long, long pause for dramatic effect.
"And it will blow away."

Nicely said professor Lux.

The other aspect of this that I think is interesting is along the same lines as the recent flurry of music posts. Poetry is historically a right brain activity with some left brain rules applied for form. When done well it is a blending of the whole brain to get an answer. As pointed out by Dan Pink in his excellent book Whole New Mind the ability to use both halves of our brain is now required to be successful. If you are a rule and "left brain" only type of person... eventually your job will be automated or offshored. But if you can utilize your whole brain there will ALWAYS be work for you.

Consider this the next time you have to clean your toilet. (I know that seems like a sudden context shift, but humor me for a moment) Your toilet brush is no longer just a toilet brush. That Target toilet brush is not just a toilet brush, it is a designer toilet brush, selected from over 5 different hues and tones from shiny red to stainless steel to be uniquely you. That is how much of a world of abundance we now live in... we now include design in our toilet brushes. (No I did not enter a contest to see how many times I could put toilet brush in a post... but once I got started I just couldn't stop) Our parents and certainly our grand parents were thrilled if they could own one home and one car. Now... many of us have multiple cars.

Consider these things as you come up with your next personal development plan. Are you just looking at adding another language? Are there other things that you could take on to grow into a more well rounded and whole brained person?

2006-10-04T18:01:00.000-07:00

You have to feed a lion to know how hungry it is

I am reading a book (Competitive Engineering by Tom Gilb) which has a lot of information on requirements as well as systems engineering. Although the book itself is a bit of a Tome (read as difficult to get through in spots as it pontificates on things I (and most of the world) don't care about) it has some bits that are wonderfully insightful and keep me reading. I thought I would share a few of what I thought were fun quotes and then expand on them a little.

You must feed a lion to know how hungry it is. In the software world this can be stated as unknowable complexity, often described with the phrase "we don't know what we don't know". In essence what this is saying is that you can't have perfect knowledge up front about exactly what requirements are going to be needed and how exactly things should work. In Agile we address this by defining the set of stories then prioritizing and delivering early and often. Do the valuable things first. Many projects get their funding cut due to one thing or another. Projects that are delivering value, regularly not only fight the complexity problem by addressing things in bits and hitting the "goodies" first but also pay as they go allowing business to see value along the way thus reducing their chance of a cut. This statement also applies to the use of Spike solutions. If you don't understand something well enough to give an estimate... don't. Do a spike solution, learn about the problem, reduce the risk and make an estimate with more knowledge.

Even gourmet decays. Another way that I have described this is with the statement regarding requirements that "it's not that I lied, it's merely that the facts have changed since last I told the truth." Requirements change. Any requirement, as soon as it is completed begins to become less and less valid as the business environment and the world the requirement was created for changes. Again, in Agile we address this by delivering value early and often and only going into the nth degree of depth when it is needed. By working with customers on the most valuable items in the backlog for each iteration we are able to keep our requirements as fresh as possible, work on the "right things" and able to more quickly adapt by picking and choosing the stories that have the most value to us at that time. Recent research has shown that only 20% of the functionality in software systems gets used often or always... lets focus on getting that part done first.

Stop the world, I want to get off. We could no sooner say that we are tired of the world spinning and stop it than we could freeze all requirements up front and hope to deliver something that will truly make our customers happy. In Agile this is again dealt with by working on the most important stories first and prioritizing and selecting the new "most important" stories at the beginning of each iteration. By attempting to freeze all of our requirements and not updating them we would cause far more problems (not to mention churn or analysis paralysis) than if we just updated our requirements.

2006-10-03T21:34:00.000-07:00

Fantasy Sports

I have now officially fallen into the psychotic reams of people who are participating in Fantasy Sports. It started innocently enough with a begged request from a friend at Church who needed "just one more person to have an even number". As I started exploring just what I have signed myself up for I ran into an interesting example of what I posted about just the other day.

Hockey has always been my favorite sport. In part because the whole idea of Hockey has always been about the team to me. You can have one guy who is super awesome and he will be fun to watch but it takes a dedicated team to get the wins. As Ken Hitchcock used to say "Hard work beats talent when talent doesn't work hard."

And now... here I am looking at stats as the sole indicator of how good of a player someone is. Where I have always been an unabashed homer who cajoles the call guys for being biased if they say anything negative about my team... now I find myself looking at who is playing who and rooting for individuals so my "team" can win. Before my rules were clear... if they were wearing the wrong sweater they were evil and obviously a cheating fake, now things are more complicated. I used to be able to know they were evil by the Red Wing logo on their chest (until they left as a free agent and signed with our team at which point their sins were forgiven) but now I need to look and see if they are on my roster.

Ain't life fun? Sorry for this non-technical ramble of a post but I just found the whole thing particularly humorous given how quickly I find myself living contrary to my own post.

2006-10-01T21:32:00.000-07:00

Are we a world of individuals who don't understand teams?

The USA lost to Europe in the Ryder Cup this weekend. According to the news here in the states between this and the loss in World Basketball competition we have too much focus in the US on individual success over team success and this is hurting the U.S. in international sports. This has even been observed in the UK press. (By the way, congratulations European team, well done)

Why do I bring this up? Especially when you consider I don't watch golf (I enjoy golf but my game is one that makes other's feel better about their game) and I have an actual spoken distaste for basketball. (They just mess up my ice no matter how amusing Cuban is.) It's not because of a sudden care about international sports. It's because I thought it brought up some interesting points about teams and teaming.

As a race of humans we have not always been about the individual. Michael Gelb (yes, another mind mapping guy) point out in an article he wrote for Create Magazine that "Prior to the Renaissance, the notion of individuality didn’t exist, because the concept of individuality, as we now understand it, didn’t exist. Paintings, for example, remained unsigned, and painters, anonymous, because the individual was considered unimportant. All creative power was vested Above."

We all have a desire to do our own thing. Be our own person. Contribute on our own merits. Make our will known and get the recognition we deserve. etc. etc. etc. I don't think that in and of themselves these are bad things. They are probably good things. But teamwork is also something that is critically important.

Teamwork is the process of taking a group of individuals and making them capable of more than they would be capable of by themselves. Some of the most valuable individuals in any group are the ones who make that synergistic bonding possible. On sports teams these are the leaders of the team, the ones who inspire everyone when things are going poorly. In business they are the people who others watch and take lead from. The positive (and sadly sometimes negative) influence that raise everyone's mood, performance and vision.

It takes a bit of self-sacrifice to put the team ahead of the individual. This stems from recognition and a desire to personally excel. In an authoritative "Boss, Rules, Controls" culture individuals are actually disincented to promote the team. That is why we all need to encourage and "be the change we wish to see" (Mahatma Gandhi) when it comes to teamwork.

Do you know someone who helps other people around them become their best selves? Talk to them today. Say Thanks.

2006-10-01T07:29:00.000-07:00

Tenets of good SOA programming

Today's post is a follow up to my post on complexity being a math problem. SOA programming just like Object Oriented programming has a select set of steps and principles that need to be followed in order to ensure the most effective use of these technologies.

Most programmers can spit out the rules of good Object Oriented design in their sleep. These same principles are very similar in my opinion to the tenets for good SOA design. OO Tenets are Abstraction, Encapsulation, Inheritance and Polymorphism.

Abstraction means ignoring irrelevant features, properties, or functions and emphasizing the relevant ones… relevant to the given project. This also covers Data abstraction simplifying.
Encapsulation means that all data members (fields) of a class are declared private. Local use methods may be private too. The class interacts with other classes (called the clients of this class) only through the class’s constructors and public methods.
Inheritance states that a class can extend another class, inheriting all its data members and methods while redefining some of them and/or adding its own. Inheritance represents the is a relationship between data types. (e.g. a FemaleDancer is a Dancer)
Polymorphism ensures that the appropriate method is called for an object of a specific type when the object is disguised as a more general type. (e.g. Square being used like a shape)

To keep things simple and play off the magic number of four tenets these are

Boundaries are explicit
Services are autonomous
Services share schema and contract, not class
Compatibility is based upon policy

Boundaries Are Explicit basically means that a caller of a service need only (and for that matter, should only) understand the interface to a service. It does not need to know what is going on behind the scenes nor should it care. This enables the implementation to evolve over time without requiring changes in its consumers. This abstracts complexity and is core to making SOA work.

Services are Autonomous is setting up services so that they are independent and able to function on their own. While a set of services becomes part of an orchestrated whole and thus build a system they should be designed in such as way as to be as self governing as possible.

Services Share Schema and Contract, Not Class accessing systems should care about the data and schema that are received and the data that is required for a service to be used. This should be viewed as a contractual interface between the caller and the service. Callers should not expect a Class file from that service as services are about data and schema not specific implementations. When integrating systems and performing remote calls the expectation and integration at the class level is fine, but it is not a service.

Compatibility is Based on Policy means that functional compatibility needs to be managed. Versioning of services and ensuring that they are backwards and forwards compatible is a wonderful goal but may not be fully achievable. This needs to be accepted and managed through policy.

Looks like there is ample ground for posts on this topic. What do you think? Do these Tenets make sense? Should there be others?

2006-09-30T19:12:00.000-07:00

Why Open Source Will Rule the Software World

About five years ago a CIO that I worked for once told me that eventually all "for profit" software companies would be overrun by Open Source software companies. I thought he was absolutely crazy at the time. Sure Apache had made in roads in regard to the web server market share at that point and Tomcat was doing ok as the reference implementation for Sun's Java servlet container but... come on. But I did stop to think about it... This guy was a retired partner from a major consulting firm who was really working because he wanted something to do and had proven on many occasions to be a very insightful and brilliant man. Now, looking at the Open Source software world I am convinced once again that he may have been crazy... but in that "there is a fine line between genius and insanity" kind of way.

It looks like even world beater Cisco may not be safe from the pressures of Open Source. A project called Vyatta is picking up steam and really looks like its is going to bring some disruption to the otherwise boring networking gear market. Why is this simple little product disruptive to a 5 billion dollar industry? Well... its an open source router... with all the features of high end network equipment... at one fifth the cost. This product is currently in beta but due for release soon. In an recent article in Business 2.0 it was explained that this project was started because researcher Atanu Ghosh was studying the future of broadband and knew that to make changes to router software, he had to submit them to Cisco or some other slow moving network leviathan. So he and some colleagues decided to write their own, caught the eye of an early Cisco employee and some venture capitalists (there is an interesting note in and of itself... venture capital firms are now funding open source software companies...) and viola. This will be a fun one to watch.

Bernard Golden had an interesting series in his blog on CIO Magazine's blog site about why open source is not only a safe choice for companies but the smart choice. In the latest installment he takes an example of a company with 32 million in revenue that has chosen JBoss as their application server. They could have chosen BEA but instead of paying 500k they paid 50k. What happened to the other 450k? That 1.5% of their annual revenues would instead be able to go into product development, salaries, or any of the other things a growing company needs.

What has made this shift in perception and belief from open source being cool but for companies who could risk their own support to software that is available and real for everyone? My opinion is that the ecosystems for open source has changed. Companies are seeing both the financial benefits of open source. Really, off the shelf software has always been a strange model with all costs in the construction and after that it essentially being free. (maintenance, support etc obvious exceptions and in custom software where the majority of the costs actually end up) Now you can download the software, see if it works for you, if it does you can get support from third parties if it is needed or even participate in the open source community as a company for the support you need. (Who would you rather have answering your email for support, a call desk person who knows what's in a system from the manual and has a series of steps to obfuscate real help or a developer who develops code in the system as part of their job?)

2006-09-29T09:00:00.000-07:00

Symphony separates good developers from great developers

I have often observed that some of the best developers I have met have a background in music. I have done a bit of thinking about why that could be and I have a postulation.

When you learn to play music you learn how one item by itself can make music. Closely after that you learn how adding additional instruments adds a whole new aspect to the music. Then, when you add more instruments, of different types you get yet another aspect. One after another the symphony builds the whole becoming greater than the parts.

I think that it is this appreciation for the connectedness of seemingly unconnected items that makes developers with a music background able to make better code. Of course just as not everyone who has ever picked up a violin is a stellar developer the understanding of symphony is not limited to those who have played an instrument. The interrelation of things is something that anyone can learn and observe.

2006-09-27T20:45:00.000-07:00

When you are going through hell keep going

I heard a construction worker make this statement and it struck me as so profound that I thought I should write a post on it. It is often said that the hardest part of any journey is taking the first step. This can be seen in almost every approach to time management.

Step 1 - break the problem into small, easily consumable chunks.
Step 2 - get started.

But into every perfect plan some problems will fall and how we react to these problems defines who we are. John F Kennedy once said "History will never accept difficulties as an excuse." So we need to make sure that we adapt, move forward and keep things going. In business and life you are not successful if you try really hard and have good excuses why you couldn't get what needed to happen to happen.

If you think about it, the people that we all like to work with the most are the people that have positive energy. They know what they want to get done and choose to make the best and most out of every day. They are grounded people who know who they are and are determined to meet their goals and they can be anywhere in an organization..

I am a big believer in the sentiment of if you never break anything you are not doing anything of value. A good chunk of the long term term success of that statement though is that you don't give up. Thus the profoundness of the statement... if you are going through hell you keep going. To me this equates to sticking to your guns and not giving up because things get hard. It doesn't mean that working 120 hours every week is ok. Sustainable pace is necessary and we need to plan the right resources, phases and release plan to make sure things go right.

2006-09-26T09:52:00.000-07:00

SOA Day 7 - From Object Oriented to Message Oriented

Day 1 - Connections = Cost to Connections = Value
Day 2 - Function Oriented to Process Oriented
Day 3 - Build to last to Build for change
Day 4 - Prolonged Development to Incremental Deployment
Day 5 - Application Silos to Orchestrated Solutions
Day 6 - Tightly Coupled to Loosely Coupled

This is the last in my series of mental shifts. Hopefully people found some value in them rather than me just preaching via electrons. I truly believe that if we take these simple shifts to heart we will have better overall systems not just a good SOA architecture. That said, on to today's post... shifting our mindset from Object Oriented to Message Oriented. This is not to say the Object Oriented programming is evil but rather SOA is an offshoot from Object Oriented development focused on integration of components and communication between them and as such needs to be approached differently.

As we start to change the other aspects of our mindset we will be required to change our thinking from specifics of application development to how application components will communicate and connect. Since we will have loosely coupled the application components we will need to think about how we will message between them. In order for orchestration to work we need to know what will be sent in the communications between components. Since we will have components that will evolve incrementally we will need to build messages that will evolve gracefully as things change.

By shifting this component we will be able to safely perform actions that before required synchronous communications to those that are asynchronous. With this shift we will be able to take advantage of queues, allowing our application components to consume and compute as fast as they are able. We will be able to maintain protections of components and between components because it will be built into the architecture.

Obviously none of these things are silver bullet's but are again steps towards allowing higher level computation once again. Just as Assembly gave way to C which gave way to C++ then to Java as languages so too will our application communication metaphors continue to evolve.

2006-09-26T09:39:00.000-07:00

SOA Day 6 - From Tightly Coupled to Loosely Coupled

Day 1 - Connections = Cost to Connections = Value
Day 2 - Function Oriented to Process Oriented
Day 3 - Build to last to Build for change
Day 4 - Prolonged Development to Incremental Deployment
Day 5 - Application Silos to Orchestrated Solutions

Today's mental shift is one that I have blogged about before in other forms; shifting our mindset from Tightly Coupled to Loosely Coupled. Coupling is another word for dependency, specifically the degree to which a program module depends upon other program modules. A certain degree of dependency is obviously going to occur within programs that actually provide value. The key is to make it no more than absolutely necessary.

This is another point that builds off of Day 3. By loosely coupling code we will be better able to "roll with the punches" or shift along with the requirements as they change. Good architectural design abstracts out the bits that don't need to be specific and keeps the coupling from being overly tight.

With SOA this is even more important because one of the classic issues that is seen with services is versioning. One version changes just a little bit and causes a ripple effect. There are patterns and best practices that mitigate this and... what do you know, they are focused on reducing the coupling and providing loose version ties.

Libraries and functions become services in themselves. Using orchestration to pull the pieces together we are then able to adjust simple properties rather than needing to rewrite and recompile code. By keeping the coupling loose systems can use underlying technologies that work the best for the application instance at rather than forcing separate components to be tied to the exact same approach. Of course, for all of this to work a shared semantic framework is required to ensure that messages have consistent meanings across them.

2006-09-24T10:41:00.000-07:00

Day 5 - Application Silos to Orchestrated Solutions

Day 1 - Connections = Cost to Connections = Value
Day 2 - Function Oriented to Process Oriented
Day 3 - Build to last to Build for change
Day 4 - Prolonged Development to Incremental Deployment

My next platform upon which I am pushing for mental shift is fundamental to several areas both organizationally as well as for projects and management. This shift is from Application Silos to Orchestrated Solutions.

It is tempting to think about applications as separate silos of specific functionality focused on particular business value or individual function type. However to really take a step forward we need to shift our thinking. Components that do searching require input and provide output. Components that control inventory take input and provide output.

If we simplify the components to their base functions and then orchestrate these solutions we enable true reuse. Components can be very specialized and it is the hooking together of the various pieces that will enable the overall solutions. Orchestration solutions are everywhere these days. Whether it is an Enterprise Service Bus or just simple wiring together of services (personally I think in many cases an ESB is overkill but as always, it depends upon the situation. Heck, there is even Open Source ESBs out there now such as Open ESB on java.net, Mule and ServiceMix from Apache as well as the zillion dollar solutions.) pulling the pieces together allow for complete solutions to be built.

This type of change is difficult to implement because there are many things that need to be specified, handled and even governed in order for it to operate smoothly. For example, when a standard messaging type or service setup is created in order for these shifts to work consistently everyone must adhere to them. You can't have someone going off and building a different service approach because they don't like the current one or didn't feel they could comply and hit a timeline.

The benefits of this approach though are many. You can really take advantage of reuse with this since the components by definition are reusable. Data that would otherwise drift to slightly different implementations behind different systems is kept consistent. The ability to get things done with a controlled budget is enhanced simply due to the reduced maintenance. Maintenance consumes most of any system budget, ironically with a successful system the longer it has been around, the more the maintenance costs become.

So while this shift requires a lot of effort, if the standards can be created and adhered to the value truly comes forward.

2006-09-23T19:38:00.000-07:00

SOA Day 4 - Prolonged Development to Incremental Deployment

Day 1 - Connections = Cost to Connections = Value
Day 2 - Function Oriented to Process Oriented
Day 3 - Build to last to Build for change

Today's mind shift is to shift how we develop and release. Prolonged Development to Incremental Deployment . If you have been listening to the Agile speakers, reading software development papers or news or otherwise been paying attention chances are you have heard this one mentioned. This mental shift is one that is under way. Iterative development has been in place for some time and really has good traction.

My point on this one is we can't stop with just iterative development. The best feedback that we receive is when something is in use. In order for it to be used it needs to be in production. So this means we need to develop things in verticals of functionality that allows us to get them out and in front of our customers. A wealth of ills can be addressed by this. It does require a more deliberate approach and understanding of your overall application architecture. With this though better quality can be released and built.

Just like a fractal grows and grows making the picture more and more complete so does a SOA based architecture. Each bit released enables other bits to be built or released. By releasing these bits in a planned manner as quickly as possible rather than taking months to develop the "complete solution" we can steer our development to what is really needed. Just as I said in Day 3 where we Build to Change we can accept that our requirements are going to shift and be ready to adapt them faster. Focus on business value not the "complete solution". Complete solutions are a fallacy because if there is business value in a developed system more development will be needed.

2006-09-17T21:32:00.000-07:00

SOA Day 3 - We have to shift our mindset if we want successful SOA

Day 1 - Connections = Cost to Connections = Value
Day 2 - Function Oriented to Process Oriented

Continuing the series of mental shifts today we will cover how we build things and the development approach needed.

From Build to last to Build for change. This is another one that people struggle to get out of. It is tempting to think that we are building a system or component that will last forever, that it will not need change. To support this we tend to build in "what if's" and architect a solution until we are blue. What we really need to do though is focus on building in abstraction layers and flexibility, accepting that the system and requirements are going to change.

An interesting item in architecture is that those requirements that have the most churn about them are the ones that need the most flexible designs. A good portion of this flexibility can (and should) be built in through abstraction. If it doesn't matter what whatchamadigger you have behind the library because the library is generic then it gives you the ability to change along the way. This goes for both custom/internal code as well as external components.

A good example of this work the various MOM efforts everyone seems to be working on. Currently they are working on building abstraction libraries for several potential message queue systems. Websphere MQ, JBoss Messaging and SeeBeyond are all potential solutions and rather than building to one specific solution a better approach is to build interfaces that abstract out which mechanism is used behind the scenes. In this manner the teams who are doing the integration don't need to worry about the plumbing and can focus on the application logic needed to differentiate applications instead of the things that should "just work". Another major benefit of this approach is that it frees teams to move to new and different technologies with minimal rework and this is where real negotiation power with vendors and true flexibility in technology by avoiding lock in can be found.

2006-09-17T21:29:00.000-07:00

SOA Day 2 - SOA Mindshift - Function over form Process over procedure

Day 1 - Connections = Cost to Connections = Value

Day 2 in the series of SOA mindshifts we need to make in order to ensure a successful SOA architecture.

From Function Oriented to Process Oriented. Thinking about functions is usually easier since that is what we break things down to from a development process. The subroutines that pull together a function makes it possible for us to take things in smaller chunks. Functions though are not what provides a business with value. Any paticular function is only valuable if it enables the business processes in which it sits. If we can make ourselves think about the process rather than sub-optimizing on the functions we can drive a better overall solution because we see the bigger picture.

This point hits many different areas. If we consider logging as an example we would rather know that a process path for booking has gone down than that a specific function within that path has gone down. With the information at the business process level we are able to know what other business processes are impacted. As technologists we want to know what piece is breaking but really that is part of the detail of fixing the problem... the business wants to know impact. Other areas are similarly impacted.

Another benefit of this mindset change is that we will naturally start to take a few steps back when modeling out problems. In the "can't see the forrest for the trees" sense it becomes easier to see the forrest because we are looking at the process and how the trees interact rather than studying the bark and how it is brown. Focusing the whole picture is important for many reasons.

2006-09-17T21:25:00.000-07:00

Mindsets must shift if we want successful SOA

In my next series of posts I will propose a series of mental shifts that must occur in order to allow effective SOA to become a reality. Each of these will be proposed in a From X to Y manner over the next several posts. Let me know with a comment if you if you think I am crazy or if these make sense.

From Connections = Cost to Connections = Value. For a long time we have had the mentality that additional connections drive cost. In an SOA environment this changes. The more connections we have the more valuable our overall solution becomes. (My informal polls tell me that this is the one that the most people struggle with switching).

This point can be seen in my earlier point on complexity being a math problem. The more and easier it is to connect the more viable business models, execution methods and doors open up. Coase Theorum has been hard for some to accept as meaningful for technology conversations because they struggle to see a connection as anything other than a cost. In the mainframe world where there was a limited number of connections this was obvious. Now that connections are as ubiquitous with the proliferation of the intenet and IP connectivity the equations have changed.

2006-09-05T21:36:00.000-07:00

A gap between stimulus and response

Not to long ago I was introduced to the concept of The Singularity. The idea behind this is that eventually computers will improve to the point that they can be smarter than human beings. Essentially this is when computers reach the point that they can improve themselves. The idea of the singularity is that at this point it becomes impossible to tell what will happen next. A similar theory called the "empty planet syndrome" involves nano-technology and a similar sort of improvement to a place that we can not comprehend and with a thought we could wipe ourselves out.

Of course there is debate about this and it's possibility. Movies like I Robot or The Matrix and others of their ilk point to a rather scary future. On the other hand though I think that there are several reasons why this won't come to pass.

My favorite reason (yes, I have favorites in my own head) is that computers are like incredibly powerful left brains but they lack right brain ability. Most fundamentally they lack the ability to truly choose their own actions. This choice is why someone who grows up in a family with a long long history of "specific problem X" can overcome their biological programming and choose to become something else. In the end a computer will only respond to the ones and zeros at the heart of it's algorithms. In AI research it is interesting to watch how once a new and great AI algorithm is created... it's not longer AI... it's just an algorithm... just code. Humans have a gap between stimulus and response and thus can choose their own actions.

Computers handle logic and rules and conditions with amazing ease. This is why Deep Blue was able to beat chess grand master Kasparov. While in the early days Kasparov was quoted as saying he would rip the computer to shreds he inevitably lost again and stated that computers would continue to win. Just as John Henry was beaten out by a machine queuing up the industrial age so to did Deep Blue usher in the information age. All of that wondrous tech aside... computers still can't recognize a face. (Algorithms are being built that key off of facial features, voice tones and other previously right brain lock activities and they will continue to improve but the basic limits are still there.)

2006-08-04T05:59:00.000-07:00

The principles of complexity

I regularly hear how complex our environment is and how many different systems, platforms, methods and names exist within the Sabre enterprise. Internet standards, specifically those around Service Oriented Architecture (SOA) are an effort to drive a simpler approach to systems and specifically the connectivity of systems. I read an interesting interview a little while back that referenced Coase Theorem regarding the interconnectivity of systems. Ronald Coase won a Nobel Prize in 1991 for the Theorem for its impact in economic modeling and in the interview it was mentioned in the context of connecting systems and that got me thinking. The original notes and thoughts had nothing to do with computer systems but the parallels are very interesting.

A simplified (grossly; My apologies to Mr. Coase) view of the theorem is that when the cost of integrating processes is high then it makes more sense to do things yourself. As the cost of integration is lowered the economics change. If the transaction costs can be modeled then you can run simulations and determine the best possible solution for your own approach. This means that as network communication costs get cheaper and cheaper and the cost of information is driven down that new models and approaches suddenly become viable that may not have previously been possible. Take the well known example of amazon.com. Initially, all Amazon did was take free catalog information, used it to build an online catalog and used the internet to make sales at a discount since they don't have the traditional overhead of a brick and mortar store.

Even following Coase Theorem there are three main factors that inhibit efficient allocation.

Imperfect information - If you don't know the costs of the transaction or it is uncertain or unstable it is difficult if not impossible to model
High transaction cost - High transaction costs is the core of the theorem and driving down these costs becomes paramount to flexibility within the model. If any one item within the formula is large it can dominate the costs of the system.
High negotiation costs - If the costs to negotiate the transaction are high it quickly erodes the value of the external connection. This could be due to too many parties in the negotiation or the overhead of the conversation to establish the connections.
A fourth item, not core to the theorem that is used to argue against the ability to effectively model the costs is in cases where the actors in the system have irreconcilable different valuations on the transaction.

SOA is intended to be an enabler to drive down the cost of connections of systems. By making integration and calls between systems as simple and cheap as possible it enables a whole new way of building systems. More focus can be placed on the Business Logic layer and the orchestration of the services to unlock business value. As services are built, focus can be made to make the individual services as efficient, cheap and scalable as possible. The cost of building new systems goes down as well enabling new and more creative businesses to be explored.

In order for this to work though there is a few big simplifying assumptions. One, that the cost of integration really is lower. This doesn't just cover the development costs that are immediate and visible but also the long term costs and benefits. Maintenance should be cheaper if services are specialized and localized. For Maintenance to be cheaper services need to be stable. This speaks to the fourth item of irreconcilable different valuations. The best services (cheapest in terms of this current discussion) are the ones that can be taken for granted. Perfect availability is the goal, just like with phone systems you don't pick up your handset hoping that there is a dial tone, you expect it. For computer systems that doesn't mean that you shouldn't harden your own system to protect itself from harm because you expect other services to be there. Quite the opposite is true. The phone company gets their "always on dial tone" through A LOT of redundancy, automatic system corrections and advanced error detection and response frameworks. ( Mary Poppendieck discusses in her Lean conversations how testing to find defsfects is waste while inspection to prevent defects is essential) These same concepts apply to good software design. It is also important to consider the cost of the negotiations. In the case of
systems that means that getting your connection and utilizing it needs to be cheep as well.

This post is beginning to get a little long so thanks to those of you who rode along with me this far. I will follow up in subsequent posts on the Tenets of good SOA design and what we can do to approach our own level of dial tone.

2006-07-03T19:38:00.000-07:00

Vision without execution is daydreaming

For those who haven't been paying attention to the news, Bill Gates has decided to take a step back on his active involvement in Microsoft and increase his focus on his foundation. You can say what you want about Microsoft and Gates' business practices or technologies but one thing that is undebatable is Gates' effect as a Technical Leader and visionary.

In one of my personal favorite leadership definitions Steven Covey defines leadership as communicating to people their worth and potential so clearly that they come to see it in themselves." A great quote, similar in power to today's lead in from Bill Gates that "vision without execution is daydreaming". Bill has been a galvanizing force for Microsoft and the technology industry as a whole for some time and while it is doubtful that his decreasing involvement spells the immediate downfall of Microsoft it certainly indicates change is coming. Ray Ozzie is another great technical leader and visionary, the father of Notes and then Groove Networks he has a definite track record. But when it comes to attracting and retaining talent, even my Mother knows who Bill Gates is.

Technical leadership is an important aspect of any company. There needs to be a Yin and Yang relationship between business and technology. If that relationship gets out of whack in either direction a companies ability to execute suffers. If the business has too strong a voice then things like deadlines and functionality get pushed. If technology has too strong of a voice functions that may not be long term valuable get pushed, beeding edge technologies and increased risk are pushed. In reality there needs to be a set of trade offs between both sides to get the best possible solution.

This active conversation between business and technology is one of the big reasons for agile methods. By going through stories for prioritization and assigning costs for each along with a fixed velocity a team can concentrate on delivering the most reasonable business value in each release. Research from The Standish Group has shown that only 20% of functionality that goes into a system is Always or Often used and the majority (64%) of things that are created are Never or Rarely used.

2006-06-28T17:37:00.000-07:00

Wikipedia defines engineering as:

Engineering is the application of scientific and technical knowledge to solve human problems. Engineers use imagination, judgement and reasoning to apply science, technology, mathematics, and practical experience. The result is the design, production, and operation of useful objects or processes.

I think this is a great definition as it doesn't dig into the specifics of implementation but rather focuses on the what, why and desired outcomes. As we look into the engineering discipline we find that a heavy dose of imagination and creativitiy is required to come up with quality solutions to technology problems.

One of the items that this definition brings up is intuitive problem solving. This is the idea that each individual will have a set of experiences that they apply to intuitively understand and resolve a problem. I see this sort of problem solving regularly within our my own teams. As we dig into a solution, individuals raise questions like "what happens when one of the nodes goes down, sure the solution will fail over but can the remaining node handle the reconnect traffic?" What this usually means is that they have successfully broken that paticular item in the past and bear the scars to prove it. Technology Scars = Better Intuitive Problem Solving. So then the question becomes what can we codify to accelerate that experiential learning curve for other people?

One interesting thing I have observed is that many brilliant technologists that I have run into have had a background in music. It's almost as if having played an instrument opened up certain portions of the creative brain and enabled them to think differently about problems. Your brain is like a muscle after all and if you don't excercise it it will atrophy. Creativity in concert with logical problem solving seems to be key to approaching any complex problem. If I can divide a problem in half, determine the side with the issue and then devide the identified half in half and do it again I will eventually find the source. What I think this means is that good engineers need to have a balance of left and right brain.

2006-06-27T19:51:00.000-07:00

Welcome to my little corner of the Internet. Where I can conjecture and cajole and care not a wit if anyone else reads, feels or deals well with anything I say. Ain't the Internet cool.

Just to set expectations (other than my own) I will blog about

technology stuff - I am a geek... I like to think of myself as a high end geek... but that's also sort of like saying I am white trash with money
Scotch - It could be that the tech sector has driven this desire to drink into me, but either way it's something I enjoy
Agile stuff - Sort of related to the technology stuff but a bit more on the fluffy side
Miscellaneous - whatever doesn't fall into the above categories... I refuse to contain myself.. it is my corner of the Internet after all... here I am LORD OF THE LITTLE PEOPLE
Meaningless information - I am frequently told I am a wealth of meaningless information as I read near constantly, remember random (importance debatable) items (though names elude me) and otherwise chase bunny trails of conversations (I have been accused of maintaining 2-4 streams of thought conversations at time with a single person... apparently not everyone enjoys this :) )

Anyway, stay or run screaming at this point this post is done.