The Cathedral and the Bazaar and the Ignorance of Crowds


CIO.com recently did an interview with Linus Torvalds and the rules he lays out for quality code development are very good. It goes well with my theory of each development team needing a Benevolent Dictator role.


Some time ago I was forwarded a great article on Innovation as it relates to Open Source. Several of the key observations were saved for the last page. I thought it would have been good to draw some suggested next steps so I thought I took a crack at it and the items below are the result.

The article by itself has a good deal of value. But for complete understanding you probably need to have the previous context (inferred from the article is probably sufficient for basic understanding) of The Cathedral and the Bazaar. Wikipedia has a good simple summary of the paper. At it's simplest though it is a paper written by Eric Raymond about how "given enough eyeballs, all bugs are shallow." This means that it becomes easier to drill to quality because more people with disparate backgrounds means better test coverage and bug finding. (And a lot more but that's not what this post is about.)

My actual observations from the Strategy and Business article: I have historically referred to the needed role to make any product development work, whether Bazaar or Cathedral, as the Benevolent Dictator role. It is someone with the vision and strength of personality to push or pull those around them into a unified functional reality of a product. Without that role you get a lack of direction and vision and problems ensue. We have seen this need prove out over and over with the hardening work that we have done. Those teams that had a Benevolent Dictator that laid down tenets for development, surrounded themselves with a “Wizard Enclave” of smart folks for enforcement and saw it through generally had much better systems. This showed in test coverage, uptime, ability to adapt and change and many other areas.

I am a big fan of Open Source developed products and have been for some time. It has certainly proven to be the case that those that are successful have this Wizard/Leader/Benevolent Dictator that surrounds themselves with the Wizards that get there via meritocracy. From this “Brain” squad comes unification and initial innovation. Ongoing innovation can be pulled from the Chaos of the Crowd but to become real will need to be driven by the Benevolent Dictator.
When it comes to our own efforts, Internal Open Source and Product Development need to be set up with this type of role and if we can’t identify someone to fill this role then we can’t honestly expect to move that product forward.

Creating a customer obsessive culture

I read a post from Harvard Business recently that was so head spinning I have to share. I have subscribed to the HBR Ideacasts (easy to add to your iPod via iTunes) for a while and listen to them on my commute to work. They are both insightful and quick and easy to understand. I have access to their published components through work and the electronic lead ins provide a great way to focus on the stuff that actually matters.

An example of that is a post titled Why Zappos Pays New Employees to Quit - And You Should To. The idea of giving people a $1000 bonus to quit sounds absolutely insane when you first look at the surface of it. Especially when you consider the high cost of talent acquisition, nut then when you think a little deeper it starts to make sense.

The people who would take such a bonus and leave are people who are asking themselves the question of "what have I gotten myself into". They may be thinking, "this is all well and good but is this really for me?" In most situations when you start a new job you are so focused on pleasing everyone you hardly stop to think if the fit you thought was there really is. As an employer you are also focused on trying to make sure someone fits and gets all the opportunity to succeed they may need. The perception of someone leaving is so very negative that no one wants it.

By taking this approach Zappos short circuits all of those negative items. It becomes not just ok, but encouraged to consider your fit. It becomes not anathema but perfectly acceptable to call it quits when there is not a fit.

It may not be something that every company goes out and does right away (or potentially even should ever do) but I certainly applaud the Zappos team for thinking differently and realizing how to get to their real goal of excellent customer service with fanatical employees.

Brilliant.

Data is no longer relational

Ok, so the headline may be a little of an over statement for effect. Perhaps I could have said that data that people are interested in is no longer relational but then it wouldn't have been nearly so pithy.

With much respect to Edgar Codd and his invention of the relational model for database storage I think it is time to move forward. Relational databases are great for things like financial models, personnel data and other Enterprise Systems as well as many other standard, repetitive data. It gave a solid reference point to learn data structures and modeling to several generations of budding Computer Scientists.

When I say it is time to move forward I am not saying we should immediately move all data systems to Object Oriented Databases or otherwise induce data chaos. What I do want to push is the idea of unstructured data. Computers are great with rules, with structure and with fundamentally binary relationships. Algorithms are starting to mature around unstructured data (for an example go search google.) but it is still not widespread or well understood.

New exciting algorithms such as Amazon's Dynamo (Werner Vogels is one of my favorite speakers and bloggers on distributed tech, if you are not familiar with him in this space you should be) database are showing in real world situations that distributed systems and distributed data are a reality.

In a lot of system designs because people are so familiar with relational data structures and systems we find Object Models that look like a relational database design. When asked why it looks like this the answers are fairly consistently things like "this is how the database stores it, for speed we need to do the same" or "it just made sense when we pulled the DBA in to help us with the model."

Objects are not relational! They are objects. Then when you get into full structures of objects or object trees there are relationships but it is not the same as a relational database. Especially as we start to build and mature distributed system algorithms it doesn't make sense to use a centralized data store. If the data can be broken up, distributed and stored with the algorithms that will use it performance will improve.

In fact I would argue that the elusive SLA of a system response can begin to be discussed if we can tie the data to the processing. Granted there are new complexities in this model for synchronization, segmentation and consistency but there are ways to solve them. Similarly consistent access to the same servers is also possible.

What other great examples of distributed computing and distributed data storage have you seen?

A time management observation

Have you ever noticed how critical decision meetings have a tendency to be scheduled at the end of the day? I have noticed this as a trend recently in several of the streams that I am working on.

My first thought was that this was due to the need to work through the day to get the information together in preparation for that meeting. Anecdotally (meaning stuff I have seen, done or messed up myself but may not be statistically significant to the world at large) this seems to be the case in 20-40% of the high criticality meetings that get scheduled. This leaves a gap of 60-80% that could happen at other times in the day.

At the end of the day you are tired. At least (anecdotally) I am since the day has been spent making decisions, digging into problems and hopefully doing real work. So my next thought was why do we push these important meetings to late in the day if we are actually going to be less well equipped to handle them?

The answer (or at least my version of it) came to me as I thought about the tasks list and task list management that I do. Each day I follow the process below.

1. Review a list of tasks. I assign each one an A, B, or C. This roughly amounts to A - MUST DO, B - Good to do, C - No chance I am getting to this, I could schedule it forward now but then that would take time from getting to the A and B stuff.
2. Assign a number to each task starting at 1 for each letter category. (Yes, I never even bother with C)

Now when I do this I can say A1 is more important and I should work on it first. Great! Only my human nature kicks in (chances are if you are reading this you have one of those as well. We tend to keep them in closets and they can get dusty but they do peak out at the worst of times) and suddenly I find myself drifting down the list to item A4 or (say it isn't so) a B1... because it's easier. My little human nature side lives for that little endorphin rush of the check mark so... hmmm... off I go.

With meetings I think we do the same thing. We schedule some simple meetings up front. Easy discussions, standing team meetings etc etc. Then we postpone some of those really important, hard meetings and conversations to later in the day.

So I am going to try to move my important meetings earlier. When I find myself setting up one of these meetings I will catch myself and ask the question "would this be better in the AM? Or now?"

Am I nuts? Is this just how business is supposed to work? Have you seen this? In a global economy with companies getting more global in audience as well as employee base these types of behaviors become not only self-pain inflicting but they don't even make sense given global time zones and the fact that the sunlight moves.

Creation of new is playing with a purpose

The creation of something new is not accomplished by the intellect but by the play instinct acting from inner necessity. The creative mind plays with the objects it loves.
- Carl Jung

I ran into this quote today and thought that it was simply excellent. When the "zone" hits and you are creating anything new, be it a program, a system, art, music or a new business model it feels like playing. When you watch someone who is really good at it, they almost seem to be having too much fun... it can't possibly be work that way right? I have always found so much enjoyment in these types of activities myself that to describe it as anything else is pretense. When your brain is in the right state, you are relaxed, the deep parts of your mind, where the real horsepower is, can bubble up thoughts and put things together that your conscious mind simply can't.

This phenomena was described to me recently as operating with the power in the frontal cortex (the default conscious section) is roughly equivalent to using the change in your pocket. When you are in "the zone" and your deeper mind is able to work it is more like the national product of the United States of America. That's one heck of a difference.

If you never play with your inner child they atrophy. Or potentially worse, get bored and go off in random directions. Kind of like any muscle in your body when you don't use it your mind gets tired too and you can't be as readily be creative... plus personally I think it just makes you prematurely old. If you ever want a dose of reality on your brain aging check out the Brain Age games on the Nintendo DS.

The moral of this post... play every day.

YAY Input Innovation

There few things in technology that really get me hopping excited any more but every now and then something pops into the tech stream that makes me happy. Today is a happy moment for me.

Input devices and Human Computer Interface design though not something that is part of my core job or direct areas of research has always been one of those items that I have had a lot of interest in. I blame a particularly engaging professor I had in college that did research in this area that helped open my eyes to how silly things are today.

For example, we still use the QWERTY keyboard layout, a model that was built to slow down typing... yes really. On typewriters it was built so that the hammers would hang up less often because commonly used letters were farther apart and people would be slower thus causing less hits. If you are old enough or lucky enough to have had the experience of a typewriter that hung you will understand why this was a good idea at the time. Now critical mass is there and the world is unlikely to change. Just try and tell a Gen-Xer that can type 100+ words per minute on a QWERTY that you are going to change the layout. Of course now Gen-Y is used to a phone and small device without the need for a keyboard at all.

Apple has filed for a patent on a new generation of input device that is focused on touch, write and multi-touch. For more info check out the Apple Insider post for great detail from the patents themselves. One of my favorite features/ideas from the iPhone has always been the multi-touch. In fact part of the reason I expect to be getting an Apple laptop now is that they are adding that ability to those as well.

While we still have not achieved a DNI (Direct Neural Interface) jack yet so all we have to do is think it's only a matter of time. By the way... that really is happening. Controlling your computer with the power of thought is currently focused on assistance for the physically disabled. You don't need to be a dreamer though to realize that the downstream implications are simply fascinating.

Microsoft bid to buy Yahoo

It has been in the rumor mill for a while now that Microsoft was seeking to buy Yahoo. When there is sufficient smoke there seems to be fire and now the deal is officially proceeding. There are reasons for it and reasons against it. Apparently Live has not pulled in the traction that Microsoft wants and the desire for ad revenue and internet traffic control as a business model is

Google brings up some interesting points on the combination having more webmail and portal traffic than anyone else and Microsofts track record of following the path of Embrace, Extend, Exterminate when it comes to competition. But that input from a competitor is unlikely to bring about any sort of block of the deal. In fact some reports are even suggesting that the deal could be good for competition.

I am not 100% decided on the deal and it's positive or negative effects. Microsoft has a long record of proven execution and when they really set their eyes on something they can make it happen. There is a wide range of incredibly bright people there that when unified to a cause can deliver. The big change that I see though is in release management. Microsoft has a history of classical software development with releases taking years and subsequent patches taking months. In the internet world, and Google is a great example of this, change is constant, the ability to roll out new versions and try new approaches every week is key to success. This will be a big change from the default Microsoft dev approach. Will this Agile development change cross over into Microsoft's standard products? Another one of those much discussed items, SAAS Software As A Service suddenly becomes a lot more of a potential reality. To me this is the real reason for the desired purchase. Ads and Eyeballs are great to monetize and transform what Microsoft has done for a long time.

No matter what finally happens, this aught to be fun to watch.

You have to pay attention to the periphery

I have long been interested in how the brain works and why we do some of the bizarre things that we do. Add my interest in innovation and viola there is a fun intersection.

Individuals as well as companies have problems spotting, let alone executing innovations. In many cases the innovations seem obvious in retrospect causing people to think "well why didn't ABC co do that? They must be idiots." The truth is that spotting something outside of our current focus is not anything that we are wired to do. In fact we are wired to do the opposite.

For a few cool examples of this check out these YouTube videos. (Normally I would do an embed for your convenience but these are not available for some reason so go check them out.)

• Change Blindness; you cannot be aware of everything
• Mind's Eye; the Brain relies constantly on Visual Memories

The defense our mind puts in place that keeps us focused and able to complete a task without all of the information available overwhelming our brain is the one that keeps us from noticing things on the periphery. But if the periphery is where the interesting things happen what do we do?

The first step is to be aware of what you are focusing on. The next is to allow yourself time to wander, or in many cases allot yourself time to wander. This doesn't mean daydreaming (remember, vision without execution is daydreaming) but it does mean looking at things outside of the immediate and allotting effort and brain cycles to it occasionally. Probably not at the same rate as your real focus but without carving out at least some you quickly find that your immediate focus is all you have time for.

Ignorance more frequently begets confidence than does knowledge - Darwin

Have you ever watched on tv or listened on the radio to one of the spots on Stupid Criminals? The premise is those criminals that get caught because the things they do are just plain dumb. For example, here in Texas a man convicted of robbery worked out a deal to pay $9600 in damages rather than serve a prison sentence. For payment, he provided the court a check... a forged check. He got 10 years. If you have time... do an internet search... and laugh.

A research paper pubished in 1999 is very relevant to this topic and I thought I would share. The postulation of the paper is that in cases where one is ignorant, that same ignorance keeps you from realizing your ignorance. The graphs in the paper are very telling regarding people's regard of their own abilities. Even after tests were administered people still felt they were in the quadrants that they were not.

So... why is this relevant in a technically focused blog? I bring this up because if you consider the rate at which technology moves it requires a near constant diligence and research to stay current. So far Moore's law that processor speeds will double and halve in cost every 24 months has tracked fairly close. Software architectures seem to have an average lifespan of 2-4 years. As processors get quicker more and more levels of abstraction are able to be added and new algorithms are enabled. Bill Gates has often been attributed with saying "640k aught to be enough for anybody"? (He denies having actually said that BTW) Now we have services that handle thousands of transactions per second with message payloads of well over 10x that.

The meaning in all of this is that work in technology needs to be one of constant improvement. Both in ourselves as well as our systems. We need to be constantly learning, innovating and expanding. We can't let ourselves individually or as a company become complacent and be ignorant of the fact that we are ignorant. We need to be blissful because we love what we are doing and are growing as part of something larger than ourselves... not because ignorance is bliss.

Assume that your skills need to be refreshed every two years... and act accordingly.

Wide Area Network --> Home Network --> Personal Network

Everything keeps getting smaller and smaller. A few years ago people wrote off the "crazy people" who started to say that eventually everything would have an IP Address. Well now it seems like EVERYTHING has an IP and certainly everything hits a network somewhere. The internet may not quite be ubiquitous but things get closer and closer every year.

While a personal network used to be the network that you ran at home. Originally for the geeks among us (yes, I have had a network in my house for YEARS), then wireless (as before, first with manual configurations but now available through every retailer under the sun with easy push button setup). Well now it seems like technologies such as Bluetooth and other personal networking technologies have people running their own networks in human form without even thinking about it.

Bluetooth on phones and laptops has been around now long enough that even my mom (well, ok maybe not my mom as she chooses to believe cell phones are evil and voice mail even worse but most moms) can turn on a bluetooth headset and wirelessly interconnect. As other devices hit the market (Watch Gizmodo for a few weeks and be amazed) these personal networks will get larger and more populated. Especially as Apple gets more and more into the general consumer personal device business (this upcoming week's Macworld will be great to watch for just this reason, last year brought us the iPhone... who knows about this year) these things will get even more ubiquitous.

What a great time to be a techie.

Are you an Egghead?

Having something of a sick interest in the meaning of words, origin of words and their use had me wondering this morning where the word Egghead came from and what the history of it's use would be.

Turning to the Internet source of all random knowledge: Wikipedia, I asked. Egghead has origins starting in the 1950s and is a slang anti-intellectual epithet directed at people considered to be out of touch with reality due to their own intellectual interests. What fun.

It was even used by Richard Nixon in the 1952 presidential campaign to describe his opponent. So it also has a political connotation to it. No real originating source was quoted. The article also mentions a potential start of the description of an Egg shaped head referring to a Bald head. (something near and dear to my heart) Also no mention of Egghead software who changed to newegg some time ago though apparently now egghead.com goes off to Amazon. Not that this is completely relevant but it's my random chain of thought.

Ain't the Internet cool?

Do you sleep work?

Have you ever gone to bed wondering how you would solve a difficult problem and dreamt up the answer? Apparently you are not alone. A recent survey found that 51% of those surveyed dreamed about work and nearly 70% of those put those dreams into action.

I have always slept with a notebook and pen on my night stand so that I could write down what I thought up while not sleeping or dreamed about. It is interesting to me how many people apparently do the same.

This is of even more interest when you think about it in terms of the "work 40 hours" credo in Extreme Programming. (Really more of a manage your spikes in hours and avoid death marches but I digress). The core of the reason behind that is if you are well rested and fresh you do your best thinking and are best equipped to handle what gets thrown at you.

So next time you have a tricky problem to work out, think about it before bed, write it down as a checklist to complete (leave it on the paper so you can actually sleep) then dream up an answer. (Remember you are still limited by the fact that you won't sprout wings extra heads or anything along those lines so it is likely best to discard those dreams... in fact probably best not to talk about them either.)