The Hangout and the Bazaar

I have been more active of late in both the Twittersphere and Facebook universes and have a few observations to share. 

Facebook is cool and is a great place to connect with friends. New and old friends all in one spot. The feel is more like a social club or hangout than an internet site. This means that people say things (as many have pointed out) that they might not say in front of their boss, mom, or who knows who else. This has led to the two-face-book approach where people have their friends profile and their work profile separate. This way when you comment on the silly photo of your friend who has had too much to drink it doesn't show up on your bosses page and so on. After all, Facebook is now for old fogies. (I suppose that's me as over the course of a few weeks I had numerous friends ping and prod me to get online as things like school reunions become the talk of the day.)  It has the feel of being a semi-private club where what you say is sort of private though you know that it is subject to repeat. 

Twitter on the other hand is like a big bazaar on the internet. Everything is there and nothing is really private. There is an ability to keep your updates to only people you approve but it is not regularly done. There is a social etiquette followed regarding followers and following, numbers and post types, frequency etc. This gives a basic structure though there is the constant pressure of those attempting to use things for their own marketing (and annoying) messages. Just search on something current like #Startrek (Hashing is a way to mark tweets for easy search and trending)(The new Star Trek is a fabulous movie by the way, both Campy and High Tech and Throw Back and Modern all at once. GO SEE IT.)  

So I am using both. For different reasons. I love connecting with old friends and reconnecting with people I have lost track of on Facebook. I also love the raw feed feel of twitter though. It feels like the early days of Bulletin Boards when you could chat all day with people you didn't know connected only by your interest in computers and ability to type. 

I watched an interesting Ted Talk that got me thinking along a couple of avenues. Margaret Wertheim presented The beautiful math that links coral, crochet and hyperbolic space. Go ahead and watch if you have 16ish minutes. I'll wait. 

I have mused before that we know less about the universe than we like to think. For simple examples look at things like Dark Matter and Dark Energy... wonderful hypothetical constructs that we don't really understand that make up only 80ish percent of the universe... 

When you look at things like science fiction and thin

gs like space travel and teleportation in particular a good amount of suspension of disbelief is usually required. Well, when you look at some of the items that actually occur in nature maybe not. 

Margaret does a great overview of Euclidean Space (Planar), Spherical Space and finally Hyperbolic Space. For those that are not visual or auditory learners though here is a quick overview of getting from destination A to destination B in each space. 

• Euclidean Space (discovered by Euclid (go figure)) is essentially planar space. There is a whole lot of math involved in a full explanation but essentially it is the rule that given a strait line and a point the only line that can be drawn through the point and not go through the line is parallel to the original line and there is only ONE of those.
  
• Spherical Space is a similar math run but can be simplified to say that when you have a line drawn in spherical space and you have a point there are ZERO ways to draw a line that doesn't intersect from that point and the line.
• Hyperbolic space has even more math and if you go back just a little bit in time much of that math is proving that it is impossible. However, nobody ever told choral that so it uses it all the time. Following the line and point theory again this basically is the statement that given a line and a point there is an infinite number of lines that can cross the point without crossing the original. Margaret does a great tactile example of this with a demonstration of a simple crochet piece.

So, that was a lot of words and explanation to say we see this type of space in multiple ways. It is easy to jump to an explanation of "multi-dimensional" space being the answer but think about the web. You can get anywhere from anywhere with one simple button push. There may be a mess of routers and switches that do math in between but to you using your browser it's click and away nearly instantly. As the math for this type of space matures and we understand it better things like "folding space" and traversing large "distances" instantly may become a reality.

What fun to think about. All from something as simple as a piece of choral made out of crochet. Next time you watch grandma spin that yarn just think, she could be paving the way to space travel.

More cloud updates

Since my last post I have been sent a few and read a few other articles that I thought might be worthy of note related to what clouds are and why it is important.

Sun's CTO Greg Papadopoulos did an interview with Informationweek and discussed some of the efforts that SUN has going on. One of the biggest items of note in that article is the new Drizzle product coming out of the MySQL work. There are many articles and blog posts on the topic as well but the gist is that it is an effort to Cloud-smarten MySQL. Seeing how "MySQL gets used, people only use a subset of the relational capabilities because it has a horizontal scale and there are all these concerns that go into large-scale deployments. So Drizzle sort of strips back down to a small core and then builds up the distributed capabilities." I have made no secret of my feeling that Data is no longer relational. So I think this is encouraging news though as seems to be the case lately with SUN it may be too late for MySQL.

(Aside... Greg Papadopoulos and David Douglas recently co-authored Citizen Engineer: A handbook for socially responsible engineering. Though it is not yet available you can check out an interview with David Douglas on the book and engineering at the Mercury News. Personally I find this to be a refreshing take on the responsibility of Engineering as a profession and all of the personal and corporate responsibility that should go along with it.)

As if they noticed Microsoft pushing cloud tech Amazon announced a few new items in their own cloud initiatives. The ability to reserve capacity for known needs. They also expanded the ability of Windows based services in another Amazon zone. All of this from a company that sells stuff, proof that necessity is the mother of invention.

Even Microsoft realizes it’s all about the cloud. (though opinions still differ on what a cloud exactly is… I personally like the blind men describing the elephant/cloud analogy)

Wired did an interview with Ray Ozzie - Long article but worth reading even if you are an ABM coalition member. (I have always thought highly of Ray Ozzie, even if I did hate Notes itself as an administrator and user, it was visionary in its setup and ahead of its time. Heck… Ray even saw P2P coming before the rest of the industry started to catch up.)

Further proof of the gap between the over-served and the under-served. The question is if Microsoft is late enough to the cloud innovation game that their delay will actually cause them harm in market share. They have rallied before and come in with overwhelming response. (e.g. I.E.) The truth is the services such as PPT, DOC etc is so ubiquitous in the work environment it will be difficult to displace them with anything else. The question though becomes more of what will happen to those types of documents and will they be service disassembled themselves enabling other innovation plugs to go in. Also, who knows maybe Microsoft will have the bones to push further change into media distribution and synch. Though Apple has a pretty strong hold and the media providers can’t seem to understand that technology can help versus hurt their business (self destroying DVRs anyone? Eventually a multi-billion dollar DVD business… sigh)

What a great time to be a technologist.

Data is no longer relational

Ok, so the headline may be a little of an over statement for effect. Perhaps I could have said that data that people are interested in is no longer relational but then it wouldn't have been nearly so pithy.

With much respect to Edgar Codd and his invention of the relational model for database storage I think it is time to move forward. Relational databases are great for things like financial models, personnel data and other Enterprise Systems as well as many other standard, repetitive data. It gave a solid reference point to learn data structures and modeling to several generations of budding Computer Scientists.

When I say it is time to move forward I am not saying we should immediately move all data systems to Object Oriented Databases or otherwise induce data chaos. What I do want to push is the idea of unstructured data. Computers are great with rules, with structure and with fundamentally binary relationships. Algorithms are starting to mature around unstructured data (for an example go search google.) but it is still not widespread or well understood.

New exciting algorithms such as Amazon's Dynamo (Werner Vogels is one of my favorite speakers and bloggers on distributed tech, if you are not familiar with him in this space you should be) database are showing in real world situations that distributed systems and distributed data are a reality.

In a lot of system designs because people are so familiar with relational data structures and systems we find Object Models that look like a relational database design. When asked why it looks like this the answers are fairly consistently things like "this is how the database stores it, for speed we need to do the same" or "it just made sense when we pulled the DBA in to help us with the model."

Objects are not relational! They are objects. Then when you get into full structures of objects or object trees there are relationships but it is not the same as a relational database. Especially as we start to build and mature distributed system algorithms it doesn't make sense to use a centralized data store. If the data can be broken up, distributed and stored with the algorithms that will use it performance will improve.

In fact I would argue that the elusive SLA of a system response can begin to be discussed if we can tie the data to the processing. Granted there are new complexities in this model for synchronization, segmentation and consistency but there are ways to solve them. Similarly consistent access to the same servers is also possible.

What other great examples of distributed computing and distributed data storage have you seen?

YAY Input Innovation

There few things in technology that really get me hopping excited any more but every now and then something pops into the tech stream that makes me happy. Today is a happy moment for me.

Input devices and Human Computer Interface design though not something that is part of my core job or direct areas of research has always been one of those items that I have had a lot of interest in. I blame a particularly engaging professor I had in college that did research in this area that helped open my eyes to how silly things are today.

For example, we still use the QWERTY keyboard layout, a model that was built to slow down typing... yes really. On typewriters it was built so that the hammers would hang up less often because commonly used letters were farther apart and people would be slower thus causing less hits. If you are old enough or lucky enough to have had the experience of a typewriter that hung you will understand why this was a good idea at the time. Now critical mass is there and the world is unlikely to change. Just try and tell a Gen-Xer that can type 100+ words per minute on a QWERTY that you are going to change the layout. Of course now Gen-Y is used to a phone and small device without the need for a keyboard at all.

Apple has filed for a patent on a new generation of input device that is focused on touch, write and multi-touch. For more info check out the Apple Insider post for great detail from the patents themselves. One of my favorite features/ideas from the iPhone has always been the multi-touch. In fact part of the reason I expect to be getting an Apple laptop now is that they are adding that ability to those as well.

While we still have not achieved a DNI (Direct Neural Interface) jack yet so all we have to do is think it's only a matter of time. By the way... that really is happening. Controlling your computer with the power of thought is currently focused on assistance for the physically disabled. You don't need to be a dreamer though to realize that the downstream implications are simply fascinating.

Microsoft bid to buy Yahoo

It has been in the rumor mill for a while now that Microsoft was seeking to buy Yahoo. When there is sufficient smoke there seems to be fire and now the deal is officially proceeding. There are reasons for it and reasons against it. Apparently Live has not pulled in the traction that Microsoft wants and the desire for ad revenue and internet traffic control as a business model is

Google brings up some interesting points on the combination having more webmail and portal traffic than anyone else and Microsofts track record of following the path of Embrace, Extend, Exterminate when it comes to competition. But that input from a competitor is unlikely to bring about any sort of block of the deal. In fact some reports are even suggesting that the deal could be good for competition.

I am not 100% decided on the deal and it's positive or negative effects. Microsoft has a long record of proven execution and when they really set their eyes on something they can make it happen. There is a wide range of incredibly bright people there that when unified to a cause can deliver. The big change that I see though is in release management. Microsoft has a history of classical software development with releases taking years and subsequent patches taking months. In the internet world, and Google is a great example of this, change is constant, the ability to roll out new versions and try new approaches every week is key to success. This will be a big change from the default Microsoft dev approach. Will this Agile development change cross over into Microsoft's standard products? Another one of those much discussed items, SAAS Software As A Service suddenly becomes a lot more of a potential reality. To me this is the real reason for the desired purchase. Ads and Eyeballs are great to monetize and transform what Microsoft has done for a long time.

No matter what finally happens, this aught to be fun to watch.

You have to pay attention to the periphery

I have long been interested in how the brain works and why we do some of the bizarre things that we do. Add my interest in innovation and viola there is a fun intersection.

Individuals as well as companies have problems spotting, let alone executing innovations. In many cases the innovations seem obvious in retrospect causing people to think "well why didn't ABC co do that? They must be idiots." The truth is that spotting something outside of our current focus is not anything that we are wired to do. In fact we are wired to do the opposite.

For a few cool examples of this check out these YouTube videos. (Normally I would do an embed for your convenience but these are not available for some reason so go check them out.)

• Change Blindness; you cannot be aware of everything
• Mind's Eye; the Brain relies constantly on Visual Memories

The defense our mind puts in place that keeps us focused and able to complete a task without all of the information available overwhelming our brain is the one that keeps us from noticing things on the periphery. But if the periphery is where the interesting things happen what do we do?

The first step is to be aware of what you are focusing on. The next is to allow yourself time to wander, or in many cases allot yourself time to wander. This doesn't mean daydreaming (remember, vision without execution is daydreaming) but it does mean looking at things outside of the immediate and allotting effort and brain cycles to it occasionally. Probably not at the same rate as your real focus but without carving out at least some you quickly find that your immediate focus is all you have time for.

Ignorance more frequently begets confidence than does knowledge - Darwin

Have you ever watched on tv or listened on the radio to one of the spots on Stupid Criminals? The premise is those criminals that get caught because the things they do are just plain dumb. For example, here in Texas a man convicted of robbery worked out a deal to pay $9600 in damages rather than serve a prison sentence. For payment, he provided the court a check... a forged check. He got 10 years. If you have time... do an internet search... and laugh.

A research paper pubished in 1999 is very relevant to this topic and I thought I would share. The postulation of the paper is that in cases where one is ignorant, that same ignorance keeps you from realizing your ignorance. The graphs in the paper are very telling regarding people's regard of their own abilities. Even after tests were administered people still felt they were in the quadrants that they were not.

So... why is this relevant in a technically focused blog? I bring this up because if you consider the rate at which technology moves it requires a near constant diligence and research to stay current. So far Moore's law that processor speeds will double and halve in cost every 24 months has tracked fairly close. Software architectures seem to have an average lifespan of 2-4 years. As processors get quicker more and more levels of abstraction are able to be added and new algorithms are enabled. Bill Gates has often been attributed with saying "640k aught to be enough for anybody"? (He denies having actually said that BTW) Now we have services that handle thousands of transactions per second with message payloads of well over 10x that.

The meaning in all of this is that work in technology needs to be one of constant improvement. Both in ourselves as well as our systems. We need to be constantly learning, innovating and expanding. We can't let ourselves individually or as a company become complacent and be ignorant of the fact that we are ignorant. We need to be blissful because we love what we are doing and are growing as part of something larger than ourselves... not because ignorance is bliss.

Assume that your skills need to be refreshed every two years... and act accordingly.

Wide Area Network --> Home Network --> Personal Network

Everything keeps getting smaller and smaller. A few years ago people wrote off the "crazy people" who started to say that eventually everything would have an IP Address. Well now it seems like EVERYTHING has an IP and certainly everything hits a network somewhere. The internet may not quite be ubiquitous but things get closer and closer every year.

While a personal network used to be the network that you ran at home. Originally for the geeks among us (yes, I have had a network in my house for YEARS), then wireless (as before, first with manual configurations but now available through every retailer under the sun with easy push button setup). Well now it seems like technologies such as Bluetooth and other personal networking technologies have people running their own networks in human form without even thinking about it.

Bluetooth on phones and laptops has been around now long enough that even my mom (well, ok maybe not my mom as she chooses to believe cell phones are evil and voice mail even worse but most moms) can turn on a bluetooth headset and wirelessly interconnect. As other devices hit the market (Watch Gizmodo for a few weeks and be amazed) these personal networks will get larger and more populated. Especially as Apple gets more and more into the general consumer personal device business (this upcoming week's Macworld will be great to watch for just this reason, last year brought us the iPhone... who knows about this year) these things will get even more ubiquitous.

What a great time to be a techie.

Are you an Egghead?

Having something of a sick interest in the meaning of words, origin of words and their use had me wondering this morning where the word Egghead came from and what the history of it's use would be.

Turning to the Internet source of all random knowledge: Wikipedia, I asked. Egghead has origins starting in the 1950s and is a slang anti-intellectual epithet directed at people considered to be out of touch with reality due to their own intellectual interests. What fun.

It was even used by Richard Nixon in the 1952 presidential campaign to describe his opponent. So it also has a political connotation to it. No real originating source was quoted. The article also mentions a potential start of the description of an Egg shaped head referring to a Bald head. (something near and dear to my heart) Also no mention of Egghead software who changed to newegg some time ago though apparently now egghead.com goes off to Amazon. Not that this is completely relevant but it's my random chain of thought.

Ain't the Internet cool?

Overtime is a productivity killer

This post may not be one of my most popular with PMs around the world but here it is anyway. Overtime, in the long run, does not help a project. When the numbers are run (and boy do productivity numbers get run...) it turns out that work that is done when mapped out with a scatter diagram across many different projects is essentially the same. [See Slack pg 64, it's cheap in paperback and well worth the read] The work that is done is the same per day... not per hour.

Intuitively this may initially seem strange if not outright wrong. How could a team that consistently works 50 or 60 hours a week end up with the same productivity as a team who works 40 hours each week? The answer lies in the long term effects of overtime, not the short term spike in productivity from a controlled burst. Controlled bursts really can be effective, as long as they are just that, controlled bursts.

Overtime itself is not an evil thing. Many are the stories in corporate lore of the team that pulled an all nighter or pushed through that special effort over the course of the last week to deliver everything on time. Good managers, [project managers, people managers, senior individual
contributors] know when to pull the trigger on overtime to push things over the goal line.

The problem comes because this can be addicting. Both to the manager and the individuals. People like being super stars, like the accolades and recognition of being really dedicated. Believing this to be a great success without evidence to the contrary managers become dependent upon the push to get everything done. Rather than pushing back on scope, push on the people. It is an easier road and you don't have to worry about an unhappy customer. At
least not right away...

With pressure comes a bit more focus. That's a good thing right? Sure, when that pressure leads to cutting out unnecessary steps and trimming requirements that are not needed. But when that pressure becomes an unending stress to deliver no matter what, it starts to become noise.
People stay, because they see others staying. People work, because others work. But the real urgency and productivity goes down. Knowing that they will be there working at night, things get put off, people talk in the hallway, time is spent surfing the web, etc. Why not, they will be there later anyway.

There are of course many other costs to too much overtime, but this is just a blog and not a book, so I will give it a rest now that I have you thinking.

The principles of Tao Teh Ching in Software Development

You may be asking if I drank a little too much Green Tea to be comparing Software development to the principles of Tao Teh Ching but please bear with me on this one. Honest, it makes sense. (At least I hope it will after I write it down and it's outside of the world inside my head) First, the basic principles. (there is a lot more in the book as its a compilation of a good deal of Chinese Philosophy)

That which remains, is easy to handle.

That which is not yet developed is easy to manage.

That which is weak is easy to control.

That which is still small is easy to direct.

Deal with little troubles before they become big.

Attend to little problems before they get out of hand.

For the largest tree was once a sprout, the tallest tower started with the first brick, and the longest journey started with the first step.

Just as these wise words say, things are easier to handle early on. Once they get rolling or get bigger though many times there is too much momentum to control it. Just like a small snowball at the top of a mountain can become a massive avalanche so too can a small problem ignored early in development become a costly, potentially project killing problem later in the development cycle.

Releaseing early and often during software development makes sense. By focusing our teams on the first things that really matter we can focus our efforts on the things that make a difference. By focusing on the things that we expect to be hard we can ensure we have enough time to manage the risk associated with them. Identifying things early allows us to adapt our plans and make the best set of decisions.

More disruptive tech.. here Wii go

I had another experience with Disruptive Technology this weekend that I thought I would share. First a quick intro story, my wife decided that I spend too much time writing blogs and doing other work at night so she gave me a Nintendo DS to play Pokemon with my son. (We were playing the card version but that is all old school and passe now.) The people at Nintendo are GENIUSES... if you purchase the new Pokemon game for DS you have two options, Diamond or Pearl. The variant between the two is the likelihood that you will run into any particular type of Pokemon. So in order to get the best mix of Pokemon you need to trade with someone else... this is where I come in. I get one type, my son gets the other... GENIUS... now we buy two games.

It get's better though, and this is where the disruptive tech comes in. If you have a Wii (and if you live under a rock and have not heard of this you need to get out more or read) then the new version of Pokemon Battle Revolution for the Wii also allows you to trade back and forth your Pokemon with the Wii version. What a Wii bit of fun. (sorry, I had to, they will kick me out of the blogger's union if I don't hit my pun quota)

This brings me to my point (already). The interface for the Wii is entirely different than the existing interfaces for video games. This has, of course, hit the press and blog pages long ago but I thought it made a great point. The Wii does not support true HD, it's tiny and has no huge sizzle effect like the XBox 360... it's even cheaper and simpler than the other boxes and yet, it is rocking the video game world. Because it changed the interface rules. The Nintendo folk's thought differently and instead of focusing on ubber graphics and interaction view(the standard thought pattern of the day in games) they changed the interaction method. With your Wii wireless controller (it's all Wii small... there I think that's quota) you control standing in the middle of your living room. It's easy enough my 4 year old can play bowling and baseball, fun to watch in and of itself.

What can we learn from this? Don't be afraid to go another direction. Wii is having amazing success and corresponding sales (if you doubt this go try to find one in a store). This causes me to think about basic interfaces and the way our users interact with systems. Not just web versus green screen but form versus map and open search versus controlled UI. This is the gist of Web 2.0 and why the interfaces work. Kids who are growing up now don't think in forms, why would they? Their interfaces are things like iPhones and Wii.

Disruptive Technologies that make you go hmmmm

Two separate events in the past few days have made me consider disruptive technology and it's impacts. While fun for those standing on the sidelines it has to also be a terrifying event for those who are in the middle of what is being disrupted.

• Disruptive Tech 1: The iPhone
• Disruptive Tech 2: Google Apps

Disruptive Tech 1: The iPhone. Yes, the often referred to Apple foray into the cell phone business (in some cases referred to as the "Jesus Phone" it's PR is so big) is a great example of a disruptive technology. It is already wrecking havoc with phone manufacturers and not because it is the worlds best phone, but it is slightly off from today's phones. The NY Times put out a great article late Tuesday night as the phone was hitting the general population with the compelling headline "The iPhone matches most of it's hype". Gizmodo posted a good summary of "Finally Confirmed: What the iPhone doesn't have" [Songs as Ringtones • Games • Any flash support • Instant Messaging • Picture messages (MMS) • Video recording • Voice recognition or voice dialing • Wireless Bluetooth Stereo Streaming (A2DP) • One-size-fits-all headset jack (May have to buy an adapter for certain headphones) • 3G (EV-DO/HSDPA) • GPS • A real keyboard • Removable battery • Expandable Storage • Direct iTunes Music Store Access (Over Wi-Fi or EDGE)]

That's quite a list for something that is still going to change the phone world. It has a controlled UI and OS. You can't load things on it and Jobs has made no bones about confirming that will remain the case. But it has a paradigm changing interface, consolidates devices for those with iPods and Phones and PDAs, even smart phones. As eloquently stated by David Pogue in the times article "It’s substance; it’s style. It does things no phone has ever done before; it lacks features found even on the most basic phones."

Personally I am going to hold off getting one. I recently purchased an 8525 (YAY I get to skip my Blackberry across a lake) and I have been quite happy with it. (heck I have already customized the snot out of it and have my Yodeling cow ringer going) But the geek in me covets parts of the iPhone and I may end up with one when I next need a phone... when it uses the faster Edge network and apps to get to business email are actually available. In any case, the reactions from the "old school" cell phone makers is going to be a blast to watch.

Disruptive Tech 2: Google Applications. I have to admit that when Google first rolled these out I thought to myself "ok... that's nice... why do people even try to beat Microsoft in this space?" Well a quiet little post on the Google blog made a light bulb go off for me. Google isn't trying to hit Microsoft at the desktop (well not right away) they are hitting them squarely in the admin.In the blog post Google indicates how they are going to give free access to advanced features of the full Google office to schools. The standard package for individuals is free already but a Premier Edition, targeted at companies is also available for $50/user. Now the Education Edition also includes the ability to integrate with existing facilities, conference and resource scheduling and other premium services.

Educators want to educate. They don't want to run systems for email etc etc. Now, they can get email, integrated chat and other niceties for free. This is where giving it away starts to make sense. It's not just about giving it away to the individual consumer in hopes of pushing ads. It's about pushing it into corporations... wow, now that has to be a scary thought. As with the iPhone, it may not truly change the world, but the potential is there and it will be fun to watch. (By the way, although corporate email isn't available via the iPhone... if you use gmail you are good to go... hmmmm)

Every integration point created WILL fail and Scalability

I have two things for today's post.

1. I received a note this AM from my "FYI- Jana RSS feed" (A good friend that works with me that sends me email) that I had to pass on to everyone. Read the article because the quote below is SOOOOOOO true.

“One of my recurring themes in "Release It"
is that every call to another system, without exception, will someday try to
kill your application. It usually comes from behavior outside the specification.
When that happens, you must be able to peel back the layers of abstraction, tear
apart the constructed fictions of "concurrent users", "sessions", and even
"connections", and get at what's really happening…”

Agile, Architecture and
the 5am Production Problem - by Michael
Nygard

2. This weekend I attended the Seattle Conference on Scalability. It was sponsored by Google and it was GREAT. Over the next few weeks I will digest what I heard and saw and report it back via blog. At the moment though I have to admit my dueling day jobs are getting the better of me so I haven't had time to couch my ideas into pithy post prose. I promise it is coming though. I will however let you know that Google recorded the presentations and they should be available by the end of the week on YouTube.

Enterprise Architecture is not technology selection

Now that I have extolled the virtue of standards like muscle building and listed our current set I wanted to post a bit on why I believe having a consistent Enterprise Architecture is important.

First and foremost on the list of why an Enterprise Architecture is important is that for a company to be successful it needs must build IT Solutions, not IT Capabilities. What is the difference you ask... (or even if you didn't I am going to tell you anyway) Capabilities can survive in non-integrated verticals but Solutions require integration. An End to End view of paths and functions is required to get a solution. An IT Capability is something like storing a customer information record in a database. A Solution provides the customer with value because it can use that information and to customize a trip.

One of the fundamental rules of development is breaking things up into smaller, solvable pieces. At the individual system level this makes perfect sense. However, when you start to look at things at the enterprise level, things implemented in pieces build piecemeal solutions. Then, these piecemeal solutions need to be integrated. This is what I have referred to in previous conversations as the fiefdom effect. Essentially, this is where individual systems or areas have their own fiefdoms and protect their system boundaries and "control their own destiny". While this may be fine a micro level, the independent pieces work fine after all, at the macro level this increases the integration burden and decrease overall company agility.

This is where we get to what Enterprise Architecture is all about... Enterprise Architecture's (and Engineering, YAY Engineering!!) job is to take a long term view so that teams can build capabilities not just fulfill the short term order of the day. If we have a solid base upon which to build (and this is an art in itself, not building the Taj Mahal or better yet the Winchester House when an outhouse will do the job) we enable everyone to move much faster and make integration between components that much simpler.

Yet another argument for a seamless SOA infrastructure. Or at least that's what the vendors are touting as the latest Technology Software solution for this problem... but we will save that post for another day.

Monkeys down a fire pole

How many monkeys can slide down a fire pole in 5 minutes?

Ok, so this question doesn't seem to have a lot to do with technology or engineering when you fist consider it. But give me a little bit of patience and let's see where we go.

I have recently begun to undertake a personal quest for education and education provision to others about threading and how to do it well and efficiently. With the introduction of multi-core processors and processor companies no longer creating faster and faster single core CPUs with clocking, over-clocking and over-over-clocking, it has become more important to actually understand threads.

Even as recently as when Java first came out the only places that you really ran into threads in open systems was in Solaris (or other Unix) systems with big multi-CPU backplanes. These were usually reserved for big 'ol database boxes though so you could optimize your system to really what amounted to one big fire pole. If we imagine each processor as a fire pole and each unit of work to run as a monkey, we can start to use my fun little analogy.

Everything you did was about queuing up the monkeys efficiently then sending them down the pole as quickly as possible for each individual Monkey thereby maximizing your monkey throughput. Great schemes for keeping the Monkeys in the right order, keeping the pole clean during non-monkey sliding moments (Garbage collection in Java-land) and other similar optimizations were the wave of the day.

Now the fire poll manufacturers (Intel and AMD) discovered that they just couldn't build a faster fire pole without violating the laws of physics (and it's not that they wouldn't break those laws, more that they couldn't figure out how) and therefore decided the best way to move forward was to build a multi-fire pole firehouse in the same amount of space. It was a great innovation... but the monkeys were still optimized for a single fire pole. So in many cases (not all, but many) the fire pole (CPU) utilizations actually was very low even though other areas were slammed. Monkey lines (memory) and even Monkey storage (I/O) became bottlenecks with a low 25% pole usage.

The time has come to build education on efficient fire pole usage. We need more effective ways to slide those little monkeys down. There are some basic practices that seem to make sense. Sun recently explained how they are building better threading into Java and things are rolling out from there. That's a great start... but that is just a start. To really use the power we are now putting into machines we need to figure out better ways to feed the cores.

Help me save the monkeys... do you have any good tips, observations, sites or do you have any bad monkey practices that you have seen that should be avoided? This practice is, of course, not limited to Java but C, C++, Ruby and others. I would like to pull these together to be shared with everyone. If you have a good monkey tail please drop a comment. I would love to hear from you.

Build your Corporate Muscle Memory

I have posted before about the need for standards to allow us to focus on differentiating functionality and I have been thinking about this again since I have been working on a project in which standards are a big part.

Have you ever considered how as you learn something it become easier with practice, eventually become second nature? Watch a small child learn to walk and it is something that they have to really focus on at first. Then as we get older it is something that we simply take for granted. Does this make us expert walkers? Or do we just establish walking as a "dial tone" activity that we no longer need to think about?

Think about a chef. When they are first learning to do things they need to focus on every cut of a vegetable. But as they become an expert they no longer even measure a teaspoonful of salt. (BAM! Just watch an episode of Emeril Live and see how closely he holds to a written down recipe for an example.) A chef can do things without any concentration at all that many of us would require many times the time to do.

A runner or athlete is another good example. Runners develop muscles that work in very distinct ways. Their develop something called muscle memory that causes the muscles to introduce an actual change to how the muscles work. This change increases the level of accuracy and flexibility for running by optimizing the muscles for the repeated action. There is a trade off however, the muscles become very good at a certain set of items and less flexible for others. Because a runner is training to be a runner, not a gymnast this is a fine trade off.

Standards are similar in that we want to establish a Corporate Muscle Memory. It will require focus and concentration at first to do the new standard processes. It will also reduce flexibility to a certain extent in that there will be specific micro-processes that could become harder but the specialties become faster. But with the goal being to move as quickly as possible with agility towards the companies direct goals this is a trade off worth making.

Focusing our efforts on learning and optimizing according to standards allows is to build our Corporate Muscle Memory. We can then go faster and focus on differentiating functionality rather than dial tone. Just as with human ability companies ability needs to evolve over time and standards are no different. When the time comes to change a standard there will be a period of concentration required for us to develop that new Muscle Memory and for it to become second nature. But as we do whole new levels of speed and capability are possible.

Developers are Artists

In a meeting a little while ago I heard a comment, at the time said in a little bit of a flip way, that developers are artists. While not necessarily meant in a way that would suggest that Developers are right brain oriented, creative, canvas painting snobs the statement did get me thinking.

We were discussing standards. A topic that can be scary for people who want to forge their own way... artistically creating the next generation of award winning software... etc. (blah blah blah)

So off my brain went on a metaphor dive and it actually started to make sense to me. While artists want to be able to choose their own tools to ensure they get the right tools for the job there is a lot to be said for simplicity. Having the right brush size is critical... but it's still a brush. Certainly, there is a difference between the 10 for a dollar brush and the $20 brush but if you compare a $19 and a $22... is that really something an artist wants to waste their time doing? I mean really... there are people who are paid to do that... shouldn't the artist be creating... painting... utilizing their skills to make a difference.

Our approach to standards needs to be the same way. We will have people who specialize in brushes. Or in this case Database Platforms, Application Server platforms, Middleware solutions etc. We have a set of brushes sized appropriately for the job. Or in this case standard containers such as J2SE and J2EE containers both basic and fully enterprise capable... Tomcat, JBoss, JBoss with Clustering, etc. We want our artists to focus on painting the canvas with differentiating impactful imagery. Or in this case business logic that implements the vision the company has for functionality.

I realize some of you may be wondering just how deep my analogy rabbit hole is going to go but I will make a few more points and abuse it just a little more and then let you get on with your life. Just like brushes need to be replaced and a new canvas is occasionally needed so to do application and development standards need to be refreshed. Standards are just a representation of the best knowledge at a point in time. They are meant to be challenged, updated, and tuned. Without that effort and focus they become dated and slow things down rather than speed them up.

So as we set and roll out new standards as part of our efforts to accelerate development and increase stability and quality keep that in mind. Let's make it so that our artists can paint... but let's also challenge the status quo.