Searching for search and finding Godot

The battle for search eyeballs is heating up again. The model has been shown to work and Google is now Ad Mogul extraordinaire. Microsoft in classic fashion has tried, learned and is trying again. Yahoo has had much publicized struggles. Even new players are coming into the space. The buzz about Wolfram Alpha is everywhere.

What this next level of competition is about is relevance of information. The reason Google initially made inroads with search was their optimization routines and ability to give people search results that were relevant to what they wanted. Recent additions such as the ability to promote and bury links based on your own preference are continued iterations of this optimization. Similarly iGoogle and personalized history of search with functions like returning the link you clicked on last time you searched for something are again continued iterations.

The next generation of search is taking more of that sea of information that is out there and turning it into something useful. This is both good and bad. It's good/great in terms of making things digestible and understandable, especially when you are researching a particular topic. But it is less so when what is important is being determined for you and you loose some of the outliers. Outliers are where innovation happens. This is not to say that results don't get buried with a raw information return because they certainly do but just as with code and layers of abstraction, with great power comes great responsibility and the need to know what is going on behind the scenes. In many cases this behind the scenes is proprietary information and algorithms that companies (rightly) don't want to share as their secret sauce. What a dilemma.

Wolfram Alpha is the new kid on the block and have gotten a lot of Internet love with an effective launch that you could watch in video live. It's cool for checking out what happened on the day you were born, doing research on specific items and generally getting a quick easy view of relevant data to concrete topics.

Microsoft Live didn't really live up to what was desired by Microsoft but they are back and ready for round two with their new search engine Bing. (I thought that was the sound the Southwest ads used for new deals?) Bing is in pre-launch and is supposed to do a lot of what Wolfram does and use other proprietary correlation tools to give better and deeper results. ComputerWorld has a nice Visual Tour of the functionality to give a good idea of what is coming.

That leaves Google. Google's new exciting new news is the pre-pumping of Google Wave. If buzz on the Internet is any indicator there is a lot of excitement and potential here as well. If it wasn't clear why there was rumored interest in Twitter from Google it is now. Wave is intended to be a merge of... well... everything Just In Time.

So, it looks like the observation that the next generation of systems are about visualization and the presentation of information, not just finding it. Boy won't this be fun to watch.

The Hangout and the Bazaar

I have been more active of late in both the Twittersphere and Facebook universes and have a few observations to share. 

Facebook is cool and is a great place to connect with friends. New and old friends all in one spot. The feel is more like a social club or hangout than an internet site. This means that people say things (as many have pointed out) that they might not say in front of their boss, mom, or who knows who else. This has led to the two-face-book approach where people have their friends profile and their work profile separate. This way when you comment on the silly photo of your friend who has had too much to drink it doesn't show up on your bosses page and so on. After all, Facebook is now for old fogies. (I suppose that's me as over the course of a few weeks I had numerous friends ping and prod me to get online as things like school reunions become the talk of the day.)  It has the feel of being a semi-private club where what you say is sort of private though you know that it is subject to repeat. 

Twitter on the other hand is like a big bazaar on the internet. Everything is there and nothing is really private. There is an ability to keep your updates to only people you approve but it is not regularly done. There is a social etiquette followed regarding followers and following, numbers and post types, frequency etc. This gives a basic structure though there is the constant pressure of those attempting to use things for their own marketing (and annoying) messages. Just search on something current like #Startrek (Hashing is a way to mark tweets for easy search and trending)(The new Star Trek is a fabulous movie by the way, both Campy and High Tech and Throw Back and Modern all at once. GO SEE IT.)  

So I am using both. For different reasons. I love connecting with old friends and reconnecting with people I have lost track of on Facebook. I also love the raw feed feel of twitter though. It feels like the early days of Bulletin Boards when you could chat all day with people you didn't know connected only by your interest in computers and ability to type. 

Making sense of nonstructured nonsense - Trend Spotting in the Over-Information Age

As we look out over the web and remember that all of this information at our finger tips didn't used to be there it's a little hard to believe. The current Generation Y has a reason to be asking as this information has always been there for them. They are used to being perpetually connected and living their lives online. Tools like Twitter and Facebook make connectivity via digital methods a default. Connectivity from access devices such as iPhones or something as strait-forward as SMS provide snippet updates and easy push of information. Even Microsoft is getting into the game with their recent Vine announcement. All the while "normal" information continues flowing out. Papers are published and go to the web, news releases feed directly out, blog posts go out every second. Lots and lots of unstructured data every single day.

This brings about the challenge of the next generation of systems. Simple links to information without context aren't necessarily useful any more. When the search engines first came out the ability to point you in the right direction was enough. It no longer is. If a search engine returns a particularly pithy 140 character tweet on my topic I probably won't be thrilled. If it returns a mesh of links based on that tweet connecting with others, blogs and research topics in a neat visualized package... well that would be cool. 

For example, I am obsessively (those that know me probably are nodding their heads right now... I would ask you to stop please) interested in ants, papers on ants and recent research about ants. I believe that how they work, organize and where collective intelligence emerges provides us with many great clues on building scalable systems out of unstable parts. But in order to keep up on this I need to keep running searches and queries against various engines, look to known research outlets, receive email pointers from friends and build my own views and filters to what I think is worthwhile.  

Another example, twitter and other outlets can give indicators of where traffic problems are. People get stuck in traffic and tweet to their friends they are stuck. What if that information were automatically gathered, from the problem areas themselves, with geocoded info and placed on a google map mashup. Talk about useful. (I will of course conveniently forget to mention the downsides of texting, tweeting, facebooking and driving. Don't do it.)

Visibility to all of this disorganized and free flow data is the next great mountain to climb. Terms such as Crowdsourcing, Trend Spotting, Data Visualization are the buzzwords of the day. For a quick and inspiring note on what you can do with this type of information check out Erik Hersman's Ted Talk.  Knowing what is getting buzzed on Twitter and where the collective consciousness is headed is good and in many cases ahead of "standard" news outlets. How to create tools to view this information in a meaningful way is the challenge we need designers and engineers to be working on. 

In the meantime if you see good ant stuff let me know. 

I watched an interesting Ted Talk that got me thinking along a couple of avenues. Margaret Wertheim presented The beautiful math that links coral, crochet and hyperbolic space. Go ahead and watch if you have 16ish minutes. I'll wait. 

I have mused before that we know less about the universe than we like to think. For simple examples look at things like Dark Matter and Dark Energy... wonderful hypothetical constructs that we don't really understand that make up only 80ish percent of the universe... 

When you look at things like science fiction and thin

gs like space travel and teleportation in particular a good amount of suspension of disbelief is usually required. Well, when you look at some of the items that actually occur in nature maybe not. 

Margaret does a great overview of Euclidean Space (Planar), Spherical Space and finally Hyperbolic Space. For those that are not visual or auditory learners though here is a quick overview of getting from destination A to destination B in each space. 

• Euclidean Space (discovered by Euclid (go figure)) is essentially planar space. There is a whole lot of math involved in a full explanation but essentially it is the rule that given a strait line and a point the only line that can be drawn through the point and not go through the line is parallel to the original line and there is only ONE of those.
  
• Spherical Space is a similar math run but can be simplified to say that when you have a line drawn in spherical space and you have a point there are ZERO ways to draw a line that doesn't intersect from that point and the line.
• Hyperbolic space has even more math and if you go back just a little bit in time much of that math is proving that it is impossible. However, nobody ever told choral that so it uses it all the time. Following the line and point theory again this basically is the statement that given a line and a point there is an infinite number of lines that can cross the point without crossing the original. Margaret does a great tactile example of this with a demonstration of a simple crochet piece.

So, that was a lot of words and explanation to say we see this type of space in multiple ways. It is easy to jump to an explanation of "multi-dimensional" space being the answer but think about the web. You can get anywhere from anywhere with one simple button push. There may be a mess of routers and switches that do math in between but to you using your browser it's click and away nearly instantly. As the math for this type of space matures and we understand it better things like "folding space" and traversing large "distances" instantly may become a reality.

What fun to think about. All from something as simple as a piece of choral made out of crochet. Next time you watch grandma spin that yarn just think, she could be paving the way to space travel.

IBM in talks to buy SUN

Well not two days after I say how their cloud innovations may be too late in coming for SUN... now IBM is in talks to purchase them. How very interesting.

This potential deal is interesting on multiple fronts.

• RISC (Not Risk as in risky but RISC as in Reduced Instruction Set Computer) This potential merger creates a big home for the remaining non-Intel chip guys. While HP does have their own this combination of fanatical Sun worshipers and the IBM "tried and true"ers is a potentially strong combo. Very different audiences and users though with one driven by tech innovation in Sun users and the other in the "never been fired for picking IBM" mentality.
• Java - Not a lot of the press on the topic brought this up. I guess because it is a shared standard and all that rigmarole. The reality though is that Sun still drives Java in a lot of ways. IBM on the other hand has been a big Java pusher on multiple fronts for development tools, open source etc. This could be a big deal. Of course there is always the gotcha side of the new languages such as SCALA coming up in popularity for the many-core problem.
• Open Source - Sun was big big in the .com era and went down when the .com boom went up. Since then they pushed into Open Source fairly successfully and actually have several software products that they are responsible for that have decent success. Grid, MySQL, even Solaris (a religion among some by itself) are big software assets. Of course the hard part is that IBM has their own efforts in these areas.
• People Logistics - If Sun has an office somewhere, so does IBM. The overhead portion of this could be a big savings opportunity. Not geeky though so I will stop there.
• Competition - Some of the press has stated this is a response to Cisco's entry into the data center space for servers. I doubt that but it does narrow down the big data center players. HP, IBM... ... well...

All told I think this could be good for Sun and IBM. If done right. (isn't that always the trick though... if it works everyone is a genius if it doesn't they were doomed from the get-go.) It gives me some hope for these products and companies for the future and in any case it will be fun to watch.

More cloud updates

Since my last post I have been sent a few and read a few other articles that I thought might be worthy of note related to what clouds are and why it is important.

Sun's CTO Greg Papadopoulos did an interview with Informationweek and discussed some of the efforts that SUN has going on. One of the biggest items of note in that article is the new Drizzle product coming out of the MySQL work. There are many articles and blog posts on the topic as well but the gist is that it is an effort to Cloud-smarten MySQL. Seeing how "MySQL gets used, people only use a subset of the relational capabilities because it has a horizontal scale and there are all these concerns that go into large-scale deployments. So Drizzle sort of strips back down to a small core and then builds up the distributed capabilities." I have made no secret of my feeling that Data is no longer relational. So I think this is encouraging news though as seems to be the case lately with SUN it may be too late for MySQL.

(Aside... Greg Papadopoulos and David Douglas recently co-authored Citizen Engineer: A handbook for socially responsible engineering. Though it is not yet available you can check out an interview with David Douglas on the book and engineering at the Mercury News. Personally I find this to be a refreshing take on the responsibility of Engineering as a profession and all of the personal and corporate responsibility that should go along with it.)

As if they noticed Microsoft pushing cloud tech Amazon announced a few new items in their own cloud initiatives. The ability to reserve capacity for known needs. They also expanded the ability of Windows based services in another Amazon zone. All of this from a company that sells stuff, proof that necessity is the mother of invention.

Even Microsoft realizes it’s all about the cloud. (though opinions still differ on what a cloud exactly is… I personally like the blind men describing the elephant/cloud analogy)

Wired did an interview with Ray Ozzie - Long article but worth reading even if you are an ABM coalition member. (I have always thought highly of Ray Ozzie, even if I did hate Notes itself as an administrator and user, it was visionary in its setup and ahead of its time. Heck… Ray even saw P2P coming before the rest of the industry started to catch up.)

Further proof of the gap between the over-served and the under-served. The question is if Microsoft is late enough to the cloud innovation game that their delay will actually cause them harm in market share. They have rallied before and come in with overwhelming response. (e.g. I.E.) The truth is the services such as PPT, DOC etc is so ubiquitous in the work environment it will be difficult to displace them with anything else. The question though becomes more of what will happen to those types of documents and will they be service disassembled themselves enabling other innovation plugs to go in. Also, who knows maybe Microsoft will have the bones to push further change into media distribution and synch. Though Apple has a pretty strong hold and the media providers can’t seem to understand that technology can help versus hurt their business (self destroying DVRs anyone? Eventually a multi-billion dollar DVD business… sigh)

What a great time to be a technologist.