Tuesday, April 28, 2009

Making sense of nonstructured nonsense - Trend Spotting in the Over-Information Age

As we look out over the web and remember that all of this information at our finger tips didn't used to be there it's a little hard to believe. The current Generation Y has a reason to be asking as this information has always been there for them. They are used to being perpetually connected and living their lives online. Tools like Twitter and Facebook make connectivity via digital methods a default. Connectivity from access devices such as iPhones or something as strait-forward as SMS provide snippet updates and easy push of information. Even Microsoft is getting into the game with their recent Vine announcement. All the while "normal" information continues flowing out. Papers are published and go to the web, news releases feed directly out, blog posts go out every second. Lots and lots of unstructured data every single day.

This brings about the challenge of the next generation of systems. Simple links to information without context aren't necessarily useful any more. When the search engines first came out the ability to point you in the right direction was enough. It no longer is. If a search engine returns a particularly pithy 140 character tweet on my topic I probably won't be thrilled. If it returns a mesh of links based on that tweet connecting with others, blogs and research topics in a neat visualized package... well that would be cool. 

For example, I am obsessively (those that know me probably are nodding their heads right now... I would ask you to stop please) interested in ants, papers on ants and recent research about ants. I believe that how they work, organize and where collective intelligence emerges provides us with many great clues on building scalable systems out of unstable parts. But in order to keep up on this I need to keep running searches and queries against various engines, look to known research outlets, receive email pointers from friends and build my own views and filters to what I think is worthwhile.  

Another example, twitter and other outlets can give indicators of where traffic problems are. People get stuck in traffic and tweet to their friends they are stuck. What if that information were automatically gathered, from the problem areas themselves, with geocoded info and placed on a google map mashup. Talk about useful. (I will of course conveniently forget to mention the downsides of texting, tweeting, facebooking and driving. Don't do it.)

Visibility to all of this disorganized and free flow data is the next great mountain to climb. Terms such as Crowdsourcing, Trend Spotting, Data Visualization are the buzzwords of the day. For a quick and inspiring note on what you can do with this type of information check out Erik Hersman's Ted Talk.  Knowing what is getting buzzed on Twitter and where the collective consciousness is headed is good and in many cases ahead of "standard" news outlets. How to create tools to view this information in a meaningful way is the challenge we need designers and engineers to be working on. 

In the meantime if you see good ant stuff let me know. 

Saturday, April 25, 2009



I watched an interesting Ted Talk that got me thinking along a couple of avenues. Margaret Wertheim presented The beautiful math that links coral, crochet and hyperbolic space. Go ahead and watch if you have 16ish minutes. I'll wait. 


I have mused before that we know less about the universe than we like to think. For simple examples look at things like Dark Matter and Dark Energy... wonderful hypothetical constructs that we don't really understand that make up only 80ish percent of the universe... 

When you look at things like science fiction and thin
gs like space travel and teleportation in particular a good amount of suspension of disbelief is usually required. Well, when you look at some of the items that actually occur in nature maybe not. 

Margaret does a great overview of Euclidean Space (Planar), Spherical Space and finally Hyperbolic Space. For those that are not visual or auditory learners though here is a quick overview of getting from destination A to destination B in each space. 
  • Euclidean Space (discovered by Euclid (go figure)) is essentially planar space. There is a whole lot of math involved in a full explanation but essentially it is the rule that given a strait line and a point the only line that can be drawn through the point and not go through the line is parallel to the original line and there is only ONE of those.
  • Spherical Space is a similar math run but can be simplified to say that when you have a line drawn in spherical space and you have a point there are ZERO ways to draw a line that doesn't intersect from that point and the line.
  • Hyperbolic space has even more math and if you go back just a little bit in time much of that math is proving that it is impossible. However, nobody ever told choral that so it uses it all the time. Following the line and point theory again this basically is the statement that given a line and a point there is an infinite number of lines that can cross the point without crossing the original. Margaret does a great tactile example of this with a demonstration of a simple crochet piece.
So, that was a lot of words and explanation to say we see this type of space in multiple ways. It is easy to jump to an explanation of "multi-dimensional" space being the answer but think about the web. You can get anywhere from anywhere with one simple button push. There may be a mess of routers and switches that do math in between but to you using your browser it's click and away nearly instantly. As the math for this type of space matures and we understand it better things like "folding space" and traversing large "distances" instantly may become a reality.

What fun to think about. All from something as simple as a piece of choral made out of crochet. Next time you watch grandma spin that yarn just think, she could be paving the way to space travel.