New Blog

I have a new blog at adamhurwitz.net on Security, BigData, and the Cloud.

You can also follow me on Twitter @adam_hurwitz

The Armory Show NYC

I visited The Armory Show last week-end, a very popular art fair (now too popular for some, of course) that draws galleries from all over the world to show off their contemporary works. They've added some modern work, but the contemporary is still the focus.

And as I walked along I couldn't help but think that there are artists ("artists") basically trying every possible idea of what could be considered art. It brought to mind a depth-first search algorithm, as if some huge computer cluster were exploring every idea and then every variation of every idea relentlessly looking for art, or at least something that people will buy.

What is BigData?

At some point quantity becomes quality. You can add a few more of something and suddenly it can become something different. For data we have crossed that threshold again over the last decade. Now whether that point is defined as terabytes or petabytes is in-itself relatively unimportant. What is important is that people have had to manage this qualitatively different amount of data, and in order to do so they have created new technologies, new techniques, and new ways of thinking about and analyzing the data. This in turn, of course, has created new opportunities. This new paradigm is known as BigData.

Practically when people speak about it, I believe they are referring to two specific things: 1. the new set of data management and processing technologies that involve either distributed or parallel computing; 2. an exploratory Business Intelligence activity that utilizes machine learning and statistics to extract knowledge from the data.

New Date Announced for NY Tech Mixer Meetup

If you're interested in the NYC start-up scene - and these days who isn't? - then you have to come to the next Demos and Drinks on January 18. At Demos and Drinks a handful of start-ups get to demo and everyone drinks and socializes. The crowd is a mix of techies, designers, biz folks, and investors. It's organized by the NY Tech Mixer Meetup which has over 1500 members and regularly attracts 300+ people per event. Check the event page for more details. Sign-up now before it sells out. It's going to be a great event and it's going to be monthly from now on.

O'Reilly Strata Conference on Big Data

I'm really honored to have been chosen to be part of the O'Reilly Strata Conference, coming up in February in Santa Clara. It is a new conference that focuses on the business and practice of data.

All industries are experiencing an explosion of data volumes. But most companies are finding it hard to keep up and make use of all the data that they generate or have access to. This is creating great opportunities for those who understand how to effectively collect and use large quantities of data.

I will be part of the Real World Applications Panel: Enterprise and Industry. My focus will be the legal industry, where I work now, and there will be two other panelists from the music and meteorological industries.

It is going to be a great event. Drop me a line if you are going to be there.

About Us is the most important

I was looking at the web stats of a company website the other day. The company is relatively small and provides services to businesses. And the thing that struck me right away is that the most popular pages were the About Us pages.

This made me realize that when I look at other company websites, I instinctively go straight to the About Us pages instead of the products and services pages. If I'm on their website, it's more than likely that I already know what they do. And if I don't, then all I need is a quick scan of the homepage. I'm really much more interested in seeing the management team, the board of directors, the investors, and maybe some history in order to understand the company. Only after I get a sense of who's involved do I consider actually reading about their services or products. I have a feeling that this is becoming the norm.

HCIR 2010

I attended the HCIR 2010 workshop last Sunday on “Bridging Human-Computer Interaction and Information Retrieval”. Dan Russell from Google gave the keynote. This is a guy who is passionate about what he does, and it seems that his job is to study how people search, including how they think about searching, how they conduct searches, how they use search interfaces, etc. and to figure out how they could do it better.

One of the biggest problems that exists, he believes, is that there is no training or education for search. The overwhelming majority of people just do not know how to search well. For instance, he found that most people don't know that you can use control-F on almost every program in order to find a word or phrase. This is something that most techie people take for granted when they are looking for something. For certain searches, you look at the results in Google, you pop open one of the results that looks promising, and then you control-F to find your keyword on the page and see if it is what you're looking for. The step of using control-F obviously speeds things up greatly. According to Dan this is something that most people just don't know about. I was skeptical and so I mentioned it to someone I know who is a professional (college educated, works in an office, etc.) and he knew that you could do that in Excel but didn't realize that it worked in all programs. I was sort of astounded by this, which was something that Dan had mentioned happens to him all the time. So for search instead of making the results somehow better, there may be a lot more value in training the user and providing tips or heuristics to help the user while they are searching.