Jasper's Implicit Web

Are we going to see Hadoop on Tape?

2012-01-24T10:10:00.000-08:00

I found some fascinating video and pdf with a contrarian analysis of how Tape vendors might see higher growth rates than Disk vendors over the next few years. Given the Hadoop I/O pattern of sequentially reading gigabytes of data, followed by sequentially writing gigabytes of data, will there be a point at which (at least some subset of) Hadoop jobs run directly off tape?

A new way to monetize the web

2012-01-22T12:39:00.000-08:00

Studying Viglink, founded in 2009 to monetize links on blogs and forums. As easy to use as AdSense; just install some javascript on your blog or forum template and forget about it. Every time your audience members click links to any of the thousands of sites affiliated with VigLink, VigLink enhances that click with the appropriate code and if your audience has any propensity to buy, checks will start showing up :-).

Now that's pretty cool compared to the hassle for bloggers of joining multiple affiliate programs and attaching the appropriate code on each link (forget about teaching your forum's users), but it got even better last November as they rolled out the LinkWeaver feature, which automatically inserts up to five links per paragraph when it detects brands, merchants, or, in the near future, products.

As is clear from the type of people they're hiring, this new feature is an ideal playground for the implicit web. At 3.5BN page views per month they must have been collecting click-through rates on huge quantities of anchor text, as well as yields on pairs of anchor text and URLs. So version 1.0 may just insert the best-yielding URL for any anchor text it has seen before. Even then, considering [Apple iPhone 4S], should they link [Apple], [iPhone] and [4S] separately, or [Apple] and [iPhone 4s], or [Apple iPhone] and [4S], or save some of the five-link-budget for the Nokia and Sony references further on in the paragraph? Likely one of the reasons they stuck with brands and merchants for now.

Even with just brands and merchants, context matters. So far they seem to address this by requiring a category at sign up; I chose "Books and Magazines". It was amusing to discover how often I refer to Amazon, even after I joined eBay, although a large portion of my references are to Amazon Web Services, not the store. On the other hand, the word "Ford" in an old post presumably wasn't linked to the automotive brand because I didn't classify my blog as "Automotive", but in a way that link would have been more appropriate than the "Amazon Store" links on "Amazon Web Services" references.

And that's just the static context. They're also considering Real Time Auctions which, beyond keywords, could eventually come with audience, social, and local data. This could well be the ultimate performance ad, immune to "ad fatigue" by its very innocuousness (other than a "Link added by VigLink" mouse over that appears after about a second they are indistinguishable from normal links), and with far more impressions than any other type of ad. That is, as long as people continue to read :-).

Facebook's implicit web trumps Google's

2011-06-07T07:53:00.000-07:00

Techcrunch has a good article on how Facebook will use its implicit web to trump Google. When will they launch web search? Or would they acquire a player .. Blekko might be a good candidate given its already quite substantial Facebook integration.

Latest entrant: Blekko

2011-05-15T08:33:00.001-07:00

Possibly the most exciting entrant to the implicit web is the new search engine Blekko, which uses user-curated slashtags to make search more relevant. Disappointingly my blog wasn't crawled yet, but hopefully my creation of the /implicitweb slashtag is going to change that :-).

Better late than never ..

2011-05-15T08:09:00.000-07:00

.. getting back to my blog after a long time I noted the subject is still central to what I do on a daily basis, although I now apply it to eCommerce at Shopping.com rather than Customer Service at Openwater.

I also discovered it ranks #6 for this term on Google, not bad :-). Perhaps not surprising Wikipedia ranks #1, funny I hadn't seen that before.

Awesome analysis of how Amazon controls e-commerce

2011-05-14T10:43:00.000-07:00

Here's a great presentation laying out Amazon's strategy to control e-commerce http://techcrunch.com/2011/05/11/how-amazon-controls-ecommerce-slides/?utm_source=twitterfeed&utm_medium=twitter

Vertical Shopping Search as a Service

2009-11-05T10:02:00.000-08:00

Here's an interesting news item about google's commerce search, searching in a SaaS model for shopping verticals. This is a serious threat to the Endeca's, Omnitures and FASTs of the world.

More intent -- will Yahoo! catch up with this?

2009-05-22T11:53:00.000-07:00

Interesting overview of new features.

The best thing since sliced bread?

2009-05-08T15:12:00.000-07:00

Read a seemingly very biased article about Sphinx.

The author seems to resent the Apache Foundation in general and Lucene/Solr and Java in particular, so whatever else he says I will take with a grain of salt. But for people who think that text search as supported by RDBs is the only search you need it might open some eyes ..

Cloud computing and Ford

2009-04-20T09:35:00.000-07:00

This article

Cloud is Bigger Than the Internet - II
— Finally coming to grips with cloud means coming to grips with its high-level potential. This thing is bigger than the Internet, probably not bigger with fire, but maybe on a par with electricity.

surmises we have no "father or mother of cloud computing".

Depending a bit on how you define Cloud Computing, one might argue that we do have one: Jeff Bezos. People tell me he pushed hard on the project to make Amazon's infrastructure available as a service, which made Cloud Computing available to everyone with a credit card.

Sure Cloud Computing is built on virtualization and sure it uses technology honed during the waves around grid computing, utility computing, all the way back to "time sharing" on Mainframes. But all of that relates to Amazon Web Services as Arpa/Internet relates to the World Wide Web.

Of course Werner Vogels would be the person who made AWS happen technically, and I'm sure he didn't do it alone, so maybe pinpointing an exact person isn't so fruitful.

I do think that Jeff Bezos/Werner Vogels/Amazon are to Cloud Computing what Ford (Motor Company) was to the car industry.

Interesting company that performs language engineering for Enterprise Search in Dutch

2009-02-26T10:03:00.000-08:00

I just discovered Gridline, a Dutch company that performs the type of language engineering for Enterprise Search we have been doing at Openwater for the last 2.5 years.

Too bad their website is in Dutch. Google Translate actually does a decent job of translating their home page.

Yet another proof point on how important language, and specifically the language of a community, is for Search.

First Woman is Dutch, two in total, both from Delft

2008-07-21T07:16:00.000-07:00

Fun to discover the Dutch women among the Anita Borg Scholars. UK and Russia have more of them, though.

Mandatory Reading On Search Results Ranking

2008-07-19T11:22:00.000-07:00

By far the most informative article I've ever read about how Google ranks its results:

Official Google Blog: Technologies behind Google ranking

Is your Service Better than Average?

2008-04-25T10:37:00.000-07:00

If you think so, you're in good company: 75% of CEOs think the same. But you might be delusional as well, because 60% of actual customers are somewhat to extremely dissatisfied with their latest service experience.

That's what I read in this morning's Journal review of The Best Service Is No Service by Amazon alumnus Bill Price and Limebridge Sales and Service Consulting Director David Jaffe.

Once at work, there was no way that WSJ's search bar would find the article (boo!), but fortunately I googled upon Guy Kawasaki's blog-interview with Bill Price . That article has a lot of gems too, like how to find companies with bad service:

"All you need to do is look for companies that hide their phone number on web sites"

and

"We have yet to find a company that couldn't improve service and cut costs at the same time"

The latter could have been a quote from our CEO Mike Rocha whose major claim to fame is having done exactly that at Oracle Corporation, and who founded our company to do it again (but better) for the rest of the world.

So how do our Service Networks help address Bill Price's Seven Principles, and how do implicit web techniques factor into that?

1. Eliminate dumb or avoidable contacts to free up capacity and slash costs

A study at Oracle showed that 99.9% of all service requests were satisfied with known information. This is why service networks focus on making existing information easily accessible, thus eliminating a huge amount of dumb or avoidable contacts. Implicit web techniques help organize the information by extracting the domain language and creating a Service Encyclopedia. Implicit web techniques also help match service requesters to the information they need -- think Amazon's "people who looked at this book ended up buying this other book".

2. Build self-service that works to free up even more capacity and cut costs even more

One of the most powerful forms of self-service is your customers helping each other. This type of participation on our Service Networks goes far beyond classic forum discussion: you can initiate a discussion on any page ("does this procedure apply to the revision that I have?"), chat with people related to the topic or issue, and share or tag relevant information such as test plans or best practices. Implicit web techniques are used to emphasize the most relevant people and information.

3. Find ways to be proactive rather than reactive because it is often cheaper than waiting

Service networks index and analyze everything that's going on so you can see issues developing much earlier -- including mood analysis that flags developing irritation. It's also easier to target news to the people that need to know.

4. Engage the real "owners" of customer problems to work with the customer service team to fix the problems

Traditionally vendors of high-tech products have flouted this principle by limiting the number of people they would interact with at each customer, because they feared losing control over the flow of information. On a service network this is no longer necessary because all people, information and transactions are linked and transparent. At any given point in time, the most appropriate people from both vendor and customer can work on the issue at hand.

5. Make it really easy to contact your business

Service networks provide myriad ways of contact (chat, email, forum dialog, phone, desktop sharing) and what's more, they're linked so you don't have to reconstruct context at every contact (please tell me again your account number).

6. Use the contacts you get to listen closely to the customer, and act upon WOCAS (What Our Customers Are Saying)

This also gets easier because the service network extends into the entire enterprise. No transferring problem descriptions from the CRM system to the developer Bug system and Requirements Management system. We just index all three systems and derive the links. On top of the indices we create dashboards to track (categories of) issues and drilling down into the specifics of any issue is just a click away.

7. Fix reporting metrics, processes, and the staffing side to deliver great experiences for customer contacts

When we were just getting started, we made an extensive study of Amy Jo Kim's Community Building on the Web : Secret Strategies for Successful Online Communities . Her #1 advice is that an on-line community should have a (published) purpose. Whenever we start a new Service Network for one of our customers, we make sure we define its purpose, which should be more specific than "offer great customer service". Instead we set targets that include one or more of Bill's CPX metrics, e.g. "Reduce the average number of contacts to resolve an issue by 20% in 2008".

Service Networks make it easy to define, explain and publish their purpose, as well as publish metrics showing how they're doing.

Twine, Applications and Green Fields

2008-03-27T13:56:00.000-07:00

Looking at Robert Scoble's interview with Twine's Nova Spivack reminded me of Freebase, Knol and countless others before. Basically the interview goes like this:

1. Sir Tim Berners-Lee* has talked about the importance of the data web, or more recently about the Giant Global Graph (GGG)*.
2. Twine has built technology that can create, maintain and query a data web.
3. Let's sit at our editor, create a little data web, and run a query
4. Look how beautiful, imagine all the wonderful things you could do with this!

* Substitute appropriate visionary here, like Danny Hillis or Won Kim
** Substitute appropriate information infrastructure technology like "semantic database", "object-oriented database", etc.

Now I did enjoy the interview and the editor does look nifty but I was still a bit disappointed.

First because the presentation is very technology centric, cleverly leaving the applications to the imagination. But what's a dataweb (or any information infrastructure) without applications? No, the editor from step 2 doesn't count as an application. Even a query interface or browser barely deserves that qualification.

Compare that to a database like Oracle. The relational database concept developed by Jim Grey et al was a major breakthrough, as was the query language SQL. But there really isn't much use to a relational database until you run business applications on top of it -- think of Payroll or Shopping Cart. Oracle didn't win the database war from IBM, Informix, Tandem, Microsoft and countless others because their database was technologically most advanced, but because they focused on the applications -- lately in the extreme by selling the database as well as its applications, either developed in-house or obtained through myriad acquisitions. Here's a list, and that's only the "strategic" acquisitions!

So I perked up when Robert asked "what are the applications you have in mind", but I slumped back when I heard Nova answer "we think this is something that would be used by work groups".

The second disappointment was what I call the "Green Field Approach". The demo starts with Nova creating a Twine, cutting and pasting some text, and then creating relations with other twines and some existing web pages. A bit further in, Nova shows how you can import information from other sources, but that seems an afterthought: "we're looking at what other sources might be interesting".

But given the terabytes of information that are already out there it seems the last thing we need is human authors creating new pages -- shouldn't the main goal be to navigate, organize, link and clean up existing information? Shouldn't creating the dataweb be as easy as tagging -- which very successfully avoids the creation of original content?

The best of luck to information technology companies -- we definitely think of ourselves as one.

But remember:

No Apps, No Glory!

and

There is Plenty Information Already!

Wikinomics and Maslow's hierarchy of needs

2008-03-23T14:17:00.000-07:00

Reading Wikinomics I wondered what it takes for wikis to work in the enterprise. Jimmy Whales, the head Honcho of Wikipedia, has said that 0.7% of users did 50% of the edits on Wikipedia. If those same numbers held for the enterprise, a 1,000 person company's wiki would hinge on the contributions of 7 people. There has been debate about the Wikipedia numbers, but beyond that debate the numbers don't translate to the enterprise because the incentives are different.

What are the incentives to produce content on Wikipedia? There is the altruistic motive of providing free (as in beer) truth to the world, there is the satisfaction of finding and correcting flaws and, perhaps most important, the value of asserting yourself as a topic expert.

Referring to Maslow's hierarchy of needs, none of these incentives translate into level one (physiological) and two (safety) needs, perhaps "belonging to the Wikipedia community" might qualify at level three.

Contrast that with creating or editing a useful page on your company's wiki. Saving colleagues time not only improves the bottom line but might get you noticed by your manager or perhaps even your manager's manager, both of which translate into food and safety (of employment).

While we couldn't extrapolate our numbers because service networks go far beyond wikis -- especially by automatically providing much-needed structure -- we have been seeing the power of those incentives during implementations of Service Networks for Upgrades. Traditionally, non-IT employees shy away from tasks in planning or testing an Oracle Upgrade, but when they see how their contributions will be visible on the network, they often jump on board. With the help of their published experience, upgrades are completed in half the traditional time and at one quarter the traditional cost.

As a lot of momentum is going into Enterprise 2.0, I hope to see more of this type of quantification. While Wikipedia and Facebook may run on Maslow's belonging, esteem and self-actualization, mainstream enterprises will need to understand the material impact on their business for Enterprise 2.0 to really take off.

Knols, Wikipedia and truth in Openwater Service Networks

2008-01-16T12:02:00.000-08:00

Last month, Google unveiled knols. In Udi Manber's words,

The key idea behind the Knol project is to highlight authors.

Unlike Wikipedia, which contains a single definition for each topic, authors write competing knols about the same topic.

Udi argues that competition is a good thing. Google will not edit the knols, but use its considerable Search Quality expertise to let the best version float to the top.

Considering Wikipedia and Knol raises interesting philosophical questions about truth. Take Hugo Chávez . Many view him as a socialist liberator, many others view him as an authoritarian demagogue.

Wikipedia editors deal with these versions of the truth in a variety of ways, labeling actions or statements as "controversial", requiring references to facts from reputable sources, flagging text as "opinion" and even introducing a separate topic Criticism of Hugo Chávez . However admirable these efforts, some bias undoubtedly remains. For example, these graphs make him look like a hero to a visual person like me. The text next to it has enough nuance that I couldn't tell. And that's where Wikipedia (or any encyclopedia, for that matter) usually leaves me -- my head full of facts from more or less reputable sources, various conflicting interpretations of these facts but no point of view.

Enter Knol. Knol isn't live yet, but I imagine we'd get at least two knols: Hugo the Good and Hugo the Bad. It will be interesting to see which Hugo will float to the top -- it will be a reflection of the community that views and rates knols.

The word "community" provides a good segueway into what we do at Openwater. Rather than demanding one version of the truth, or letting several truths compete, we build service networks for communities (specifically the installed base of a high-tech product), each of which creates its own Community Encyclopedia.

As in Wikipedia, anyone in the service network can edit a topic and each topic has experts assigned who monitor the quality. But like Knol, the editors don't have to balance every sentence that appears to carry an opinion or bias. And perhaps the biggest difference with both Knol and Wikipedia: there is no need to consider all possible meanings of each word -- just the current meaning within the installed base community is fine.

As a result, topics can be more specific and concise than those in either Knol or Wikipedia, and at the same time they can index more specific (proprietary) information and people in the installed base community.

There is one other major difference I'd like to mention in this blog -- this is what makes us an "implicit web" company. Unlike Wikipedia and Knol, topic pages don't start as an empty page and a blinking cursor.

In an Openwater Service Network, we start by indexing all the structured and unstructured information that's already available to the installed base, such as technical documentation, forums, bug tracking systems, LDAP directories, mailing lists, project plans, etc.

From the index, we then extract candidate topics by applying Natural Language Processing techniques to the unstructured text and by applying queries to the structured information. These candidate topics come with links to relevant documents, people, terms and other information. Users select suitable topics and refine them into the Community Encyclopedia.

The resulting pages look a lot like those in Wikipedia and Knol, but they're a lot more relevant to the community and a lot less effort to produce.

Because of the way it's built, we can also use the encyclopedia to program the network, but that's another story.

Spock attacks the identity crisis

2007-12-21T09:49:00.000-08:00

Two days ago I got Spocked! I'm impressed with their approach to solving the identity crisis:

- Index as many external "people" sites (linkedin, plaxo, address books from the largest webmail providers, DBLP bibliography database, etc., etc.)

- Let users invite each other into trust networks

- Let users tag each other with relevant attributes

- Let users vote on tags

- Allow users to tag relations -- i.e., assert a relation between two different people. Other than in Freebase I haven't seen this, but Spock is much more liberal, allowing any type of relation (with the flip side of not establishing inverse relations).

- Use all the resulting metrics (believability of a tag based on votes, authority of a user based on number of correct tags, social metric of people according to relations with other people and their authority) to create an equivalent to pagerank (Spock Power)

- And the kicker; allow users to merge records about people

Obviously the latter one is a huge contributor to solving the identity crisis using collective intelligence. I think Spock recognizes the power of particularly that type of contribution because after I merged a couple of records about people I know, my Spock power went from 161 to 1729!

There has been a lot of discussion on the (lack of) ethics of the Spock team here and obviously they will have major hurdles fighting spam, but the Spock uri (e.g. mine is http://www.spock.com/Jasper-Kamperman-z32I1yG ) might become a pretty usable online identity.

But being in the top 1% of users after only two hours of tagging and merging and the top 1000 leader board having scores as low as 2,500 leads me to believe their number of users is less than a million -- which includes a lot of europeans. So they don't appear to cover a very large part of the web just yet.

Solving the identity crisis

2007-11-16T14:29:00.000-08:00

Will the large scale Semantic Web ever happen? As in a significant portion of the web explicitly annotated with classes, taxonomies, properties, even microformats? I'm not holding my breath, and a lot of implicit webbers with me.

Just consider how hard it is to design a sound ontology. How much harder it is to standardize on one ontology. Or to map between ontologies. Then imagine how all web publishers are going to deal with those issues. Hundreds of millions of them, including you and me.

Implicit webbers don't wait for publishers. We like shallow taxonomies and we guess if we need to. And of course we'll accept any help we can get, even if it's labeled "semantic web".

Some of that help may come from OKKAM. These people seem to have found a tractable corner of the Semantic Web. Here's my take on their recipe:

Forget about classes, taxonomies, properties and description logics and build a service for just identity -- a taxonomy doesn't get flatter than that.
Create a unique ID for Jasper Kamperman. Resist the temptation to classify him as a Person, Male, Musician, Dutchman, Sunnyvale-dweller, Computer Scientist, Openwater Architect. Don't try to maintain his current phone number, address or affiliation. Don't send him email to "update his ecard".
But do record that jasper@cwi.nl, jasper.kamperman@cwi.nl, jasper.kamperman@idr.nl, jasper.kamperman@reasoning.com, jasper.kamperman@intel.com, Jasper F. Th. Kamperman, Jasper Kamperman, PhD, http://www.linkedin.com/in/jasperk64 , jasperk64 at yahoo dot com, and jasper dot kamperman at openwaternet dot com all refer to the same entity.

The result is a unique ID for each entity and a large set of clues that helps us make the right guess.

I wonder if or when they'll have an OpenSocial adapter .. imagine the possibilities!

Along came the Implicit Web

2007-11-02T16:17:00.000-07:00

A conference about the implicit web inspired me to finally start this blog. The concept isn't very well defined yet, but then again, how much fun would that be?

"Implicit Web" caught my attention because it captures so well what we're doing at Openwater -- building service networks that connect people and information to create better service.

And right there you have an example; how do you know if service is better? That's simple, you conduct a survey. Or do you? Well, I don't know about you, but personally, I detest surveys. And I get really mad at software like WebEx's Meeting Manager that insists on a survey after every single meeting. And rating those movies on Netflix gets old quickly, too. Surveys are annoying exactly because they require you to be explicit.

So how about analyzing service network activity and creating implicit measures of quality?

Take online forums: How long does it take for a question to get an answer? Is it really an answer? How many people read that answer? How many people link to it? Does anyone write "thank you"? Do they come back?

Or take a technique called Search Analytics, explained eloquently here by Gery Angel. How often do users type slight variations of a search query? How many queries return any results at all? Do users click the first, the second or the third search result?

There are endless possibilities in measuring service quality alone, which in itself is only a small but necessary part of what we do. Which brings me back to the reason for starting this blog -- "implicit web" seems to cover a whole swath of subjects we're working on at Openwater. Stay tuned for more.