RE: distinctions between blog search engines and traditional search engines

After posting my recent post entitled "Is the distinction between a blog search engine and a traditional search engine growing meaningless?", I decided to call it to the attention of the Blogspotting crew in a creative manner by issuing them a challenge in a comment on their blog post entitled "Remove "blogs" from the headline?":

"... the distinction between a blog search engine and a traditional one is growing meaningless."
---

Sorry for the delayed comment, but I wanted to think about this a bit more thoughtfully before shooting from the hip.

I have written up a full blog post commenting on this point of the meaningfulness of the distinction between blog search engines and traditional Web search engines.

My challenge for you (or any reader):

Try finding my post using both a blog search engine (of your choice) and a traditional Web search engine (of your choice) and think about what the experience was like and then tell us about the distinctions or lack thereof and their meaning or lack thereof.

-- Jack Krupansky

To be honest, I have not tried this challenge myself. Actually, I do have an excuse: I need to wait for the various search engines to crawl and index my blog post. I'll give you one guess which form of search engine will show my post first!

-- Jack Krupansky

Is the distinction between a blog search engine and a traditional search engine growing meaningless?

In his blog post on his Blogspotting blog entitled "Remove "blogs" from the headline?" Senior Writer Stephen Baker of BusinessWeek opines that "the distinction between a blog search engine and a traditional one is growing meaningless" (actually, I think he is attributing that to Steve Rubel.) Superficially, I think that is absolutely correct, but there are some nuances.

What exactly is a "blog search engine"? Presumably it is something like Technorati, Bloglines Search, Google Blog Search, or Micrsoft Windows Live Search Feeds, among others. But, do any of these so-called "blog search engines" really search through all blog posts? Some of them may, but in general they are only indexing and searching the content of the feed (e.g., RSS feed) associated with a blog.

Although so-called "blog search" (feed search) is indeed quite useful, especially when looking for timely content, there are several difficulties when contrasted with generic Web search.

First, the popular usage is that RSS stands for Really Simple Syndication. That is okay and all well and good for popular, non-technical consumption, but it belies what is really going on. The hard-core, technical meaning of RSS is RDF Site Summary (where RDF stands for Resource Description Framework.) The semi-technical meaning of RSS is Rich Site Summary. The operative word from either of these two proper meanings is summary, indicating that the feed contains only a subset of the content of a site (such as a blog.)

The issue of summary comes in three forms. First, only of subset of the number of content items may be present in the feed. Some feeds may have a specific numeric limit such as 10 or 20 items. Some feeds may be time limited, such as all content within the past day, week, two weeks, months, 90 days, etc. In short, a blog search engine typically does not index all of the posts that exist on a blog and its archives.

A second form of summary is for the feed to have an abstract summary of the main content or an extended version of the headline. This is very common for traditional news stories or feature-length articles where the feed is designed for browsing, in contrast with reading the full content.

A third form of summary which is used by some blogs is a short intro of the post and then they insist that you click on a "More" link to read the full post. The author of such as blog then usaully has the choice of having only the summary in the feed or to put the full post in the feed. There are great debates in the blogosphere on the merits of those two approaches. The author may opt to include only the summary in the feed as a "teaser" to help drive traffic to their main site. On the positive side, short summaries greatly facilitate rapid browsing of large numbers of posts in feed readers.

The issue for all forms of summary is that a blog or feed search engine that looks only at the feed will not have the full blog post text to index and search.

OTOH, a feed has a very structured format for tags and other blog metadata that can facilitate more refined indexing and searching of blog posts, at least for the blog posts that are included in the feed.

In general, for each blog post there will also be a distinct permalink web page which has all of the text of the blog post in a form that has its own URL address and can be fully indexed and searched by a traditional, generic Web search engine. The tag information for a post is also there, but not always in a uniform, structured format that the non-blog search engine can index and search anywhere near as well as a pure feed-based search engine.

One important distinction between using a blog search engine and a traditional, generic Web search engine is that the latter tend to order search results strictly by so-called relevance, while the former frequently also allow you to order the results by date (reverse chronological as with a traditional blog) or a mixture of relevance and date, so that you can easily access timely content even if the top authoritative content may be too old to even show up in a typical web feed.

Technically, there is no good reason that a Web search engine could not also permit these options, but today they (in general) do not. Actually, there are some technical difficulties with date-ordering the general Web, but I suspect that most of them could be worked around (to some extent) if the search engine teams put their hearts into it.

What I would really like to see is for each of the main Web search engines to have options as to what mix of blog and non-blog content to display along with a reverse date weighting. Even with the general Web I frequently want to search for fresh content simply because I have already seen all the old stuff and want to see the new stuff or stuff that I haven't marked as read. That alone is a great argument for the need for blog search engines to continue to exist, but if the big search engines did just a little better job of integrating blog search the need for separate blog search engines would vanish.

In summary, for now I still find blog search engines (marginally) useful, but their days could be numbered, especially if they do not try to break out of their niche and start adding value beyond simply indexing and searching the (limited) content of blog feeds.

-- Jack Krupansky

Incompatability of TiVo and Twitter and the ownership of time

Stephen Baker of BusinessWeek and the Blogspotting blog in a post entitled "Twitter and TiVo: a bad combo" opines about the real-time immediacy of Twitter being incompatible with the desire to time shift sporting events with TiVo and the inability to time shift all of the suspense as well when using Twitter since all the good parts would have been spoiled by the real-time twttering.

I had a comment related to one of his points:

"The safest thing to do while the game is brewing is take shelter in the 19th century, and read a book or a magazine."
---

I would add to that list "or write a blog post on some long-tail topic which is more likely to have significant long-term social value than the most popular twitter topics of the moment."

Maybe that is a measure of how far blogs have sunk is the social networking hierarchy... they feel so "19th century", like reading books or using a typewriter or writing letters in longhand ("Blogging -- the 'longhand' of the social networking universe!") or licking postage stamps.

I gather that you are one of those people who would prefer not to "know" the future. How odd to be willing to speculate about the future but not be passionate about actually knowing it. Maybe there is a gene that is predisposed to preferring surprise. Maybe it is simply an addiction to the adrenaline rush that results from observing possibility become fact.

A later commenter (Jon Garfunkel) made a reference to "your time", which got me thinking and led me to make another comment:

"... watch the game on your time."
---

 

"Your" time? "My" time? Huh? That seems to suggest a belief that time is proprietary and "owned" by individuals.

 

I thought that in the (idealistic) Internet we weren't really supposed to "own" anything and that everything would be shared in "the commons."

 

I would surmise that the unstated goal of tools such as Twitter is to tear down such "walls" that seem to have the misguided purpose of giving the illusion that time is something that any of us could "own."

 

In the community of the commons, my time is your time is our time. It is all "open source time."

 

If the community were to permit or tolerate proprietary ownership of time, the next thing you know people would start asserting that they own their minds! Obviously that is completely incompatible with "the commons" of a socially-networked online world.

 

After all, aren't we supposed to believe that "information wants to be free"? Is time so different from information? How can any of us legitimately attempt to hold time a prisoner to our own narrow demands when its freedom is of so much greater value to "the community"?

 

That said, I do sincerely apologize for taking up so much of "your" time.

 

Back to the game!

It does seem as if the real-world sense of time is not particularly compatible with time in the online world. Something to think about. When I have the time.

-- Jack Krupansky

Google Blogger starting the new year in great shape

Google's Blogger blog service certainly has had more than its share of service problems over the past couple of years, but I am happy to say that Blogger finished 2007 and entered 2008 on notes of strength. I cannot even recall when I last had a service problem with Blogger. I decided to write this post since I noticed that with increasing frequency I can email a new post to one of my blogs and Blogger sends the confirmation email fast enough so that my email client is able to pick it up on that one click of the Send button. Go Blogger!

-- Jack Krupansky

Evolution of RSS in media: vanishing into transparent ubiquity

Heather Green of BusinessWeek and the Blogspotting blog continues to ponder how blogging has evolved since her 2005 BW cover story on blogs and asks "Did RSS Live Up to Expectations?" Her post closes with the assertion that "RSS doesn't seem to be mainstream yet." I could not quite agree with that and commented so on her post. I reproduce those comments here for your consideration:

"RSS doesn't seem to be mainstream yet."
---

1. Are there *any* MSM web sites that do not have those little orange "thingies" available right there within easy reach? How much more mainstream do you need to get?

2. As far as actual mainstream end-user usage of web feeds beyond techies using exotic feed readers, maybe that is mostly a question of the extent to which people use aggregation "portal" pages such as My Yahoo, iGoogle, Live.com, et al, without even necessarily knowing that RSS is the technology under the hood. Please try to gather some numbers on this type of usage, since I think it is a better gauge of penetration into the general online population.

3. Is RSS as a very specific technology what you really wanted to focus on, or is the more general category of "aggregation technology" of greater interest? For example, Google News is popular, but I suspect it is still the case that the bulk of the MSM stories may be collected from more traditional news feeds rather than specifically using RDF or Atom XML-based syndication formatting technology. I mostly read NY Times articles by clicking on links in a daily email message that take me directly to the story web page or maybe I just read the short summary contained directly in the email, bypassing the overall structure of the web site. Another example is my ISP's main web page which presents a bunch of boxes with lists of stories in various categories but without any apparent dependency specifically on "RSS" per se. So, the general concepts of syndication, feeds, aggregation, and direct access are there regardless of the specific technology used to implement those concepts.

So, in a number of senses, RSS is quite mainstream, but I suspect that as time goes by fewer and fewer users will recognize the literal term "RSS". Sure, today your site and other MSM sites let the user "manually" refer to "RSS feed", but increasingly the truly user-friendly interfaces such as My Yahoo, iGoogle, Live.com, et al allow the user to select sources with no reference to "RSS."

Ultimately, that is the true test of whether a technology has genuinely "arrived" and become hard-core mainstream: it vanishes into transparent ubiquity.

BTW, one point to make about "RSS": it hides the distinction between "blog" content and traditional web pages. For example, a NY Times RSS feed might have both blog pages from their site in addition to traditional MSM story web pages. So, in a very real sense, RSS is helping to level the playing field between blogs and traditional MSM.

-- Jack Krupansky

Sure, there are still plenty of nooks and crannies on the Web where RSS is not fully exploited, but RSS has in fact penetrated the overall Web to a surprising degree.

-- Jack Krupansky

Communication versus self-expression in social networking and blogs

Heather Green of BusinessWeek and the Blogspotting blog had solicited feedback from Netscape/Ning founder Marc Andreessen on doing an update of her 2005 BW cover story on blogs and has summarized his feedback. This was all well and good, but I had trouble with the way Marc seemed to be distinguishing social networking from blogs in terms of communication versus self-expression. I commented on her summary and reproduce my comments here:

I am a little baffled by the overloading of the term "communication" in the context of social networking as posited by Marc.

Before reading this post, I would not have ever imagined that anybody would have suggested that social networking sites such as MySpace and Facebook were any less about self-expression than about "communication." Nor would I have ever imagined that a blog was not just as much about communication as an act of self-expression. Blogging and non-blogging social networking both seem to be about both communication and self-expression.

Anybody want to clarify the distinction for me?

Be forewarned, I am adamantly opposed to attempts to overload simple English words with meanings that are not already in the dictionary. The idea that "communication" should have some narrower, constrained meaning in the context of non-blogging social networking seems a bit bizarre to say the least.

That said, I am quite curious why Marc is trying to use "communication" as a distinction between social networking in general and blogging in particular.

Just guessing, maybe he simply means that a social networking site provides a broader range of communication features while blogging provides a narrower range. But, that is simply a guess on my part.

Another possibility, that has nothing to do with communication per se is that a standalone blog tends to focus on "me" with links mostly as references to "sources", while a social networking "site" adds a strong emphasis on "my group" and a sense of community.

So, I am now guessing that Marc is really using "communication" as a placeholder for a tighter sense of "community." Again, this is simply a guess on my part.

Blogging also has a sense of community, but in a somewhat looser sense. Sure, you can "network" to some extent using blogs, but not anywhere near as effectively as with a "networking" site.

Finally, if you are going to broaden the story to cover non-blogging forms of social networking I would suggest that it include a discussion of "professional networking" as exemplified by LinkedIn. Sure, you can do professional networking with Facebook, but it seems a bit odd referring to professional acquaintances as "friends."

-- Jack Krupansky

I got no response in that post. Maybe somebody here could respond.

Maybe this is simply one of the overall symptoms of the current state of blogging in particular and social networking in general: vague confusion.

-- Jack Krupansky

Blogs 2.0 for BusinessWeek

Senior Writer Stephen Baker of BusinessWeek has a blog post on his Blogspotting blog entitled "Remove "blogs" from the headline?" which ruminates about some angles for an update or revision to the May 2, 2005 cover story in BusinessWeek coauthored by himself and Heather Green entitled "Blogs Will Change Your Business - Look past the yakkers, hobbyists, and political mobs. Your customers and rivals are figuring blogs out. Our advice: Catch up...or catch you later." The immediate question posed by his blog post title is whether "the story" has gone far beyond simple blogging. Even the original article discussed RSS feeds and podcasting, and shortly after publication Blogspotting also segued into wikis and looking at "how cutting-edge technologies are changing business and society" in general.

Technically, blogs and all these other technologies are known as social media, social networking, or social computing. If you are non-technical, the handle Web 2.0 covers the field.

There was also a conception that blogging and all this other stuff was about the conversation, and that The Cluetrain Manifesto by Christopher Locke, Doc Searls, and David Weinberger was the guide to helping professionals understand that #1, "Markets are conversations." The groundbreaking book Naked Conversations : How Blogs are Changing the Way Businesses Talk with Customers by Shel Israel and Robert Scoble nails that point. Robert is now off promoting video and every new social media tool that comes along while Shel is off educating us about global neighbourhoods. Nonetheless, the focus on the conversation continues.

I just finished rereading the original BW article and do not feel that it did a good enough job of zeroing in on that concept of the conversation. Sure, companies do now pay a lot more attention to what high-profile bloggers say and what appears on Techmeme, Techcrunch, or other professional blogs, but I do dearly wish that BW would try to make the point of encouraging corporate folks to be more conversational with the people in their markets.

I have a bunch of other thoughts about an update or revision or follow-up to the 2005 cover story, but I want to think about this more deeply before diving in too deep.

I know that "Blogs 2.0" is a lame title, but I think it is at least an appropriate placeholder for the title since it expresses exactly what is being discussed, namely the evolution of social media tools and online services far beyond basic blogging. Besides, even superficially, somebody seeing "Blogs 2.0" (or "Web 2.0") is going to be automatically curious what the authors are saying.

-- Jack Krupansky

LinkedIn public profile

The LinkedIn professional networking site now allows you to make your profile public and even personalize the URL for your profile. Mine is at http://www.linkedin.com/in/jackkrupansky.

My own LinkedIn network is still fairly small at 91 connections, but I do get a slow trickle of incoming acceptances of old invitations and an occasional new invitation.

I still have not gotten any great business value from LinkedIn, but it is a useful "rolodex" and makes it convenient to keep track of former associates. I occasionally get an email from LinkedIn highlighting job changes for people in my network.

-- Jack Krupansky

Edge question for 2008: What have you changed your mind about? Why?

John Brockman's Edge question for 2008 is now out: What have you changed your mind about? Why?

The Edge Annual Question -- 2008

When thinking changes your mind, that's philosophy.
When God changes your mind, that's faith.
When facts change your mind, that's science.

WHAT HAVE YOU CHANGED YOUR MIND ABOUT? WHY?

Science is based on evidence. What happens when the data change? How have scientific findings or arguments changed your mind?"

165 contributors; 112,600 words

That's right, there are 165 responses to the question.

Just a few of the responses that I have glanced at so far:

• Nassim Taleb (the Black Swan guy): The Irrelevance of "Probability"
• Danny Hillis: Try the Experiment Yourself
• David Gelernter: Users Are Not Reactionary After All
• Kai Krause: Software is merely a Performance Art
• Jaron Lanier: Virtual reality for the treatment of psychological disorders such as post-traumatic stress disorder
• Tim O'Reilly: Skeptical of the term "social software"
• Clay Shirky: Compatibility of Religion and Science

Personally, I do not have much of an answer to this question yet, even after reading these few responses. In general, my thinking evolves incrementally over time. I am having trouble recalling very many "Aha!" moments where I had a clear change of mind rather than that I had evolved a somewhat different perspective as I gradually incorporated new information into my thinking. Something to think about. And that is the whole point of Edge questions.

-- Jack Krupansky

10 (more) marketing trends to watch for 2008

In Ad Age, Richard Laermer, CEO, RLM PR, and author of "Punk Marketing: Get Off Your Ass and Join the Revolution" and "2011: TrendSpotting for the Next Decade", offers his list of advertising and marketing trends that are worth watching in 2008, entitled "10 (More) Trends for 2008." Read the details there, but here are the "teasers" for the trends:

1. GEEKS ARE OUT; SORRY GUYS!
2. NO PRESS IS GOOD PRESS
3. BANDWAGONING BECOMES THE RAGE
4. PEOPLE STOP BITCHING ABOUT TRAVEL
5. BAIT-AND-SWITCH COMES OUT OF CLOSET
6. AMERICAN MEDICAL ASSOCIATION DEFINES "COMPUTER DEATH" AS ACTUAL ILLNESS
7. GEORGE W. BUSH ASKS AMERICANS TO STOP DEMEANING HIM
8. AND THEY ALL LIVED HOLLYWOOD AFTER EVER
9. GOT TO START USING PLAIN OLD SOAP AGAIN
10. VONNEGUT GETS THE LAST WORD (DIDN'T HE ALWAYS?)

I'll detail one of them here just to give you a feel for their depth:

VONNEGUT GETS THE LAST WORD (DIDN'T HE ALWAYS?)
A couple of months before he died, fantastical thinker Kurt Vonnegut put forth an idea platform-ready for 2008 voters: he lobbied for the next president to install in his cabinet a secretary of the future. "It would help us live a more sustainable life, not pollute a place where generations in the future have to live," said he.

-- Jack Krupansky

Micro Trends That Will Affect America in 2008

In Ad Age, Mark Penn and Kinney Zalesne, authors of the book "Microtrends: The Small Forces Behind Tomorrow's Big Changes", offer a list of demographic and other "micro" trends in America entitled "Micro Trends That Will Affect America in 2008." Read the details there, but here are the "teasers" for the micro trends:

1. WORKING RETIRED
2. SUN HATERS
3. NEGLECTED DADS
4. CAR-BUYING SOCCER MOMS
5. SEX-RATIO SINGLES
6. OFFICE ROMANCERS
7. THIRTY-WINKERS
8. SOUTHPAWS UNBOUND
9. PROTESTANT HISPANICS
10. WEB TO WEDDING

I'll give one of them here just to give you a feel for their depth:

SEX-RATIO SINGLES
Single women are on the rise and now make up the second-largest group of homebuyers in America, just behind married couples. Watch for their interests -- in everything from home security to vacations -- to get some attention.

-- Jack Krupansky

Marketing trends to watch for 2008

On Ad Age, Bob Liodice, President-CEO, ANA, offers a list of advertising and marketing trends that are worth watching in 2008, entitled "Trends to Watch in 2008." Read the details there, but here are the "teasers" for the trends:

1. MARKETERS HIT A ROUGH PATCH
2. INNOVATION AND CREATIVITY RULE
3. GET SERIOUS ABOUT ACCOUNTABILITY
4. DIGITAL, DIGITAL, DIGITAL (AND PORTABLE TOO)
5. THE 'BRAND SWARM'
6. GETTING COMPENSATION RIGHT (PLEASE)
7. NEUROLOGICAL MARKET RESEARCH
8. EMERGENCE OF THE 'RENAISSANCE MARKETER'
9. THE POWER OF STRATEGIC ALIGNMENT
10. PRIVACY, PRIVACY, PRIVACY

I'll detail one of them here just to give you a feel for their depth:

THE 'BRAND SWARM'
Marketers will move decidedly in the direction of DDB CEO Chuck Brymer's "swarm theory" -- the notion that people and their opinions coalesce to form critical forces that massively influence marketplace ideas and concepts. "Swarm theory" will elevate social networking to new levels, confirming the immense impact that consumers have on each another. Marketers that embrace this trend can form consumer brand "advocates" and drive brand loyalty and trust to new heights -- if done responsibly.

-- Jack Krupansky

New Ad Units for Google AdSense delayed but working

Even now, only three of my seven blogs are showing their new Google AdSense ad units as active, but that is progress.

The good news is that I am also seeing page views for several of my blogs that had not shown up for more than a month. That is reasonably good progress. Whether I see revenue pick up again remains to be seen.

-- Jack Krupansky

New Ad Units for Google AdSense

The Google AdSense program for publishers has added the ability to create and manage Ad Units, which maintain most of the ad parameters on the Google server so that you can edit the parameters without having to edit the ad script code on your blog or web pages.

In theory, the old ad script code should continue to work fine, but I have seen my AdSense ad revenue plummet over the past few months and am not even seeing any page views for most of my blogs. I only heard about the new Ad Unit feature when I was poking around in AdSense to try to figure out what might be going on.

Technically they are called "AdSense units", and I also see reference to them as "slots", but I will refer to them as "Ad Units."

I am in the process of converting all seven of my Google Blogger blog templates to use the new Ad Unit code script snippet. It takes a while after creating a new Ad Unit before it is fully active on the Google server. So far, two of my seven ad units are active. AdSense says it takes up to 10 minutes to activate, but it seems more like 20 minutes oe even longer. And, I get this full-page "Google Server Error" error page display that tells me to wait 30 minutes before retrying, shortly before the Ad Unit does become active.

I created a separate Ad Unit for each of my blogs and each has a distinct channel for ad tracking.

I am not sure if any of this will fix the problem that was causing most of my blogs not to be counted even for page views, but it was an appropriate cleanup step anyway.

-- Jack Krupansky

Lessons from 2007

Lately I have found a lot of Robert Scoble's blog posts to be somewhat off-key in various ways, but he just came out with a great 2007 year-end summary entitled "What I've learned in 2007." Actually, I think it may be his best post ever. It is rather long, but proves that length is not a factor since he structures it as a list of crisp, succinct numbered bullet points, which is precisely my kind of preferred style.

One of his lessons that made me stop and think was this one:

39. Big mistake? Not spending more time working on posts. The ones where I thought about the post for hours turned out great. The ones I banged out really fast without thinking too much? They are the stupid ones.

I am not sure I totally agree with it, but I at least half-agree with it. It is clearly true for this specific post of his which clearly embodies a lot of deep, reflective thinking. An example of the negative side of his point is his recent post entitled "Why isn't Scoble against 'thought crimes bill?'" which was clearly just a knee-jerk reaction and off-the-cuff response. I am sure he could have turned it into a great post with some real insight and social value with just a brief amount of reflection. OTOH, he might have been better advised to steer clear of political debates on a technology blog. Either way, a post that is "banged out really fast without thinking too much" is probably worse than no post at all, unless you happened to believe in the old adage that "No PR is bad PR", which I do. The beauty of blogs is that you can experiment and all will be forgiven.

I had half-hoped that one of the benefits of technologies such as Twitter and Facebook would be to be magnets for all the off-the-cuff casual comments and that stand-alone blogs would continue to be refined and turn into more of a true "craft" rather than endless cruft. Maybe next year. Hopefully Robert will focus on more of his good posts and fewer of his "bad" posts.

Now... I suppose I should be working on my own list of lessons from 2007. Sigh.

-- Jack Krupansky