It is amazing – bloody amazing.  Its very hard to remember what life was like before the Internet.  The amount of information and the things you can do on the Internet is mind-blowing.  Search engines are amazing too – simply by virtue of the fact that they can scan all of that information in a very very short amount of time.

Search engines are a big part of the Internet.

I think that search engines have achieved an incredible feat.  They are both incredibly incredible and yet more or less completely useless.  The problem is that the search begins when you get a search engine involved, they are what the say they are – a powerful engine for searching i.e. not find engines.

Social search is supposed to be the answer to this problem.  This is where you consult your networks for answers and recommendations.  Rather than chucking keywords into Google or Bing, you put out a tweet, or go to a forum or post a question on your Facebook.  Nice idea in theory but one that falls down unless your network of friends and their brains are equally distributed across the required range of topics and experiences (which is unlikely).

The other challenge is that its not very easy – unless you’re someone like Mike Arrington, Lady Gaga or Russell Crowe – to actually get people to respond.  In my experience the people that respond in social networks are tight connections (thanks guys).  It’s rare that a distant connection responds with an answer or piece of advice via a social network – even though you know they have answers and opinions.  Its good when they do.

It’s different in the real world. People will generally give you an answer if you ask them a direct question.  Its rare that you get a mute blank stare as a response to a question or request for advice although it has happened to me once or twice in France (I drove off).

This is probably why 80% of word of mouth still takes place offline.

What is the solution to this problem – I can’t foresee a future where I ask a question on Twitter and I receive 300,000 results in 0.24 seconds and neither would I want this.

I could probably do something with (a very) Advanced Search in Google but I am not convinced it would work that well.  Natural language processing isn’t the answer either as the real problem is the amount of dud, repetitive information on the Internet.

I’d like a search engine that trawled a limited number of trusted quality sources.  I would have to chose them, though I would be open to recommendations and lists (it would be dangerous to leave the selection process to any third party).  I should have complete control over my sources and should be able to add any that I please.  Could someone create this? Have they already?

Also, has any noticed that Google News is broken?

I doubt that most consumers of digital media are even aware of this phenomenon. The political implications are massive.

It has significant implications for the PR and marketing industries. My first thought is that it steers marketers to advertising, which would add up given that this is the primary revenue stream for the sites in question.

http://m.democracynow.org/stories/11898

According to comScore, Google still has nearly two-thirds of the U.S. search market, but dropped a fraction of a percent from 65.7% in Dec 2009 to 65.4% in Jan 2010 [source: Information Week]. While Google is still the leader in the search space, and Facebook only accounted for just under 400 million searches in January, that is a gain of 13% over December. If this trend continues, Google may have ample reason to fear Facebook.   

The Compete data shows that Facebook is the second most popular site in the US with 134m unique visiter in January 2010, ahead of Yahoo! and just behind Google.

Facebook is in the ascendancy but the company must make changes to its own Search engine if it is to become a genuine rival to Google, the vastly dominant player in the space. 

Facebook, like other social networks, represents opportunity for brands because of the simple fact that this is where a lot of the action is taking place online, as demonstrated by slews and rising traffic.  Switched on brands are  already tapping into the Facebook community.  But Facebook Search serves marketers poorly today as a place of discovery.  Check out my search results for ‘mobile handset’:

Not particularly useful.  Three users groups with 210 members between them.

No sign of the brand sponsored pages where Facebook Users can learn about new products, participate in competitions and promotions, chat with other users and potential customers, communicate directly with the company and link through to relevant pages on the Web.

The Sony Ericsson WorldPage has more than 445,ooo Fans (Sony Ericsson is a client) but the way that Facebook Search works prevents this page, which is clearly relevant to the search term, from appearing. 

This is an issue for Facebook.  

We see the same issue if we run a search for ’sneakers’:

The retailer Sole Provider Sneakers comes out on top here, simply by virtue of having the search term embedded in its company name (also the name of the Page).  Yet Sole Provider Sneakers sells a lot of Nike trainers and a search for ‘Nike’ would not have produced their Page in its results.

Facebook has to play a delicate balancing act here.  Ultimately, user activity, personal profiles, user content, sharing and discussion are the currency of social networks such as Facebook.  The company needs to avoid giving users the impression that they are being marketed to via the network, failure to do this could well become its undoing.

One way around this could be for Facebook to adopt a model similar to Google’s Universal Search, which would allow users to chose and filter the types of results that are presented to them via Facebook Search.  This could be built into the privacy settings that Facebook has been so keen to promote recently.

Interestingly, Google listed Facebook as a formal competitor for the first time in a recent 10K filing, as reported by SearchEngineLand:

Our business is characterized by rapid change and converging, as well as new and disruptive, technologies. We face formidable competition in every aspect of our business, particularly from companies that seek to connect people with information on the web and provide them with relevant advertising. We face competition from:

•   Traditional search engines, such as Yahoo! Inc. and Microsoft Corporation’s Bing.
• Vertical search engines and e-commerce sites, such as WebMD (for health queries), Kayak (travel queries), Monster.com (job queries), and Amazon.com and eBay (commerce). We compete with these sites because they, like us, are trying to attract users to their web sites to search for product or service information, and some users will navigate directly to those sites rather than go through Google.  
• Social networks, such as Facebook, Yelp, or Twitter. Some users are relying more on social networks for product or service referrals, rather than seeking information through traditional search engines. (my emphasis)

Some subtle and simple changes to Facebook Search would accelerate the trend towards Facebook and other social networks as a primary channel for Search. 

The challenge for marketers will (continue to be) to resist the tempation to sell via social networks and to engage with social networking users in a way that adds value, build relationships, earns trust and facilitates creativity and connectivity.  Facebook will have to manage the sensitivities of its users delicately but if managed well Facebook could become a natural home to Search, delivering value to users and marketers alike. 

The FT reported today that News Corp and Microsoft are in ‘discussions’.  Microsoft is determined or desperate to catch up with Google in search, the latter being streets ahead.  Microsoft has made it clear that it will invest heavily to achieve its goals. 

Could Microsoft turn the tide in search through exclusive partnerships with publishers? 

Update: Some interesting perspectives on today’s developments…  

Tom Foremski at SiliconValleyWatcher: Is Murdoch Setting Up a Bidding War for News Corp. Index?

Danny Sullivan at Search Engine Land: Why an Exclusive WSJ Deal Wouldn’t Help Bing

The updates – named Caffeine – are described by Google as “a next-generation architecture for Google’s web search“.

Thanks to my old colleague Ged Carroll and his RT for Stephen Waddington to this econsultancy.com article via Twitter.

Google describes the changes as “the first step in a process that will let us push the envelope on size, indexing speed, accuracy, comprehensiveness and other dimensions.”

Google has opened up a web developer preview to collect feedback: www2.sandbox.google.com/

Google is inviting feedback: Engineers will be reading the feedback, but we won’t have the cycles to send replies.

I ran a few searches against my name (daniel young) and found that:

• My Facebook profile result goes from 5 in old Google to 2 with Caffeine
• My FriendFeed profiles goes from 29 in old Google to 8 with Caffeine
• My Twitter profile goes from 18 in old Google to 14 with Caffeine
• This blog goes from 7 in old Google to 24 with Caffeine

Caffeine seems to be:

1. Favouring the sites that are updated more frequently
2. or, favouring social networks over other content
3. or, both of the above – since they are inter-related

The jump in FriendFeed could be attributed to the Facebook acquisition and the increased importance on real-time properties is most likely a counter strategy to the launch of Bing and Facebook’s real-time search capabilities.

Take a look at the results in Caffeine when we use the search term ‘pr agency’:

• Top result is the same in Caffeine and old Google – Punch PR
• GlobalCom PR replaces Yetis 10 at number 3 in Caffeine. Yetis 10 drops to rank 51!  Down 48 places.
• Edelman drops out of the top ten with Caffeine
• VitisPR, Brands2Life, Berkeley PR enter the top ten with Caffeine
• At the expense of Edelman, About PR.net (directory) and The Publicity Agency

My sites are fairly inconsequential but changes to the way that Google produces search results could result in lost business opportunities for organisations that have invested heavily in the Web or are reliant upon it.

The secrecy surrounding the Google algorithm is legendary but it seems odd that Google is asking the user community to tell it about the impact that the change is having on search results.

Does Google not have an obligation or duty to let organisations and individuals know what impact the changes will have on their web properties?

The changes aren’t objective – Google is making subjective decisions about the value of different types of content, sites, activity, formats and so on. These subjective judgements take on particular significance in the context of Google’s extraordinary market position – in excess of 90 per cent market share here in Australia.

This goes to the nub of an issue, which was explored in some detail at today’s AIMIA event in Sydney: ‘Advertiser’s Rights and Consumer Privacy‘.

The event focused on deceptive advertising techniques and unethical practice online.

Unfortunately for Google , it bears the brunt of much of the criticism due in part to its market dominance but also due to its lack of transparency.

The whole value chain came under scrutiny this morning – search engines (Yahoo!, Ask), aggregators, mobile carriers, social networks and digital advertising networks.

Speakers argued that there is a significant lack of accountability in the sector, which is characterised by very complex networks and inter-relationships.

Take two examples cited in today’s event, which featured Ben Edelman – an assistant professor at Harvard Business School:

1. Deceptive advertising tactics are widespread to the point of ubiquity within paid search. Section 230 of the Communications Decency Act provides immunity from liability for providers and users of an “interactive computer service” who publish information provided by others:

No provider or user of an interactive computer service shall be treated as the publisher or speaker of any information provided by another information content provider.

This means that online publishers are not responsible for the content they publish, no matter how misleading or deceptive it may be – resulting in many instances of dishonest and misleading search marketing tactics.

Search engines are, as a result, failing to take proactive action to clear out these deceptive practices.

2. Google and its partners inflate conversions to claim credit for traffic that advertisers would otherwise have received for free. Edelman outlines four examples of this here.

I don’t believe that Google is an evil enterprise but I am convinced that it has excessive market power and that this is not in the long term interests of our digital economy or the broader community. Perhaps it gets a hard time as a result of its market dominance but leaders in every field have a duty of care and obligation to do the right thing.

Google is pervasive yet untouchable. Mr. Edelman provided an example of this when he highlighted a laughable  and ironic complaints and claims process for Adwords.

The following is taken from Google Adwords Terms and Conditions for Australian customers:

10 Miscellaneous. The Agreement must be construed as if both parties jointly wrote it, governed by California law except for its conflicts of laws principles and adjudicated in Santa Clara County, California. The Agreement constitutes the entire agreement between the parties with respect to the subject matter hereof, and supersedes and replaces any other applicable agreements, terms and conditions applicable to the subject matter hereof. Any conflicting or additional terms contained in additional documents (e.g. reference to a purchase order number) or oral discussions are void. Each party shall not disclose the terms or conditions of these Terms to any third party, except to its professional advisors under a strict duty of confidentiality or as necessary to comply with a government law, rule or regulation.  Customer may grant approvals, permissions, extensions and consents by email, but any modifications by Customer to the Agreement must be made in a writing executed by both parties. Any notices to Google must be sent to Google Ireland Limited, AdWords Program, 1st & 2nd Floor, Gordon House, Barrow Street, Dublin 4, Ireland, with a copy to Legal Department, via confirmed facsimile, with a copy sent via first class or air mail or overnight courier, and are deemed given upon receipt.  Notice to Customer may be effected by sending email to the email address specified in Customer’s account, or by posting a message to Customer’s account interface, and is deemed received when sent (for email) or no more than 15 days after having been posted (for messages in Customer’s account interface).

In short, please send an email to the Google engineers if you want to provide feedback on Caffeine, the new search algorithm (but don’t expect a reply), and submit notices regarding modifications to Adwords agreements by snail mail to their office in Dublin, Ireland.

Wave will allow users to insert comments in in-line conversations, in the same way that you might made edits to an email that you have received in the body of that emails (as opposed to in your reply). The difference with Wave is that it annotates this edit with an image of the person that has commented, the date and time. These conversations are hosted as opposed to an isolated email client. It’s kinda like the Facebook Newsfeed but within selected groups of contacts (whereas the Facebook Newsfeed is exposed to all by default).

Within this shared page, users can also send instant messages to individuals – in addition to the messages that can be sent to groups. With today’s IM clients, the user types the message and then clicks ’send’. With Wave instant messaging characters are transmitted in real-time character by character – so as the recipient you can see the message being typed as it is being typed – errors and backspaces and all (presumably).

Why? Google says that with today’s IM services you spend half your time looking at the box looking at the ‘Daniel is typing’ message. This ‘live transmission’ means that readers are reading and responding at the same time. You can start formulating your reply before the other party has finished theres. A check box allows users to switch off this feature.

The Engineering time that developed Wave is lead by two brothers (who also brought us Google Maps) and the work for Wave was completed in Sydney.

New parties can be added to this ongoing conversation by drag and drop. With email this is done by adding the new entrant to the cc: or to: field, which has a downside. This new participant in the conversation won’t be party to the all of the previous back and forth in the email conversation. Wave conversations are hosted, new participants are added to the hosted conversation and the history of that conversation is exposed to the new entrant – kinda like a wiki, which has been updated by two people and then shared with a third for their involvement.

The new entrant is ‘on the Wave’. A playback feature, allows the new entrant to proceed step by step through the history of the conversation, so that they can see how the conversation has developed.

With three participants in the conversation users have the access to send private messages to individuals.

Attachments can be dragged ‘onto the Wave’ and shared with the other participants. Apparently, html5 can’t yet support the dragging of images from the desktop to the browswer. Google is looking to add this to the standard but Gears can allow this to happen.

Google was keen to show off the contextual spell checker in Wave, which auto corrects. The tool has ‘enough confidence’ to auto correct the following: ‘icland is an icland’ (Iceland is an Island).  These principles are applied to URLs which become hyperlinks when they become complete. Wave includes a real-time language translation engine.

A Google Search extension allows users to perform search and view results in the message field, which can then be inserted (links, images, maps) with a single click.

Waves can be embedded in other forms of online media, such as blogs. And third parties – not involved in the original conversation – can respond by sending a message to the original participants.  Google has been working on this for 1 and a half years.

If successful, this will introduce a wide range of new capabilities that will further strengthen Google’s market position as well as opposing vast amounts of user information to Google.  Google stressed the openess of the system (i.e. allowing for developers extensions and add-ons).

In the same edition WARC reported on a 15% reduction in global advertising revenue – across the board i.e. digital and traditional media.  

The ‘measurability’ of digital marketing remains one of the key drivers for investment in the Web by marketers and this will continue to be the case while the industry matures and develops a method for measuring and tracking engagement and influence (i.e. qualitiative metrics).

The traditional media sector is obviously already suffering at the hands of digital. To the digital advertising sector – and particularly the massively dominant Google - the measurability of digitial advertising is a major strategic advantage. 

As competition for marketing budgets increase, I think we can expect to see companies like Google and others making more and more user data available in order to maximise their share of the total overall investment. At no cost and at the touch of a button, these organisations can enhance their strategic advantage over traditional media.  

There is a strong lobby arguing that this strategic advantage comes at the expense of consumer privacy but Google has been commended in parts for its approach.  

Could the launch of Google Voice be a strategy designed to overshadow media interest in targetted advertising.

The New York times provides a guide to Google’s privacy controls here. Industry standard story < there and an interesting counter view for Gawker here.

I think of this as a type of Content Application Integration and thikn that we are likely to see many more examples of this.

New media is driving increases in the usage of audio, video and imagery (rich media) and so it makes a lot of sense to build features into creative applications that offer search functionality, access to the Web and easy one-click insertion. See my earlier post on a related topic.

CAI (CAI) offers many potential applications… imagine, for example, being able to click on any word in a working document (a Word doc say, or PowerPoint) and going straight to the Wikipedia page for that word or opting to open a search in a Web browser on that term. This would be a variation on the linking strategy that underpins Web collaboration and sharing.

These functional capabilities are available today but CAI would make the integration between the productivity applications and content engines much tighter – furthering enabling the development and creation of multi-media content.

It would seem that Google has a much stronger position in this space as it is suited more to Web based applications as opposed to on-premise apps.

It’s not the first time that Microsoft has tried to find ways to work with Yahoo!

There are obvious synergies and opportunities for Microsoft. The deal would give Microsoft’s advertising sales division – which has been recently bolstered by the acquisition of aQuantive and other small online ad sales players – a vast amount of well trafficed Web property to sell. Online advertising sales are expected to double in the next three years to US$80bn.

The deal would provide an established channel for a Web based version of Office to compete with Google Docs and there would be value – potentially – in combining the Hotmail and Yahoo! email constituencies.

The following section is from Yahoo! Finance:

Yahoo would give Microsoft dominance in Web banner ads used by corporate brand advertisers. It also attracts more than 500 million people monthly to sites devoted to news, finance and sports, and Yahoo Mail is the No. 1 consumer e-mail service.

On the topic of email, I must say that I wasn’t impressed by the new Yahoo! mail service and have reverted to the ‘Classic’ product.

On the down side, you’d have to wonder how easy it would be to align the diverse cultures of these two companies and how keen the Yahoo! employees would be to work for the dark empire Microsoft (Will the talent leave? Hello Google!). Also, Microsoft’s cultural leaning towards software release cycles wouldn’t represent a good fit with the dotcom ethos where innovation and development is a part of every working day.

Search is clearly a massive lead generator for an Internet company and Microsoft pursues this market aggressively and keenly. One wonders if two companies that have struggled in search, or at least have failed to take the lead, would represent an effective competitor to Google anytime soon.

If the bid is successful, both companies will take their eye off of the ball while they deal with the integration of the two companies, providing Google with an opportunity to extend its lead. Google already has a 75% market share in search advertising compared to Yahoo!’s 9%.