Recently, I have encountered an interesting paper (2006) about Chubby – Google’s (Paxos based) distributed lock service.
I was especially amazed by the observations made on the Google engineering capabilities and mindset inside a “formal” research publication.

Although one can easily get into a cynical state of mind reading this paper… I feel that this “pragmatic view” which combines a deep architectural and algorithmic know-how with keen understanding of the social factor in software development is exactly the key to create legendary software.

Anyway, very well written – highly recommended reading…

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***

“Our developers sometimes do not plan for high availability in the way one would wish. Often their systems start as prototypes with little load and loose availability guarantees; invariably the code has not been specially structured for use with a consensus protocol. As the service matures and gains clients, availability becomes more important; replication and primary election are then added to an existing design.”

“Developers are often unable to predict how their services will be used in the future, and how use will grow.  A module written by one team may be reused a year later by another team with disastrous results … Other developers may be less aware of the cost of an RPC.”

“Despite attempts at education, our developers regularly write loops that retry indefinitely when a file is not present, or poll a file by opening it and closing it repeatedly when one might expect they would open the file just once.”

“Developers rarely consider availability. We find that our developers rarely think about failure probabilities.“

“Developers also fail to appreciate the difference between a service being up, and that service being available to their applications.“

“Unfortunately, many developers chose to crash their applications on receiving [a failover] event, thus decreasing the availability of their systems substantially”

Fashionably late*, Google Search’s

Simply click on the checkbox next to SearchWiki and you will “Hide the ability to share, promote, remove, comment, or add your own results”

All good now :)

* Friendly reminder: Marissa Mayer promised that Google Search Wiki would soon have a toggle button that allow people to turn it off (“early Q1.”/2009)

Google, Yahoo, Ask and Lycos have released* their top search terms for the past year (2008) and I have aggregated it to your convenience in one happy table below.

I don’t have anything smart to say about it but I did manage to pull out five intriguing  insights.

My Five Cents:

• As done last year, it seems like Y!  have removed all the  navigational queries from their report (I wonder why ;)

• “Poker” is the “Top Search Term Of The Year”  for for the 3rd consecutive year on Lycos… (what is Lycos? :)

• Though she didn’t make it to the White House, US vice-presidential candidate Sarah Palin captured the zeitgeist of internet users in 2008 while Obama in the 6th place.

• IMHO, Ask.com is just being too honest in their report – 50% of Ask search terms are navigational queries and the rest are boring.

• Britney Spears has been the most popular search term at Yahoo for seven of the past eight years! 

Top Search Terms | 2008

Update (18 Dec. 2008): Top 10 search queries that people used on Delicious in 2008 are: news, blogs, reference, wiki, restaurants, hotels, css, web 2.0, artists, music… I think it is loud-and-clear that the biggest bookmarking site isn’t fulfilling its search potential (!)

____________________

* Note:  Microsot (Live) didn’t released the updated list until now and AOL didn’t break out overall terms so wasn’t included here.

This post is a weird collection of three Internet Conspiracies from the last 24 hours.

Note: I do realize that this post is a creepy testimony to the fact that I might be building a search engine and reading too many blogs and the affect it might have on my sense-of-judgment…

Conspiracy I (Google thinks Facebook is a dangerous Phishing Site)

Google Chrome browsers around the world have claimed today that Facebook is a dangerous Phishing Site (read more on Facebook Developer Forums)

Conspiracy II (Searching for a compatitor ’search engine’ with Yahoo)

Searching ‘Google’ in Yahoo, results with suggestion to use Yahoo search: “You could go to Google. Or you could stay here and get straight to your answers.” (it also works with ‘ask’, ‘aol’ and ‘live’ :)

Conspiracy III (Apple Anti-Virus or not)

Two weeks ago, Apple updated a technical note on its Support Web site that says:
“Apple encourages the widespread use of multiple antivirus utilities so that virus programmers have more than one application to circumvent, thus making the whole virus writing process more difficult.”

Yesterday, Apple removed the KnowledgeBase article from its support site (KBase Article HT2550 now points to a bare error page – see below) and Apple spokesperson Bill Evans explained:

“We have removed the KnowledgeBase article because it was old and inaccurate… The Mac is designed with built-in technologies that provide protection against malicious software and security threats right out of the box… However, since no system can be 100 percent immune from every threat, running antivirus software may offer additional protection.”

…

“You can always start a weekend experiment but you can never know how it will end”  (Moti Karmona, 2008 ;)

I was very curios and wanted to take the GAE for a quick test drive but the Google documentation have few inaccuracies and isn’t sufficient if it is your first Python encounter so I lost three hours of precious beauty sleep and compiled this blog-post-capsule for future generations.

How to setup your development environment to work with Google-App-Engine 

• Get yourself a GAE Account  
• Install Google App Engine SDK, Python and Eclipse.
• Install the PyDev Eclipse extension
  • Help –> Software Updates –> Available Software –> Add the http://pydev.sourceforge.net/updates site –> Install it –> Restart Eclipse
  • Configure Eclipse to use the Python interpreter by navigating to Window –> Preferences –> PyDev –> Interpreter –> Python –> New and select the location of python.exe on your system
• Create your first project
  • File –> New –> PyDev project and click next –> Name your project and make sure Python 2.5 is selected.  
  • Add necessary App Engine libraries to your project | Right click your Project folder –> Properties –> PyDev – PythonPath (see image below)
    • C:\Program Files\Google\google_appengine
    • C:\Program Files\Google\google_appengine\lib\django
    • C:\Program Files\Google\google_appengine\lib\webob
    • C:\Program Files\Google\google_appengine\lib\yaml\lib

• Create your project files (e.g. GAE examples)
  • app.yaml  
  • Your first python file 
• Test your Local application
  • Update your Run configuration (see image below)
    • Change the Main Module field to C:\Program Files\Google\google appengine\dev_appserver.py
    • On the Arguments tab, type in “${project_loc}/src” 
    • Name this configuration and click apply and click run.
  • Open http://localhost:8080 with your browser and enojoy your stupid app :)

• Upload your code to Google App Engine
  • Open a command prompt and navigate to your application directory.
  • Type appcfg.py –email=yourgmailaccount@gmail.com update src\ 
  • To publish code within Eclipse, open Run –> Run configurations and make a copy of your previous run configuration.  Change the Main Module to appcfg.py, and change the argument tab to –email=yourgmailaccount@gmail.com update ”${project_loc}/src”
  • Your code is now live! Test it by going to http://yourapp.appspot.com

Enjoy :)

P.S. More about the framework in future posts.

With Google SearchWiki, signed-in Google users can now customize their search experience by re-ranking, deleting, adding and commenting on search results. 

So…

• The re-ranking changes you make are private and only affect your own searches. 
• Your comments are visible to the public 

Random thoughts on Google Search Wiki:

• You need to be very brave to change usability patterns in your world-leading-search-cash-cow (!)
• Why wasn’t it tested as yet another interesting Google Lab project?
• The arrows “soup” is really too much for the lonely-searcher –> way too many arrows if all you wanted is just search.
• The comments I saw until now are mainly spam or not interesting.
• The most important feature in Search Wiki is a way to turn it off but it is still missing…
• It is a good time to change your default search engine ;)
• Is it only me or Search Wiki have the lively smell all over it?

I must be missing something since the Google guys are very far from being stupid (to say the least) and it will be a very interesting to see if Google will change the search experience yet again with this move.

Update (10 Dec. 2008) : Marissa Mayer promised that Google Search Wiki would soon have a toggle button that allow people to turn it off (“early Q1.”) – I can’t wait… :)

From: Developer Day <developerday.il@gmail.com>
Date: Mon, Nov 3, 2008 at 11:00 AM
Subject: PLEASE READ: Unauthorized network activity at Google Developer Day
To: ####@#####.com

Dear attendee,

First of all thanks for attending Google Developer Day yesterday, we hope you found it useful. Unfortunately, we need to let you know about an incident which took place during the conference which you may need to take precautionary action on.

We identified unauthorised activity on the public wired Ethernet network which was provided by the convention centre for conference attendees to access the Internet. This may have affected a limited number of  attendees accessing websites and online applications through the wired Ethernet connection. We have no evidence so far to suggest that the wireless network also provided at the event, and which was used by most attendees, was affected.

Due to the unauthorised activity, there is a chance that if you used the wired network, any user name and password entered to access a website may have been put at risk. When trying to access a secure website (a website using https), you may have received an alert indicating that the page had an invalid security certificate. In any case, we advise users as a precaution to change the passwords for any websites or services they accessed through the wired connection during the conference.

We’re really sorry that this has happened but we believe that the vast majority of attendees won’t have been affected by this incident. In the meantime, we look forward to seeing you at future events very soon.

The Google Developer Day Team

early voting results 2008

poll results obama vs mccain

free stuff for voting // a little suprising… ?! :)

The 1st  modern SSD was developed by StorageTek in 1978 (which was acquired 27 years later by Sun for US$4.1 billion)

Google plan to use Intel’s SSD in Production Search Systems in Q2 2008 – The company’s adoption of solid state drives will save energy, speed search, and potentially lead to a shortage of 16-GB and 32-GB NAND flash chips…

Prices for SSD drives are expected to halve every ~9 months (!!!)

In the future everything will be on SSD ;)

We have bought a simple OCZ 128GB – SATA II 2.5 SSD for benchmarking and we are not too happy with it yet.

…

My pragmatic ROIDB (ROI Driven Blogging) approach brought the Google-Translate-Widget to the left pane of my blog with ~20 minute copy-paste work (this post included).

I almost tripled my blog exposure from 350 Million (English) Internet users to 1 Billion* internet users (85% of the world internet users) and I also hit the pareto princple (80/20) jackpot on the way :-)

But… I can’t really use the “I don’t have enough traffic due to I18N readiness” excuse anymore and the 3 people that allegedly asked for I18N support only wanted to “help” with my desist Nigerian cousin will arrangement so I am not so sure it was worth it after all…

Feel free to get a taste of this I18N perfection using my new left-pane state-of-the-art translator widget and with my personal favorite Spanish Blog Flavor

—

* 365 English + 184 Chinese (!!!) + 101 Spanish + 86 Japanese + 59 French + 58 German + 47 Portuguese + 34 Korean + 31 Italian + 28 Arabic = ~1 Billion Internet users (Based on internetworldstats.com statistics)