Recently, I have encountered an interesting paper (2006) about Chubby – Google’s (Paxos based) distributed lock service.
I was especially amazed by the observations made on the Google engineering capabilities and mindset inside a “formal” research publication.

Although one can easily get into a cynical state of mind reading this paper… I feel that this “pragmatic view” which combines a deep architectural and algorithmic know-how with keen understanding of the social factor in software development is exactly the key to create legendary software.

Anyway, very well written – highly recommended reading…

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***

“Our developers sometimes do not plan for high availability in the way one would wish. Often their systems start as prototypes with little load and loose availability guarantees; invariably the code has not been specially structured for use with a consensus protocol. As the service matures and gains clients, availability becomes more important; replication and primary election are then added to an existing design.”

“Developers are often unable to predict how their services will be used in the future, and how use will grow.  A module written by one team may be reused a year later by another team with disastrous results … Other developers may be less aware of the cost of an RPC.”

“Despite attempts at education, our developers regularly write loops that retry indefinitely when a file is not present, or poll a file by opening it and closing it repeatedly when one might expect they would open the file just once.”

“Developers rarely consider availability. We find that our developers rarely think about failure probabilities.“

“Developers also fail to appreciate the difference between a service being up, and that service being available to their applications.“

“Unfortunately, many developers chose to crash their applications on receiving [a failover] event, thus decreasing the availability of their systems substantially”

“imagine that your mother-in-law believed her daughter could do better… and then imagine that she moved in with you… that’s what Scrum is like”

Think about it…

Assuming we shouldn’t aim to completely avoid all errors in software development (since this is an inherent part of any human creation) but rather to spot them as quickly as possible before they become real problems.

And… since Scrum is indeed a very good “tool” to bring the problems in-your-face without any mercy in a daily manner.

So without even getting into the continuous improvement possibilities with mother-in-laws, I really liked the Mother-In-Law allegory :)

By the way, with great anticipation I have proudly joined the Haiku contest @ the famous Ktorium – Wish me luck! :)

Justin Kruger & David Dunning have tested and verified the following predictions:

• Incompetent individuals tend to overestimate their own ability and performance 
• Incompetent individuals fail to recognize genuine skill in others (it takes one to know one ;)
• Incompetent individuals fail to recognize the extremity of their inadequacy (“One puzzling aspect of our results is how the incompetent fail, through life experience, to learn that they are unskilled”)
• If they can be trained to substantially improve their own skill level, these individuals can recognize and acknowledge their own previous lack of skill (There is still some hope)

Do you have the confidence that this post isn’t about you? 

Think again… (!!!) - “ignorance more frequently begets confidence than does knowledge” (Charles Darwin)

Do you think this post is about you?

Might be considered vain but interesting enough, the same research have shown that the top performers tended to underestimate their own performance compared to their peers (see chart below). 

So… if you find this post boring, obscure, stupid, annoying, poorly written or inappropriate than please keep in mind it isn’t something I have committed knowingly.

When “The Moscow Cats Theater” came to New York, the Russian clown Yuri Kuklachev was interviewed:  “the secret of training them is realizing that you can’t force cats to do anything [...] If the cat likes to sit you can’t force her to do anything else [...] Each cat likes to do her own trick [...] Maruska is the only one who does the handstand. I find the cat and see what they like to do and use that in the show [...] I have a cat now that loves to be in the water…”

– REUTERS, 2006

__________________________________________

Personally, I think that managing engineers is much more complicated than herding cats (although I didn’t have the twisted pleasure to herd a cat yet)

When you go out of your way to hire the best people around than soon enough you will find yourself herding a superior, class A, hyper-developed mutant Ligers* who are much more knowledgeable than the herder (a.k.a. you)

In this environment you have to learn to simply trust your people (although this is not simple at all :), mark the vision, let them loose and only help to get rid of the stones in their way (this concept was best described as the Open Kimono** policy in Peopleware)

Well…. Managing the Delver Engineers is like Herding Legendary Ligers and you need to make a superior effort to see what these ligers “likes to do” and run fast enough to set the Vision and move the rocks out of the way.

__________________________________________

* The Liger, is a (huge) hybrid cross between a male lion and a female tiger

** Open Kimono Attitude: You take no steps to defend yourself from the people you have put in positions of trust.

By the way, The best answer I found on the origin of the term “Herding Cats” was in Google Answers

Dunbar’s Friends is my definition (and trademark ;-) to those few “real”, trusted and known people in your huge** online social network***.

—

* Primatologists have noted that, due to their highly social nature, non-human primates have to maintain personal contact with the other members of their social group, usually through grooming (and not “poking” as you might be expecting :-). The number of social group members a primate can track appears to be limited by the volume of the neocortex region of their brain.

** Did you know that Robert Scoble is following 21,060 people in Twitter, 2,992 in FriendFeed and only 71 “lousy” friends in Flickr…

*** Social Network for Dummies – Lee and Sachi LeFever (a.k.a. the CommonCraft’s family :) have created a wonderful video explaining social network in plain English.

* 7 most common Americans surnames are Smith, Johnson, Williams, Brown, Jones, Miller or Davis.

* 4 million surnames  are held by only one person.

…

* This can be very helpful when you are trying to guess someone name… with ~2,376,206 Smiths, I suggest Mr. Smith.

* This could be problematic when you are looking for Mary or James Smith and all you have is the name…

* Moti Karmona isn’t even there ;-)

In this context, last summer I wrote, coined and trademarked the Pessimistic Developer Paradigm.

Early this week I have interviewed a very interesting dude* who saw this old article hanging on my office and introduced this amazing poem by Robert Graves which I must share in this pessimistic context…

In Broken Images (by Robert Graves)

He is quick, thinking in clear images;
I am slow, thinking in broken images.

He becomes dull, trusting to his clear images;
I become sharp, mistrusting my broken images.

Trusting his images, he assumes their relevance;
Mistrusting my images, I question their relevance.

Assuming their relevance, he assumes the fact;
Questioning their relevance, I question their fact.

When the fact fails him, he questions his senses;
when the fact fails me, I approve my senses.

He continues quick and dull in his clear images;
I continue slow and sharp in my broken images.

He in a new confusion of his understanding;
I in a new understanding of my confusion.

* Thanks Tomer and Good Luck!

This unique first-person story opens a window into the mind of a 27-year-old autistic savant with Asperger’s syndrome.

Daniel is capable of incredible feats of memorization and mental calculation. Besides being able to effortlessly multiply and divide huge sums in his head with the speed and accuracy of a computer; Daniel, learned Icelandic in a single week and recited the number Pi up to the 22,514 digit, breaking the European record (3-14-2004 Pi Day)

Daniel also experiences synesthesia which is an unusual neurological syndrome that enables him to experience numbers and words as shapes, colors, textures and motions.

“I was born on January 31, 1979 — a Wednesday. I know it was a Wednesday, because the date is blue in my mind and Wednesdays are always blue, like the number 9 or the sound of loud voices arguing. I like my birth date; because of the way I’m able to visualize most of the numbers in it as smooth and round shapes, similar to pebbles on a beach. That’s because they are prime numbers: 31, 19, 197, 97, 79 and 1979 are all divisible only by themselves and 1. I can recognize every prime up to 9,973 by their “pebble-like” quality. It’s just the way my brain works.”

“The number 11 is friendly and 5 is loud, whereas 4 is both shy and quiet — it’s my favorite number, perhaps because it reminds me of myself. Some are big — 23, 667, 1,179 — while others are small: 6, 13, 581. Some are beautiful, like 333, and some are ugly, like 289″

“One of the most common questions I was asked … was: Why learn a number like pi to so many decimal places? The answer I gave then as I do now is that pi is for me an extremely beautiful and utterly unique thing. Like The Mona Lisa or a Mozart symphony, pi is its own reason for loving it.“

Daniel stated in his book that it wasn’t easy to find enough (digits) Pi in the web, so inspired from his book, I have created an online “backup”* for the first 10 million digits of Pi @ http://pi.karmona.com

3.1415926535897932384626433832795028841971693993751058209749

44592307816406286208998628034825342117067982148086513282306

64709384460955058223172535940812848111745028410270193852110

…

…

…
* During August 1995, Dr.Takahashi and Dr. Y.Kanada have managed to calculate pi up to 4,294,960,000 decimal digits (current world record) using a supercomputer at the University of Tokyo – The University ftp server was Daniel’s (& my backup) source…

I think that human capital is the silver bullet* for successful software projects – productivity, personalities, teamwork and group dynamics will make or break a project.

Picking the right people is maybe the most important managerial task so on your next interviews please remember** that knowledge can be easily acquired but personality is there to stay.

Don’t spend 90% of your interview time on knowledge when personality (and potential) is the real key for successful recruitment.

——————————————————–
* Although “Peopleware” have a full chapter on how there is no silver bullet… but I partially agree since I never said it will be easy to get to the human capital silver bullet…
** Also remember: Somewhere today a project is failing… and I can personally guarantee that people were somehow involved in its failure!

Green Managers – Five top common mistakes with two cent tips.

Vision???
Managing is more then juggling day-to-day tasks – Make a difference, lead to change… Construct a vision, set goals and encourage innovation.Delegation-less: “Never mind, I will do it…”
Simply start delegating like hell!

Sagemet – (Hebrew Slang, The sickness of a green officer in IDF) – You don’t let yourself be human and you fall in-love with your new title.
Remember, management title does not elicit automatic respect and obedience and just because you are the boss doesn’t mean you can’t be human – Feel free to laugh, show emotion and you can even make an occasional mistake ;-)

Mr. Know-all – You think you know everything.
Be sure you don’t know everything is maybe is maybe the most important part of getting into new managerial position. Listen to the people around you and keep an open mind.

Ooops, employees…
As a manager you must remember the three most important success factors: 1. People 2. People and surprisingly enough 3. People
Listen to your employees, take the time to know them, empower them, tell people what you want, not how to do it, don’t put policies ahead of people etc.

Good Luck!