Karmona Pragmatic Blog

Pragmatic Software Management, Internet Trends, Life and more…

Karmona Pragmatic Blog

Chubby Hubby

August 10th, 2009 by Moti Karmona | מוטי קרמונה · No Comments

Chubby Hubby

Recently, I have encountered an interesting paper (2006) about Chubby – Google’s (Paxos based) distributed lock service.
I was especially amazed by the observations made on the Google engineering capabilities and mindset inside a “formal” research publication.

Although one can easily get into a cynical state of mind reading this paper… I feel that this “pragmatic view” which combines a deep architectural and algorithmic know-how with keen understanding of the social factor in software development is exactly the key to create legendary software.

Anyway, very well written – highly recommended reading…

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***

“Our developers sometimes do not plan for high availability in the way one would wish. Often their systems start as prototypes with little load and loose availability guarantees; invariably the code has not been specially structured for use with a consensus protocol. As the service matures and gains clients, availability becomes more important; replication and primary election are then added to an existing design.”

“Developers are often unable to predict how their services will be used in the future, and how use will grow.  A module written by one team may be reused a year later by another team with disastrous results … Other developers may be less aware of the cost of an RPC.”

Despite attempts at education, our developers regularly write loops that retry indefinitely when a file is not present, or poll a file by opening it and closing it repeatedly when one might expect they would open the file just once.”

Developers rarely consider availability. We find that our developers rarely think about failure probabilities.

Developers also fail to appreciate the difference between a service being up, and that service being available to their applications.

“Unfortunately, many developers chose to crash their applications on receiving [a failover] event, thus decreasing the availability of their systems substantially”

→ No CommentsTags: Development · Google · People · Software

Google-App-Engine Development Environment

November 21st, 2008 by Moti Karmona | מוטי קרמונה · 4 Comments

“You can always start a weekend experiment but you can never know how it will end”  (Moti Karmona, 2008 ;)

I was very curios and wanted to take the GAE for a quick test drive but the Google documentation have few inaccuracies and isn’t sufficient if it is your first Python encounter so I lost three hours of precious beauty sleep and compiled this blog-post-capsule for future generations.

How to setup your development environment to work with Google-App-Engine 

  • Get yourself a GAE Account  
  • Install Google App Engine SDK, Python and Eclipse.
  • Install the PyDev Eclipse extension
    • Help –> Software Updates –> Available Software –> Add the http://pydev.sourceforge.net/updates site –> Install it –> Restart Eclipse
    • Configure Eclipse to use the Python interpreter by navigating to Window –> Preferences –> PyDev –> Interpreter –> Python –> New and select the location of python.exe on your system
  • Create your first project
    • File –> New –> PyDev project and click next –> Name your project and make sure Python 2.5 is selected.  
    • Add necessary App Engine libraries to your project | Right click your Project folder –> Properties –> PyDev – PythonPath (see image below)
      • C:\Program Files\Google\google_appengine
      • C:\Program Files\Google\google_appengine\lib\django
      • C:\Program Files\Google\google_appengine\lib\webob
      • C:\Program Files\Google\google_appengine\lib\yaml\lib

  • Create your project files (e.g. GAE examples)
    • app.yaml  
    • Your first python file 
  • Test your Local application
    • Update your Run configuration (see image below)
      • Change the Main Module field to C:\Program Files\Google\google appengine\dev_appserver.py
      • On the Arguments tab, type in “${project_loc}/src” 
      • Name this configuration and click apply and click run.
    • Open http://localhost:8080 with your browser and enojoy your stupid app :)

  • Upload your code to Google App Engine
    • Open a command prompt and navigate to your application directory.
    • Type appcfg.py –email=yourgmailaccount@gmail.com update src\ 
    • To publish code within Eclipse, open Run –> Run configurations and make a copy of your previous run configuration.  Change the Main Module to appcfg.py, and change the argument tab to –email=yourgmailaccount@gmail.com update “${project_loc}/src”
    • Your code is now live! Test it by going to http://yourapp.appspot.com

Enjoy :)

 

 

P.S. More about the framework in future posts.

→ 4 CommentsTags: Development · Google · Internet · Software

Wasting Time on Blog Performance

November 2nd, 2008 by Moti Karmona | מוטי קרמונה · 2 Comments

Last weekend I was wasting time on my Blog performance and all I got is 10 sec. and this lousy post.

Quick profiling with FireBug and YSlow FireFox extensions have done great wonders with my amazingly slow, not-really-that-interesting, hosted (webhost4life) WordPress blog.

Main action items:

  • I have removed the sitemeter performance penalty widget
  • I have installed the WordPress Super Cache Plug-in which wasn’t that simple with my webhost4life hosting limitations.
  • I have removed couple of images and reduce the blog top image size from 80k to 5k (!!!)

Results:

  • Main blog page returns after ~1 sec.
  • I am still getting a lousy grade with the YSlow tests (62) but it seems like the YSlow blog post announcement have failed too… (57 :)
  • Post pages are still slow (~2-4 sec.) but I know what I have to do the next time more than 1 million readers will complain…

Have fun!

→ 2 CommentsTags: Blogging · Development · Internet · Tools · WordPress

Yahoo Open Strategy

October 28th, 2008 by Moti Karmona | מוטי קרמונה · 1 Comment

Yahoo have released the Y!OS (Yahoo Open Strategy) 1.0 platform.

This is a cool set of simple APIs that can give you access to everything you ever wanted in Y! but was afraid to ask for…

Yahoo! Social Platform (YSP)
// The Yahoo Social Platform is a set of RESTful APIs for Profiles, Connections, Updates, Contacts and Status.

Yahoo! Query Language (YQL)
// The Yahoo Query Language is a web service that functions much like SQL (see example below)

OAuth Authentication
// OAuth is the authentication and authorization standard Yahoo has decided to use when giving third parties access to Yahoo user data.

Yahoo! Applications Platform (YAP)
// Currently very limited and in a restricted sandbox.

________________________________

Example: How to use YQL APIs to access MyBlogLog profiles?

Simply ask for all the community members of MyBlogLog community with this YQL:

select * from mybloglog.members.find where community_id in (select id from mybloglog.community.find where name=”Karmona Pragmatic Blog”)

And once you have the IDs you can ask for my personal profile by:

select * from mybloglog.member where member_id =”2008070609482910″

Well… together with the existing BOSS API, this set of APIs is a powerful enablers to the Y! development network and I am sure some cool stuff are going to emerge from this innovative move…

Amazing!!!

________________________________

* You can have more YQL experiments using the YQL Console

** Boss Hack Day is coming to Tel-Aviv | November 6, 2008 @ Feature (!!!)

→ 1 CommentTags: Development · Disruptive Technology · Internet · Search · Software

Base64 Encode – Decode Online Widget

October 20th, 2008 by Moti Karmona | מוטי קרמונה · 5 Comments

I am very “proud” to introduce the ultimate geek widget: Base 64 Encode / Decode Online Widget

Q. Where can I see this dark magic?

A. Here… :)

Q. How can I add this cool Base64 widget to my blog?

A. Simply copy-paste this little script:
___________________________________________

<script type=”text/javascript” src=”http://blog.karmona.com/base64widget.js” ></script>

___________________________________________

Q. Does this blog widget support ALL blog platforms?

A. Sure… (including dasBlog :)

Please contact me if you have any issues / questions / suggestions,

Have fun!

→ 5 CommentsTags: Blogging · Delver · Development · Tools