Memory Based Architecture For Enterprise Application – Introduction

We had this architecture discussion in one of the technical meetings in company recently and I was assigned to share all details on Memory Based Architecture. Sharing details from those sessions.

Memory, changing philosophy with enterprise applications and  Memory Based Architecture:

The main memory is high bandwidth and low latency component that can match performance of the processor in the computer. The bandwidth of main memory is around few GB per second as oppose to disk which is around hundred MB per second. The latency of main memory is in nanoseconds range where as that of disk is in milliseconds range. Traditionally main memory was considered as expensive resource and therefore it was scarcely used. However this perception that RAM is expensive component is now changing due to sharp drop in prices over past several years. Same time enterprise applications require more scalable and performance oriented use of those each chunk of available physical memory. Today they have enormous amount of such main memory cheaply available. Many applications are using memory in Gigabytes and Terabytes. The main memory empowers application architectures to achieve linear scalability and high performance. These qualities are extremely important to the modern enterprise applications for delivering guaranteed high performance under intensive and unpredictable workload.

As enterprises are using more memory, software vendors have flooded the market with several types of memory based products in order to size this new business opportunity. These products are targeted towards supporting various business use cases and architectural scenarios. This series is intended to introduce various memory based product categories along with business uses and architectural scenarios supported by them. 

When we think of any memory based products then high performance is the first thing that comes to our mind. Yes, high performance is primary reason why memory based products are used, but it is not the ‘only reason’. Many a times they are deployed to reduce IO operations over network or address the high latency issues with disk based products like databases. Typically with N Tier Architecture, properly design application code can easily scale out by adding more application servers. However the main scalability barrier is disk based database which is centrally access by all the clustered application servers. Here memory based products are typically deployed to overcome scalability bottleneck pose by disk based database and make application servers more scalable. Thus following can be considered as primary scenarios for any memory based product.

– Improve application performance
– Reduce network & disk IO Operations
– Overcome scalability barriers & make application servers more scalable.

The memory based products can be broadly classified as Caching(Standalone & Distributed Caching), In Memory Data Grid (IMDG), Main Memory Database (MMDB) and Application Platforms that enables Space Based Architecture and covered in great details under this series.

Passing command line to jetty through maven plugin

Passing command line to jetty through maven plugin is easy three step process.

mvn jetty:run -DcommandLineName=value
<plugin>
       <groupId>org.mortbay.jetty</groupId>
       <artifactId>maven-jetty-plugin</artifactId>
       <version>6.1.10</version>
       <configuration>
          	<systemProperties>
      			<systemProperty>
       			<name>arg1</name>
       			<value>${commandLineName}</value>
     		</systemProperty>
    	</systemProperties>
        </configuration>
</plugin>

With Jetty context, set parameter can be accessible from anywhere in code as given below.

System.getProperty("arg1");

Double Checked Locking With Thread

While implementing a circular non-blocking queue, we observed a pattern which could boost a performance.

When you have condition based thread safe block, it is always advisable to check same condition before and after lock.

Before lock will avoid threads to get locked unnecessarily and after lock will ensure that any state change within synchronized block does not violate protocol.

In detail, for example, there are ‘n’ threads entering into this piece of code and say queue(ConcurrentLinkedQueue) is full, If queue is not full we should not lock threads, hence check at Line 1, All thread will wait at line 2. First thread (whichever will get CPU based on OS priority/roundrobin mechanism for processes) will enter and modify queue at line 4. Because of this, queue may not satisfy condition for other waiting threads, so thread entering second should not run code if condition is not satisfied, hence check at line 3. We could omit line 1 actually but adding it will have performance boost especially when condition is important to pass before entering to synchronized block.

   if (this. linkedThreadSafeQueue.size() == this.maxLimit) {
      synchronized(this.linkedThreadSafeQueue) {
          if (this. linkedThreadSafeQueue.size() == this. maxLimit) {
                        //Modify queue
          }
      }
   }

This helps when developer is dealing with singleton (condition based) and multithread implementation where he does not want to initialize same object again and also not to lock thread unnecessarily.

Who has the problem? The appraiser or the appraisee?

Hanuman’s mother once asks him why they went through all the trouble of building a bridge across the sea, fighting demons, and killing Ravan to save Sita. “You could have just lashed your tail and in a single sweep gotten rid of the demons and saved Sita without any trouble. So why didn’t you?” Hanuman replies: “Because no one asked me to. Besides, it was Ram’s story not mine.” This tale draws attention to two points. The appraisee (Hanuman) knows that he is being celebrated for his compliance, not his capability. But he does not begrudge the appraiser (Ram) as he knows the story is about Ram’s exploits and not Hanuman’s.

So whose story is your appraisal system measuring and what exactly is it measuring? It is not a measure of the employees alone or in isolation; it is a measure of the employee in a particular context; it is a measure of what the employees were asked to do and whether the resources allocated to them was good enough to enable them to reach the results. If the results are bad, then the problem is not just with the employees, it is also with the organization’s expectation and resource allocation. The appraisal system reveals how good or bad the organization is.

But that is not how we see appraisal system. We use it to determine how much we should pay employees, who should be promoted. In other words, it is a tool that is used to determine distribution of company expenses. We use it to judge employees and measure them against each other, determine how much a person should be paid. We do not see it as an appraisal of organizational capacity and capability. It is as much about the subordinate as it is about the appraiser and the organization as a whole.

Check what makes the shareholder and CEO happier: more profits (company has done well) or better bonuses (people have done well). This will reveal the attitude towards the appraisal system.

Try this simple process. Check how many times the CEO checks if the sales target has been achieved. And then find out how many times he checks if the appraisal has been done. You will find that if you do not chase the CEO, the appraisal is often not done. But no one has to chase the CEO to ensure the sales are closed on time. He in fact is busy chasing the CFO to delay closing.

What does this mean? It reveals how much people matter in a company. How much do we truly value them. Are they simply a means to an end or as valuable as the shareholder? We talk about company growth but often that is regardless of people growth so there is no connection between the balance sheet and the appraisal system. But people join a company for their own growth, not the company’s growth. Nobody says this publicly; it is not politically correct. This detail is not appreciated in modern management. We are too driven by institutional models to value individuals. And so naturally, the appraisee feels invisible and is never happy with the report card. Even if he has given more his spectacular best in a particular market, he knows the bell curve of the organisation will democratize him, make him average and equal to others, which is never good enough. Naturally there is lack of ownership and a lot of angst.

Courtesy & Reference: Economic Times

SCUBA diving in india

I was on work trip in UK where I met with one of my friends (Bijal Bhatt). He came up with a proposal of going for Sky Diving near Huntingdon. I decided to join him, but could not make it as it was pretty far from where I was staying.

That regret of not doing Sky diving landed me to do something else after returning to India. Planning started with an inquiry to a top SCUBA training school in India, in Jan 2012.

I and a bunch of crazy adventurer lovers (Akash, AshishDipesh and Pratik) decided to take up SSI certificate course with Barracuda. We booked the tickets and resort, and planned the trip.

It was a totally different experience and for the first time in a my life I inhaled air into my lungs under water. We crawled at the bottom of the sea for 4 times in 2 days; and dived 4 to 12 meters deep in the ocean. It was really fascinating to see some marine creatures (the most scary of the lot was Electric Eel) and plants. It was hard to identify each one of them but Venkat and the team did really good job educating us about them.

All diving sites were around the Grand Island. I remember two of them, Suzie’s Wreck and Bounty Bay.

It was not only pure fun, but also some sweat to earn that badge. To get that certificate there was criteria set as a part of clearance process. One has to swim (no snorkel or any support) 200 meters in the open sea (in my case it was from island shore to sea and back) and 10 minute cycling (using legs and without any hand movement) in water. Tough, especially when you have not tasted chlorinated water of pool since for many years. Well, but somehow I survived.

I thought it is all over and now I will get certificate to dive alone, but wait, that was not end of the story. We came to know that there will be objective exam after returning back to the base. That was also cleared; after all I wanted to do something which is was not a child’s play. And here it is…

SSI C-Card

How to configure SSL in Apache server

By following below steps you would be able to install SSL certificate in your apache web server.

1. Generate Key file from apache server.

You can use Apache\bin\openssl.exe to generate key file. Execute following commands to generate key file.

genrsa -des3 -out private.key 1024
Enter pass phrase for private.key: <Enter password> (recommend Alphanumeric password)
Verifying - Enter pass phrase for private.key: <Re-enter the password>

File private.key will be generated in \Apache\bin\ folder.

2. Generate CSR (Certificate Signing Request) file from apache server.

Execute following command to generate CSR.

req -new -newkey rsa:2048 -nodes -keyout private.key -out <domain name>.csr

You will be asked to enter information that will be incorporated into your certificate request as follows.

Country Name (2 letter code) :<Country Code>
State or Province Name (full name) :<State Name>
Locality Name (eg, city) :<city name>
Organization Name (eg, company) :<organization Name>
Organizational Unit Name (eg, section) :<Organization Unit>
Common Name (eg, YOUR name) :<Domain Name>
Email Address :<email id>
A challenge password :<password>
An optional company name :<optional Name>

Your CSR file <Domain Name>.csr will be generated at \Apache\bin\.

3. Send CSR file to your SSL vendor. Vendor will provide you 2 files i.e. Webserver Certificate and Intermediate Certificate. Rename the certificate as follows:

Webserver Certificate: public.crt
Intermediate Certificate: intermediate.crt
Private Key : private.key (use the key file which you have used at the time of CSR creation)

4. In the Virtual Host settings of your site locate the httpd.conf file. Verify that you have following 3 directives within this Virtual Host. Please add them if they are not present:

SSLCertificateFile /usr/local/ssl/crt/public.crt
SSLCertificateKeyFile /usr/local/ssl/private/private.key
SSLCertificateChainFile /usr/local/ssl/crt/intermediate.crt

5. Restart the apache server.

6. To verify the SSL installation, hit the URL on browser, if it does not shows any error, SSL is successfully installed. You can verify the SSL by comparing company information and validity date.

Git – A distributed SCM

Git is a distributed source code management system. A very powerful, fast and reliable way of managing source code. I started working with Git in 2010 and I found it better than any other non distributed SCM. I thought of sharing all basic Git operation which one can use and become productive very fast.

Please note : This article does not focus on Git vs SVN or CVS. This gives information on most useful commands with Git which can help a person to quick start with it.

Most useful commands with Git:

SSH operation (Key generation): Use this to allow your repository to securely communicate with Github, Beanstalk or some other hosting platform or even with your team mate.

$ ssh-keygen -C "email-id" -t rsa

Git configuration

git config --global user.email email-id
git config --global user.name "Dharmesh Borad"

Repository operation

1> How to create Git repository : $ git init

2> Add remote repository : $ git remote add github git@github.url.com:/platform.git

Branching and Merging operation with Git

1> Creating branch under this repository

$ git branch branchname

2> Listing all branch under this repository

$ git branch

3> To switch between branches. Below command will switch to newly created branch i.e. “branchname”.

$ git checkout branchname

4> To merge with master branch (branchname -> master)

$ git checkout master

$ git merge branchname

5> To delete/remove branch

$ git branch -d branchname

6> To delete branch on remote machine(origin alias of remote machine/repository)

$ git push origin :branchname

Add operation

1> Adding files to repository index for further commit

$ git add filename

2> Adding folder to repository index for further commit. This will add files from that folder also.

$ git add foldername/

Remove/Delete operation

1> To remove file/folder form .git index but to not remove physically

$ git rm –cached filename

2> To remove files from index as well as from physical storage. Use –r for recursive remove.

$ git rm -r myfolder/

$ git rm myfile

Commit operation

1> Committing to local repository. –a is for all and to skip staging area. –m is for message.

$ git commit -a -m "commit message"

Remote operation

1> Pulling from repository

$ git pull remotename branchname

2> Pushing to repository

$ git push remotename branchname

Help operation

1> To get help on any command

$ git help commandname

To fetch branch from remote repository

git remote show origin

1> If the remote branch which you want to checkout is under “New remote branches” and not under “Tracked remote branches” then you need to fetch them first from remote

git fetch branchane

2> Now it will allow to create branch out of fetched one

git checkout -b local-name origin/remote-name

In case if you are looking for some very specific operation please write me back. I will try to give details in my free time.