Architecture of a Modern Web Application

Some time in my career, I was working in a RFID tracking system and I was designing the real time event notification web application for that, I was using Google Map API to show the activities on a particular facility/ floor map, where RFID sensors were mounted. I tried lot of techniques and APIs and finally, I could able to push messages to the clients.. Now the question is “how did I do that???” It was a long back and at that time there were very few APIs available, which were providing server side pushing. I took the DWR Reverse AJAX and in a layman’s way to explain, clients were subscribing for the messages for a particular criteria with the server. But  to be very honest, let’s say after some 7 - 8 hours, the browsers used to be almost inactive and user couldn't even do anything, may be because of memory. Few years back, when I started using the TweetDesk a Chrome app, I realized it uses the same technique to get the messages and it is damn good and efficient. I was always very eager to know what exactly and how they are doing that.. Today facebook.com, amazon.com are very popular and they also follow some kind of the same approach… One more thing, I noticed with firebug, that for most of the operations, there is a AJAX call and only the data is flowing in terms of JSON/ text; there is no form submit to any action.. And the biggest advantage is that they are user-friendly, interactive and kick ass fast.

Long time back, there was a standard approach for all the web applications. Most of the cases, everyone was following the MVC or MVC 2 design with a service layer and DAO layer. And those days one application means a huge and fat war or an ear. Companies were using heavy application containers like weblogic and jboss, with lot of dependency. It was like a fat ugly man needs an extremely customized environment to live his life, or else he will die in no time. After that some time back companies were moving towards the SOA. A big application was further divided in to small chunks and they were interacting with each other by messages.. SOAP based web services were very powerful technique to interact with services, but now the world is going towards RESTful web services as it is simple and easy to develop. With the SOA the applications were really simple and maintaining a low cohesion. The applications used to be light weight and the servers used to be healthy all the time. But still there was a room for improvement, why the UI elements are getting streamed from server and consuming a major band width of the network.. In order to answer this question, designers moved the MVC to front-end and back end is now only the services.. If you are a back-end developer, you will be thinking, how it is possible and may be feeling insecurity about your job.. But believe me, now it is time to adapt the so called ugly JavaScript to develop beautiful and interactive applications. As the MVC concerned with front-end, there are lot of JavaScript APIs are available. So I can now conclude something that all the modern web sites like facebook.com, amazon.com and some Chrome apps like TweetDesk are using the same concept.

I was part of the Spring One Conference India Dec, 2012 and there the things were got cleared with a talk from Jeremy Grelle. Jeremy is a Spring Source Staff Engineer and one of the open source contributor for many projects.. Next, I am going give a picture of what Jeremy presented in that conference. In this article, I will take you to “Where we’ve been”, “Where we’re going” and “How we will get there”.

Static Web Pages with HTML:

When the www started, all the pages were static and served from server. There used to be separate page for separate data. The world was with URL and links.

• Pros:

1. Low Computation: As they are static, there is hardly any computations from a programmer point of view.
2. Highly Cacheable: They are very easy and perfect candidate for caching.
3. Highly Indexable: They are easily get indexed by leading search engines for their static data content.

• Cons:

1. Hard to Update: If you need to change anything, you have to go to the HTML file and change that  and a developer is required for the same.. Bad dependency; Isn’t it?
2. No Personalization: You can’t have something which you really looking for.. You have to see a generic view, even if you are not bothered about the rest 90% of the content.
3. Poor UI: UI used to be poor with bad response from server.

Dynamic Web Pages with Scripting:

After some time, when the requirements and needs changed, pages were dynamic on user inputs. Users could able to get dynamic data on the same page.

• Pros:

1. Dynamic: Now the pages went dynamic depending on the user inputs and different other criteria’s.
2. Selectively Cacheable : As the pages were dynamic, so dynamic contents were not applicable for caching.
3. Highly Indexable: They were also well indexed by the search engines.
4. Personalization: As the user is able to provide inputs, they could able get the user a view that user is looking for.

• Cons:

1. High Computation: To make a page dynamic, it required a computation overhead in different manner. Lot of scripting technologies had been adopted to create dynamic web pages.
2. Hard to Create: Same page different data representations made very difficult for the browser to cache.
3. Poor UI: Even though the pages went dynamic, the UI used to be poor, less interactive and less responsive.

Dynamic Pages with JavaScript:

Introduction to JavaScript to the web development made the pages bit more interactive and responsive. Developers started using JavaScript for basic form validation and other activities in DOM.

• Pros:

1. Enhanced Usability: With the help of JavaScript, developers could able to give a good response to some of the actions, where there is no back-end data involvement.
2. Reduced Trips to Server: As most of the validations went front-end, there was a network band width save. Developers eliminated the validation calls from server-side.

• Cons:

1. Abused Annoyed Users: As JavaScript is a client-side technology, lot of users started misusing that.
2. Business Logic Often Implemented Twice: With reference to validation, sometimes the validation logic and other business logic were implemented both at client side and server-side.

AJAX – Web 2.0:

Google came up with a new approach of interacting with server. With the help of Web 2.0, Google launched Map API and Gmail. Server interaction made possible both synchronously and asynchronously with out submitting an action to the server.

• Pros:

1. Killer UI: With the introduction of AJAX, the look and feel of the UI went out of the world with lot of UI components.
2. More Responsive Apps: User were able to interact with application easily without much effort and as with AJAX only the data used to be get streamed the via network, the response for a particular action used to be very fast.

• Cons:

1. Difficult to Cache: As there are hardly any static content, at least, it was very difficult to cache the whole page.
2. Impossible to Index: With dynamic data, indexing a page for a search engine was quite impossible.
3. Required JavaScript: With introduction to AJAX, there was a need for skilled UI developers in industry.

Unobtrusive JavaScript:

After some time, JavaScript proved to be a clumsy language unsuitable for serious application development. This has been largely due to inconsistent implementations of the language itself and the Document Object Model in various browsers, and the wide spread use of buggy copy-and-paste code. Run time errors were so common (and so difficult to debug), that few programmers even tried to fix them, as long as the script behaved more or less the way it was supposed to, scripts often failed completely in some browsers. At that moment the industry was again fallen back to the basic HTML way of presenting the view. And this is the time when some MVC 2 frameworks were evolved. Unobtrusive JavaScript is a general approach to the use of JavaScript in web pages. Though the term is not formally defined, its basic principles are generally understood to include Separation of functionality (the "behavior layer") from a Web page's structure/content and presentation, best practices to avoid the problems of traditional JavaScript programming (such as browser inconsistencies and lack of scalability) and progressive enhancement to support user agents that may not support advanced JavaScript functionality.

Architecture of a Old Web Application

• Pros:

1. Wider Compatibility: As again the UI started coming from server as a HTML, it has a better compatibility with wider range of browsers.
2. Just as Rich UI: As the UI was getting streamed from server, it was a raw HTML with rich components.
3. Just as Responsive: As designers were still stick to JavaScript for some non-business related activities, the response of some activities were still good.

• Cons:

1. Higher development cost: Generally to make a simple application, now you need a big team with developers, architects and some managers to make sure everything happening as per the plan, which is a time consuming process..
2. Requires thoughtful engineering: As you can’t make changes to the final product as it would have lot of dependencies and you never know one change will break how many other things. So you have to have a very good design looking at the future scope.

Client Side Applications:

Over the time designers noticed one common thing in all approaches, i.e. there are unnecessary data like HTML are also flowing from server to client and that is a major problem for slowness of the application. Now there is a significant contribution to the JavaScript APIs from the open source contributors around the world.  Business logic is no more part of the server, all moved to the front-end along with the MVC. Application state was started storing in front-end, only the permanent state has been stored in back-end. As there are no more MVC and business logic in backend, the back-end part of the applications are turning to services. SOA also has a greater impact on the current design, instead of thick  fat web application, it is getting divided in to smaller chunks to make the back-end healthy and independent with each other. Introduction of RESTful web services made front-end directly interact with services. Recently Google announced the Chrome Laptops, which are totally on the browser, so looking at the current processor architecture and memory capacity of personal computers and laptops, the browser is capable of doing complex computations and also capable of storing significant data required for a particular page.

Architecture of a Modern Web Application

• Pros:

1. Reduce server workload : As the MVC and business logic is moving to the client side, there is minimal computation at the server side, may be the CRUD operation on a object. And again with service call only the data is flowing in network, so there is a significant save in network band width.
2. Application is highly cacheable: As the business logic and MVC is moving to front-end, front-end is making use of system resources to store data.
3. Extremely rich UI: As I explained above, there are lot of contribution to some of the open source projects and there are lot of light weight JavaScript APIs are available now.

• Cons:

1. Content not indexable: As the content is 100% dynamic, there is no scope for the search engines to index the page.
2. JavaScript: Now it is the time to learn JavaScript and start using that. Lot of companies are recruiting dedicated UI developers.
3. Often requires a modern browser: Along with JavaScript, there are some other technologies like HTML5 and CSS3 are also adopted with context to the new architecture.. So some time user need a updated browser to use the application.

So now the web application is moving from server side application to smart clients and services. Simple templates can render on client, JSP will no more render the client side. Some of the modern techniques like web socket(for subscribing messages from server), web worker(worker and job model like Executor Framework) and DOM event will be adopted for the web application to make it more responsive and interactive.

With the current configuration standard of a personal computer or a laptop, this approach is proving it’s capability.. As discussed about some of the web sites like facebook.com, amazon.com, ebay.com etc. and some of the browser apps like TweetDesk, RSS Readers etc. are very popular for their performance.

The definition of back-end and front end are shifting to front-end != client and back-end != server. There are lot technologies are evolving and there is a huge support from the open source contributors. Now it is the time for a java developers to take interest in JavaScript and start learning the new technologies like css3, HTML5 with some JavaScript libraries like jQuery, s2js, cujo.js, scripted, SockJS, monty-hall, monty-hall-ui etc.

Even though, I have drafted this article for you guys, I want to give the credit to Jeremy Grelle, SpringSource Staff Engineer and open source contributor. I would also like to thank VMWare for inviting me to SpringOne India 2012 Conference.

I will soon be developing some small applications using this architecture and will probably deploy those in the Cloud, so that you people can play around that.

Read More

Some limitations Of DWR

DWR is a great technology to add interactive AJAX calls to your application. The best thing about DWR is, it works with wide range of browsers and you don't need to bother about browser dependent code. While the run time DWR generate the intermediate java-script to call your server methods. But selecting DWR is a tricky task and you need to take care of the following aspects of your application. Recently I came across the following few thing..

• I was trying point my javascript to a different apache server using RewriteRule in apache and tomcat or application server was still there in my local machine. But the DWR was not working.. May be there will be some configuration to make this happen.

• Same way I pointed the application server to a different server using ProxyPass in apache. And the same thing happended to me. The DWR was not working. Basically I thing if you are using DWR, your javascript and the application should be hosted in the same server.

• If you are using SOA(Service Oriented Architecture), then this a bad idea to use DWR, as you can not make cross context calls.

I am posting to get solution for all these problems if any.

Related Topics:

LOG TAILER

REVERSE AJAX(DWR)

Read More

SPEED UP WITH MEMCACHED

Is your website running into performance bottlenecks? Does the database or backend feel like a really expensive resource, even though you’ve got a huge cluster set up to improve parallel processing? Read on to find out why you should be including Memcached, in your SOA based application.

Caching is a concept that almost all developers use in some form or the other in their applications. It’s basically about storing a piece of information in memory so that it can be retrieved quickly. Caching is mostly used for data that is accessed repeatedly, so that instead of calculating/retrieving from the disk repeatedly, which takes time, we can instead directly look it up in the cache, which is much faster. Caches can be used at multiple places in the application stack, so you have quite a few options when it comes to choosing where to cache, what to cache and how to cache.

Here are some of the techniques on how and where you might like to cache data:

• Browser caching: As Web developers might be aware, some data can be cached on the client-side in the browser, like images, etc., so that they are automatically used when repeated requests for that resource are made.
• Server-side caching: Data or objects can alternatively be cached on the server-side itself. This can either be a local server cache, a centralised caching server or a distributed cache.
• Local database query cache: A good database caches the database queries or data internally, so as to improve the speed of looking up data as well as the performance of the database.

You may choose to implement a cache in one way or another, or you might use a combination of more than one technique to cache different types of data at separate levels. But more importantly, it is helpful to know whether you even need caching in the particular application/use-case you are thinking about.

Most people, in the process of implementing a cache, actually lose out because it was wrongly implemented. So the cache ends up slowing down the application, instead of speeding it up. Getting fancy software with fancy features doesn’t always make sense, but using even the modest ones in the right way, does.

Why you need Memcached

This discussion assumes that you have set up a cluster, and you want to implement caching. In this case, what happens if you start caching on each node independently? You will see that some nodes face memory issues, while others have quite a bit of memory left. Moreover, most of the data stored in their individual caches is redundant.

This calls for a centralised caching mechanism that makes sense, such that the data being cached is evenly distributed and unique for the whole cluster. And memcached is the right thing to choose.

It provides a solution in which the available memory in the cache is the sum of that on all nodes on which the Memcached instance is running. So if, for example, you have 10 nodes, with each being allocated 1 GB of memory for caching, you get a total of 10 GB of cache available for the whole cluster. Here are some features in Memcached that might lure you into using it within the context of your application:

• Easy scalability: This feature is applicable for almost any software with the tag of “distributed”, but still, it is worth noting that Memcached needs minimal configuration to add a new node, with almost no special interconnect requirements. Your available memory in the cache just increases on the fly.
• Hidden complexity: Memcached hides beneath it all the complexity associated with storing/retrieving the data from the cache. All we need to provide is the key associated with the data. The whole task of determining which node to store the data on, or to retrieve it from, is done by the Memcached client itself.
• Minimal impact of a node failure: Even if a particular Memcached node does fail, it has almost no impact on the overall cache other than reducing the available memory, and a minor increase in the number of cache misses.
• Flexible architecture: Memcached does not impose a restriction on all nodes to have a uniform cache size. So, some of your nodes with less physical memory can be set up to contribute perhaps only 512 MB to the cluster, while others may have 2 GB of memory dedicated for the Memcached instance. Apart from this, you can even run more than one instance of Memcached on a single node.
• Multiple clients available: Memcached has client APIs available for various languages like PHP, C++, Java, Python, Ruby, Perl, .NET, Erlang, ColdFusion and even more.
• Cross-platform: Memcached is available for a wide variety of platforms including Linux, BSD and Windows.
• Multi-fetch: With the help of this feature, we can request values for more than one key at once, instead of querying them in a loop, one by one, which takes a lot of network round-trips.
• Constant time functions: It takes the same amount of time to perform an operation in memory, whether it is a single key or a hundred. This corresponds to the Multi-fetch feature discussed before.

Related Topics:
THIRD PARTY MEMORY VS JVM MEMORY
MULTICORE ARCHITECTURE AND SOA
MULTITHREADING AND MULTICORE CPU
JAVA CONCURRENCY - PERFORMANCE BOOST
Java 5 Executor Framework
Concurrency: Callable and Future

Read More

REVERSE AJAX (DWR)

It had been a long time, I last worked in DWR. Around one year back at my previous company, I was working in DWR. I used reverse AJAX technique to some part of the application, which was quite impressive. So I started playing around this again to do some new things.

This part of the post is just a basic demonstration of reverse AJAX. The basic idea is to push different messages to different clients.  Let's take a example; there is a page with javascript links to load different shelf's of a ware house in to a div element. Client "A" is viewing shelf "SH-01" and client "B" is viewing shelf "SH-2". In this case client "A" and client "B" should have been pushed with different updates.

Environment set up:

1. Install JDK 1.6 in your system or any platform you want.

2. Install Tomcat 6 followed by JDK installation.

3. Deploy the war file to " webapps" of Tomcat installation directory and start the server.

4. Now you can view the demonstration by entering "http://localhost:8080/DWR" to the browser location bar.

Eclipse setup:

1. To work with the source code, please use eclipse 3.3 or higher version.

2. Download the source code.

3. Create a dynamic web project with name "DWR".

4. Copy the whole content of downloaded "DWR" folder to the workspace "DWR" folder and refresh the workspace.

5. Configure Tomcat to eclipse and add the project to that.

About the application:

The application has got the following components.

1. Event Engine

2. Event Listener

3. Script Session Manager

Event Engine is nothing to do with reverse AJAX. I just created to demonstrate the message pushing technique. Event Engine will get stated when the server starts for the first time. This engine will keep on create the messages and notify all the listeners.

Event Listeners will be listening to all the events generated by the Event Engine and use the Script Session Manager to push the message to respective client.

Script Session Manager is responsible for registration and de-registration the client script sessions. And it is also responsible for message processing and pushing to respective client.

The whole process goes something like this.

1. When you start the server, the event engine will get started with that.(Created a servlet with init block and configured in web.xml to create a event object at startup)

2. Now you can view the demonstration by entering "http://localhost:8080/DWR" to the browser location bar.

3. Select Any parameter from the dropdown.(The application will create a script session at server side.)

4. Now when ever any message for that parameter will get generated, the message will be pushed to the client.

5. You can change the parameter in UI and get the respective updates accordingly.

6. If you close the window without removing the session, the session will get expire after 500ms.

Shared Items:

1. Source Codes or you can pull it from GitHub.

2. War File

3. Java Doc

Read More