In the cloud computing Google group, there was this interesting question asked: what cloud services can a small IT firm provide? This firm specializes in consulting, development and management. I replied to this question however I’m posting it on my blog as well.

An IT services firm can offer turnkey solutions involving business modeling, requirements gathering, analysis/design, development, deployment and management for applications/services using compute clouds (such as Amazon EC2, GoGrid, AppNexus etc). While business modeling and requirements gathering remain mostly unchanged from the traditional (non cloud) application development or hosting, however, the differences from traditional practices are mentioned below:

a. Application design would include architectural strategy and heuristics. For example, when to use components such as Amazon’s SQS for business benefits etc.

b. Application deployment or delivery can include packaging techniques such as a virtual appliance. This can further be extended to a full-fledged cloud service. For example a utility component in the application can be provisioned as a service.

c. Integration among various services such as using any of the SaaS integrators or online message queue service providers. SaaS integration would involve configuration, however, message queuing would involve coding.

One important thing to note is to make sure that any cloud service that you would use satisfies a business need or improves the efficiency of a business process.

1.Follow the performance queue theory when tuning. The bottom line here is to efficiently balance the availability of system resources vs. the work to be done for optimal application response time. The rule of thumb followed is the amount of requests that can be processed at the Web Server tier must be higher than the requests that can be processed at the Application Server’s Web Container level. Further, the number of requests that can be processed at the EJB container must be less than the Web container level however this number must be greater than the number of requests that can be processed at the JDBC connection pool or other enterprise integration endpoints.

2.Size the JVM appropriately. This is the most difficult task in the application performance tuning exercise. However keep in mind that large heap memory does not necessarily mean better performance. Sizing of the JVM also involves choosing the appropriate garbage collection algorithm. Typically parallel garbage collection algorithms perform better.

3.Tune the Web Server with appropriate thread pool size, this number (across all web servers) should be large enough to accommodate all your requests per your traffic needs and make them wait at the Web Server tier when busy or greater than the anticipated number of requests.

4.Size the JDBC connection pool appropriately. The idea is to reduce the time/resources required when creating a connection. For example, Websphere and Weblogic application servers provide initial and maximum connection pool size, by setting the initial size to a number greater than zero will help. However such numbers are application architecture specific and must be found from stress testing. Enable caching of prepared statements on the JDBC Connection Pool.

5.If your application utilizes point to point (PTP) messaging and order of message processing is important then obviously Message Driven Beans (MDB) for processing inbound messages limits scalability. For example if the messaging provider is Websphere MQ, such scalability issues can be alleviated by using simple POJO’s based on file system look up for MQ objects.

6.If your application is data intensive then consider having a caching layer, for example Gigaspaces IMDB. Such decisions are better off when made during the application architecture design phase, rather than, retrofit it when performance problems are observed.

The above mentioned are some of the tuning tips applicable to a Java enterprise application however is not comprehensive list. Some of the other important components of an enterprise application tuning involve tuning of network, operating system, database, messaging platform and edge tier components such as Content Delivery Network and H/W load balancers. These must be tuned as well for best results.

Other articles/postings of possible interest:

1. Top five Java application performance management tools

2. To scale out or not using Gigaspaces

3. AdventNet, one product for all performance/service management needs

4. Gnip, Online Message Oriented Middleware (MoM)

IDC recently published the findings from a survey on IT cloud services (cloud computing). The survey was conducted on 244 IT executives/CIOs and their colleagues about their companies’ use of, and views about, IT Cloud Services. Summary of top three findings from this survey is mentioned below in two categories:

Benefits:

1) 63.9% find the number one benefit of cloud services in easy and fast deployment

2) 61.5% of the respondents find the pay only for what you use as benefit #2.

3) 57% responded agreeing the #3 benefit of using cloud services is, less in house IT staff and cost.

Challenges:

1) 74.6% find security as the number one challenge

2) 63.1% find performance as number two challenge. Not sure how performance is #2 concern. Isn’t cloud computing about providing servers on demand?

3) 63.1% find availability as #3 challenge

The blog post providing more details on this survey is available here.

Guaranteed delivery pattern is about delivering of messages by the sender when the target messaging system has failed.

This pattern is one of the most misunderstood patterns especially when the EAI application is hosted in Weblogic Server and when IBM Websphere MQ (WMQ) is used as the underlying messaging platform.

In Weblogic, the inbuilt JMS Server option provides guaranteed message delivery when the WMQ Queue Manager is unavailable. How? The messages are temporarily stored in a file, or in-memory or on a database. When the target WMQ system is made available, then the stored messages are forwarded to the destination queue manager. Though Weblogic server implements this pattern through JMS Server/ MQ Bridge, however, it is not recommended. Because this would decrease the economies of scale.

WMQ provides scalable and simple solutions to such problems. Both these solutions are mentioned below:
a) Mount the in-memory persistent queues on a SAN that is configured in High Availability (HA) mode.

b) Configure the queue managers in HA mode and load balance the workload across them.

Configuring the queue managers in HA mode usually involves clustering of queue managers when Message Queuing (MQ) intercommunication is based on WMQ Server to Server.

However in the situations of using WMQ client or connecting to WMQ through HTTP/REST (available straight out of box in WMQ v7.0), the HA mode can be configured through a load balancer of choice. Note this method of messaging is applicable to applications that are integrated in asynchronous mode, where order of message processing is unimportant.

Model View Controller (MVC), is an architectural pattern and is defined as:

“It is common to split an application into separate layers that run on different computers: presentation (UI), domain logic, and data access. In MVC the presentation layer is further separated into view and controller.

MVC is often seen in web applications, where the view is the actual HTML page, and the controller is the code that gathers dynamic data and generates the content within the HTML. Finally, the model is represented by the actual content, usually stored in a database or in XML nodes, and the business rules that transform that content based on user actions.” Courtesy: Wikipedia (including the image below).

Note that the MVC pattern does not provide guidance on where the integration code should be (for example when connecting to a MQ system) and which layer in MVC should invoke it?

What is messaging system integration layer or gateway pattern?

In the world of integrating applications (also called EAI) using messaging, the purpose of a messaging system integration layer or gateway pattern is to decouple the application from the destination message queuing system. For example assume if the web application architecture is implemented following a layered pattern in combination to MVC, then the messaging system integration layer or gateway pattern code would be responsible for four things:

a) Establishing connection to the destination message queuing system

b) Handling data format (marshaling or un-marshaling to and from XML data format)

c) Exception handling

d) Sometimes failover (applicable only when this is implemented in the code).

The problem and the solution:

While the gateway pattern or integration layer abstracts the code required to connect to messaging infrastructure, however, which layer in MVC should invoke this pattern? Obviously this determines how loosely the application would be coupled to the messaging layer/infrastructure. The design heuristics to solve this problem is largely influenced by the application architecture. From my experience, the greater the abstraction of the messaging layer from the application and invoking it from an application’s domain layer or domain model pays off pretty well. What are your thoughts? Share them in the comments below.

Performance tuning of an application helps reduce the issues that arise from performance bottlenecks in an application. Performance tuning is typically attributed to the optimal application response time to a user’s request.  The application response time is acceptable as long as it satisfies the business Service Level Agreement (SLA).

Performance management of an enterprise application involves end to end monitoring and tuning, starting from the Web Server, Application Server, database, MoM and the JVM. However one must include performance parameters related to edge network (F5, ZXTM, etc), Network and OS before certifying the performance of an enterprise application. This posting discusses the tools that support performance management for applications executing in a Java/JEE environment. Typically the data counters that are instrumented by such tools fall in these categories: Web Container, EJB container, JDBC Connection Pool, JVM, Java socket connections, Java I/O, Java process CPU utilization and associated threads with each of the executable queue’s, for example Web Container Thread Pool.

1. CA Wily Introscope

Arguably CA Wily Introscope is the market leader in the application performance management segment. Introscope uses an agent based technology for data collection. The agent is deployed on the server hosting the Java application that instruments the application in real time by modifying the application’s compiled byte code. The instrumented data is then transferred over to a centralized server, to be persisted in a database. This persisted data is later used for conducting offline or real time analysis. Since the agent merely instruments the data, there is no overhead incurred from the agent on the application. The genius of the tool is in its simplicity and the minimal overhead on the application response time, even under a stress test. Introscope provides a comphrensive list of instrumented data counters for performance monitoring and tuning. Though originally the tool supported only applications based on Java/JEE, however, it later extended support to MS .net applications.

2. Dynatrace Diagnostics

Dynatrace Diagnostics is similar to Introscope in some capabilities and using the agent topology for data collection. However they use a proprietary technology called “Purepath” for application instrumentation, rather than byte code wrapping. Dynatrace showcases its strength in the monitoring of remote calls by presenting a complete list of data counters per remote call executed across Java or .net applications. For example, the data counters associated with remote calls are: bytes transferred per request & response, object count per request/response, time spent in serialization/de-serialization, response time associated with client/server, network latency among others. Dynatrace provides an aesthetically appealing GUI and supports applications based on Java/JEE and .net platforms.

3.Quest PerformaSure

Quest Software is the maker of popular Java memory profiler called JProbe.  However Quest PerformaSure fills the gap in identifying performance related issues across application’s tiers including web servers and databases. Arguably PerformaSure is the only product that supports the most Java Application Servers, operating systems and databases. PerformaSure like the above two tools, uses an agent based architecture to collect instrumentation data however employing a proprietary technology called “Tag and Follow” for collecting data at a transaction level, for analysis. The unique features in PerformaSure are, a SQL Browser for diagnosing SQL query related issues from a Java application and a tree based view of method calls to identify resource contention in method calls. The amount of instrumented data that can be collected is configurable however it has a slight overhead associated with it.

4.HP Business Availability Centre (HP Diagnostics)

HP Business Availability Centre (BAC) is one of the most sophisticated tools to measure performance, availability and in reporting on the business impact of an issue. HP BAC is a collection of products, each product focusing on one area related to performance , availability management that integrate with a CMDB and other Service Management components to provide a business centric view of the performance issues. While HP BAC uses synthetic transactions to monitor the user’s perceived performance, however, this may not report the actual user’s experience of a website, especially, if your user base is geographically dispersed.  One of the products in HP BAC suite for finding root cause of a performance issue is HP Diagnostics. HP Diagnostics provides a low overhead and comphrensive performance management solution supporting traceability of an issue across tiers. However the value add of using HP Diagnostics for performance management, is realized only when it is integrated with HP BAC.

5.IBM Tivoli Composite Application Manager

Like HP, IBM offers a suite of products for enterprise management under the brand name Tivoli Monitoring. However, for application performance management, IBM Tivoli Composite Application Manager (ITCAM) can be used. ITCAM provides end to end application performance management capabilities and probably the only product that provides traceability across tiers including middleware such as Websphere MQ and Websphere Broker, and virtual servers (e.g. VMware ESX) among other components. ITCAM can be integrated with other IBM Tivoli products (such as Tivoli Netcool Omnibus, Impact and Webtop) to provide a comprehensive management and availability picture of an enterprise.

Other articles/postings of possible interest:

1. To scale out or not using Gigaspaces

2. AdventNet, one product for all performance/service management needs

3. Gnip, Online Message Oriented Middleware (MoM)

I mentioned in the article on SaaS offering of ITIL, that the cloud computing or as SaaS matures, more infrastructure software components would be made available in the cloud.  For example on demand Message Queuing (MQ) service.

In house MQ software has played an important role in the enterprise application integration world for over a decade. However in the integration of services in the Web 2.0 world, the traditional message queuing model has some limitations such as:

1. Lack of accessing MQ over HTTP(S) protocol straight out of box. In order to accomplish this, service providers had to build HTTP(S) Adapters over MQ.

2. Scalability issues arise, when integrating over HTTP(S,) because of a lack of MQ platform agnostic and proven implementation of HTTP(S) Adapters.

3. Additional costs to cover licensing and server provisioning when service demand increases.

4. Troubleshooting requires a deep understanding of the MQ product. Obviously highly skilled MQ admin personnel don’t come cheap.

5. Full life cycle management of MQ environment has a cost associated to it and is directly proportional to the number of integrations and the number of messages processed.

6. Web 2.0 companies are interested in focusing on their core offering as opposed to managing integration infrastructure and challenges associated with it.

The above limitations make sense to use MQ service on demand.  Some examples of such service providers are:

1. Amazon Simple Queue Service (SQS)

2. Gnip

3. OnlineMQ

SQS is a basic queuing service provided by Amazon in EC2 cloud computing platform. One of the advantages of SQS is in guaranteed availability of messages when dropped on the SQS queues. Second, if your service is hosted on EC2 then using SQS is cost saving because intra EC2 data communication is free of charge. However publish and subscribe style of messaging is not yet offered. Neither does it offer guaranteed once and only once of message delivery. One has to programmatically control that the same message is not delivered multiple times.

Gnip (see our coverage here) has a success story on offloading traffic from high volume web sites such as Digg, Twitter among others. Gnip uses Amazon’s SQS and wraps the access to it based on SEDA; probably this is the reason why they can handle high message volume. Gnip provides more than a traditional MQ, for example they also provide services such as message transformation and routing. You can use Gnip if you are integrating with any of the sender or receiver from the list of supported consumers/producers of Gnip. Any custom integration or integration between your application components would find Gnip limited.

OnlineMQ is a new service and still in beta. However they seem to be providing a comprehensive MQ service. OnlineMQ provides an intuitive GUI based admin tool and supports both styles of messaging i.e. PTP and Pub/Sub models. You can enable or disable a queue for read only or write only.

From the above solutions none of them is an ideal MQ solution for all situations. However choosing Amazon’s SQS provides you with the flexibility of building the functionality that you need over it.

If you liked this article, then you may find the below mentioned cloud computing and SaaS articles interesting:

1. VMware, Citrix want to be enterprise cloud enablers

2.  Service-now, a SaaS offering of ITIL