Background
Message Queuing (MQ) middleware technology makes integration of services simple. For example MQ allows services to communicate in a flexible way by abstracting the connection details of the target service, data format and the underlying integration protocol. MQ allows the developers to build services focusing on the business logic rather than the integration mechanisms. This technology is built around two concepts: Messages and Queue’s.
A message is any piece of information that needs to be communicated from one service to other and a queue is a container of messages. The MQ infrastructure ensures that the integrity of the messages is maintained and makes sure that the messages are delivered correctly to their destination(s). One of the important benefits of queues is to loosely couple applications by acting as a buffer between the sender and receiver applications. Here’s an excellent post detailing the benefits of using queues and messaging concepts in the context of Amazon Simple Queue Service (SQS). SQS is an Amazon implementation of message queuing model accessible online. However SQS supports only http(s) based integration styles. (Stay tuned for more on SQS in another post). Messaging can happen in two styles:
Point to Point (PTP): In this style, messages arrive once and only once at a single correct destination. PTP messaging can be of two kinds:
Send and forget: In this case the messages are sent and an acknowledgement or the knowledge of further action at the receiver’s end is not made known to the sender. Typically integration architectures involving asynchronous style of messaging use this style of communication.
Request/Reply: In this kind of PTP messaging, a response is sent to every request that is processed. This is also known as synchronous style of communication between services or applications. Typically used in applications or services where order of processing is important.
However in the integration world PTP messaging has some limitations in scenarios involving processing of state information associated with an action or event information. Here’s where Publish and Subscribe style of messaging comes in.
Publish and Subscribe (Pub/Sub): In this second style of messaging, any number of interested consumers can subscribe to a message produced by one or more producers. A producer of a message is called publisher and consumer of that message is called a subscriber.
Context
Integration of services from Web 2.0 landscape involves the same challenges faced in the enterprise world. For example integration between Digg and Plaxo involves setting up a channel for communication and processing of data besides agreeing upon a standard message exchange format. If Digg were to integrate with other services then eventually a channel has to be setup for each of the services that it would integrate with. The costs associated to configure and setup an infrastructure for such integrations, data standardization per communication channel, protocol abstraction and its maintenance is not of any competitive advantage to many companies especially for those whose business models are around aggregation and presentation of user generated content. Here’s where Gnip comes in, an online message oriented middleware aka integration service provider, offering capabilities such as protocol abstraction, channel optimization through hub and spoke model, data standardization. Gnip is implemented over Apache’s Active MQ and Amazon’s SQS using Java. Interestingly they use Terracotta (rather than Gigaspaces) for node replication. Why Active MQ? It was found that Amazon’s SQS has latency issues for their usage.
Market Segment
Gnip plans in the future to offer more features such as data transformation, polling and identification. While identification might be unique to Gnip, however, data transformation and polling (pub-sub architectures help in polling scenarios) have been prevalent in the enterprise.
Gnip has an enormous potential to grow into an online service orchestration provider, uniquely positioning it in the centre of integrating Web 2.0 sites. However the threat is from IBM, only if it were to provide its messaging middleware products as services from its cloud.
Products from IBM such as Websphere MQ (WMQ, used as a messaging backbone), Websphere Message Broker (used for message transformation and routing), Websphere Transformation Extender (WTX used to handle large messages), Websphere Enterprise Service Bus (ESB used for service orchestration, message routing and message transformation etc) are proven in the enterprise market to be reliable and scalable. Many of the features that Gnip is planning to offer in the future, noted above, are already available from IBM’s middleware products. Other products in the messaging middleware and ESB segment are Fiorano MQ, Apache Active MQ, Zero MQ (the fastest MQ provider), Mule ESB, Apache ServiceMix, and Open ESB.
Currently Gnip services are offered for free, however, I’m speculating that their revenue generation would come from a charge back model that is typically used in the enterprise integration world.
References
This posting from Gnip’s blog discusses the rationale behind choosing Terracotta with a visual representation of their system architecture.
Other postings of possible interest: