Developing with Grizzly / Building a Example Client Server Application

Contributed and maintained by John Vieten
June 2008
[Revision number: V1-0]
This publication is applicable to Grizzly NIO framework 1.8.4 latest revision


This tutorial tries to give some hints on a example published in Grizzly framework samples. The example only builds on the basic Grizzly framework module. The code snippets shown might be interesting to readers who need to integrate Grizzly in their applications and can not find the needed functionality in any higher level Grizzly modules.

For more information about working with Grizzly see the overview page on the Grizzly web site.

Expected duration: 30 minutes

Contents

 

Tutorial Requirements


Before you proceed, make sure you review the requirements in this section.

Prerequisites

This tutorial assumes that you have some basic knowledge of, or programming experience with, the following technologies.

Software Needed for This Tutorial

Before you begin, you need to install the following software on your computer:

Note: just to help the reader to distinguish grizzly framework classes from example's classes the later will be shown in green.

top

The Rough Picture


As an example a client will send messages to a server over a single non blocking socket connection. This connection will be held open until the client disconnects. Of course the user can create as many clients as he wants. Because communication should be very responsive large messages should not block other small messages. Therefore a custom protocol will be developed in which messages will be broken into fixed length packets. Note: In the next two figures many details are skipped or simplified and hopefully will be explained in the next sections.

Figure 1: Client sends messages


  1. A message can contain any data : bytes of an image, serialized objects, etc.
  2. Actually a client constructs a message class which can be a RequestMessage, ReplyMessage, MessageError, etc.
  3. ProtocolOutputStream which is part of the example streams a message into packets and adds protocol headers
  4. The server receives bytes and with the help of the protocol parses or reconstructs these bytes into messages
  5. Each client uses only one permanent open socket connection

top

Figure 1: Server receives messages


  1. Businesslogic receives client messages. Note if for example a large XML file was transmitted logic could start working as soon as first packets arrive
  2. If message is meant for another client, server would construct a RequestMessage and send it to this other client. So a server can initiate requests

top

The Example Custom Protocol


The example custom protocol has the following characteristics:

  1. Fixed length header to indicate the size and also classify data
  2. Maximum packet size of 8192 bytes. Large amounts of data will be split into several packets
  3. It must be assured that on a connection all bytes of a single packet are always send in a whole piece
  4. Packets belonging to different messages can be interleaved though.
  5. Flags to indicate the kind of message.
  6. Request Id to track the reply and allow interleaving data packets
  7. Session Id to recover from a connection disconnect

Protocol Layout


23 bytes variable (max 8169 bytes)
4 bytes 1 byte 1 byte 1 byte 4 bytes 4 bytes 4 bytes 4 bytes n bytes
Magic Version Type Flags Fragment Id Packet size Request Id Session Id Variable length body

Some Details on Custom Protocol Messages


The example custom protocol currently defines 6 different kinds of Messages.
Note: The message names are similar to the Messages defined in GIOP Standard (but of course GIOP is much more than this simple example protocol)

MessageRequest used to request an operation on the other endpoint side
RequestMessage allows the other side to reply.
FragmentMessage enables long messages to be chunked
MessageError for protocol error handling
MessageCloseConnection close a connection in a defined way
MessageCancelRequest used to cancel requests that take to long

A client sending a RequestMessage normally expects a reply. Because a client may be multithreaded a Request Id is associated with each request and its reply. This allows a server to reply to a client request in different orders. The protocol also supports fragmented messages. So RequestMessage and ReplyMessage can be split into several fragments. The first fragment is the normal message with a set more-fragments-flag. Each following FragmentMessage has the more-fragments-flag also set to true. The last MessageFragment has the flag set to false.

top

Setting up Grizzly Framework on Client and Server


Some boilerplate code needs to be written first to basically give grizzly framework connection information and plugin the example ProtocolParser which recreates the transmitted messages.

Note: The following code samples just show key points. Please download grizzly-framework-samples package for complete working code and complete context.

A common Grizzly idiom is to provide a lot of default behavior on its components. These components are themselves extendable and have many configuration properties. The following CustomProtocolClient configures Grizzly to work with the tutorial's example protocol.

Setting up the Client



Snipplet: public class CustomProtocolClient {...}
Controller controller = new Controller(); //  see 1)

TCPSelectorHandler selectorHandler = new TCPSelectorHandler(true); // true for Client side
selectorHandler.setSelectionKeyHandler(new BaseSelectionKeyHandler());//  see 2)
controller.addSelectorHandler(selectorHandler);

DefaultProtocolChain protocolChain = new DefaultProtocolChain();// see 3)
protocolChain.addFilter(CustomProtocolParser.createParserProtocolFilter( null); //  see 4)
protocolChain.addFilter(new SampleDispatcher()); //  see 5)


TCPConnectorHandler connectorHandler=(TCPConnectorHandler) controller.acquireConnectorHandler(Controller.Protocol.TCP);
.
.
callbackHandler =new CallbackHandler {...}; //see 6)
connectorHandler.connect(address, callbackHandler,selectorHandler); // see 7)

Code Comments

  1. Controller administers all Grizzly components which need to collaborate together to provide the Grizzly enhanced NIO Service
  2. The default Grizzly behavior is to close a client server socket after 30 secs. Since a connection should be permanent,
    Controller will be given a new SelectorHandler with an adjusted Keyhandler
  3. The BaseSelectionKeyHandler has no default Timeout. So now connections will be long lasting
  4. The ProtocolChain will be invoked every time chunks of bytes arrive on the connection
  5. The example CustomProtocolClient filter will parse these chunks back into messages
  6. The example SampleDispatcher filter will invoke the businesslogic layer
  7. Since the connection will be non blocking a CallbackHandler is defined
  8. After the Controller has been setup and started a ConnectorHandler uses a CallbackHandler to build a connection.
    This ConnectorHandler will also be used to write messages to the server

top

When setting up a client to be non blocking a CallbackHandler needs to be provided to Grizzly. Broadly Grizzly does not change the events native java NIO exposes. So when a connection is ready to be processed
NIO produces Connect, Read, Write Events and Grizzly invokes the equivalent CallbackHandler On-Methods.



Snipplet: public class CallbackHandler {...}
 public void onRead(IOEvent ioEvent) {   
    Context ctx = ioEvent.attachment();
    .
    .	
    ctx.getProtocolChain().execute(ioEvent.attachment()); // see 1)
	
}
public void onWrite(IOEvent ioEvent) {
   // see 2)
}


Code Comments

  1. On the client side Grizzly default behavior is to not use a ProtocolChain at all. For this reason we need to invoke ProtocolChain from onRead
  2. Client will handle writes with an ConnectorHandler. So this method stays empty


Setting up the Server


Setting up a server is even simpler. Using a ProtocolChain is the default so no need for a CallbackHandler. The server will reuse the client example CustomProtocolParser. Of course it needs to also add a server filter which invokes buisnesslogic.


Snipplet: public class Server {...}
   Controller controller = new Controller();
   TCPSelectorHandler tcpSelectorHandler = new TCPSelectorHandler(); // server side

   BaseSelectionKeyHandler keyHandler = new BaseSelectionKeyHandler();
   tcpSelectorHandler.setSelectionKeyHandler(keyHandler);
   tcpSelectorHandler.setPort(SERVER_LISTEN_PORT);
   controller.addSelectorHandler(tcpSelectorHandler);
   final DefaultProtocolChain protocolChain = new DefaultProtocolChain();
   protocolChain.addFilter(CustomProtocolParser.createParserProtocolFilter(null));
   protocolChain.addFilter(new ServerDispatcher()); // Filter with Server Businesslogic
   protocolChain.setContinuousExecution(true);

top

Getting work done with Protocol Filters


WorkerThread


Now because of the above configuration when NIO signals "bytes are ready to be received" Grizzly will start a ProtocolChain. The chain will not be run on Grizzly's main Thread.
Instead the chained Protocol Filters will be run by a WorkerThread.
One special Property of WorkerThread is its ByteBuffer in which a Filter called ReadFilter stores bytes it reads of the connection.

Snipplet: public abstract class WorkerThread extends Thread {}
public abstract class WorkerThread extends Thread {
  protected ByteBuffer byteBuffer;
   public void setByteBuffer(ByteBuffer byteBuffer){ // see 1)
        this.byteBuffer = byteBuffer;
    }
    public ByteBuffer getByteBuffer(){ // see 2)
        return byteBuffer;
    }
	 public void run(){        
        // calls Filters in ProtocolChain
	}
...
}


Code Comments

  1. The to be discussed CustomProtocolParser uses setByteBuffer(..) to hand Grizzly a new clean ByteBuffer
  2. A Filter can always get hold of the current ByteBuffer

Parsing bytes to Messages


Note: Please see Scott Oaks's Blog on Grizzly Protocol Parsers for an detailed explanation on ProtocolParser and ParserProtocolFilter basics

The CustomProtocolParser works hand in hand with a Grizzly ParserProtocolFilter. A ParserProtocolFilter extends ReadFilter which reads any waiting bytes into WorkerThread's ByteBuffer.

After the read has been done this ByteBuffer might contain :
  1. just a few bytes of the header
  2. a chunk of a packet
  3. a whole packet
  4. several packets
The CustomProtocolParser must cope with each scenario. Here is the ProtocolParser contract that CustomProtocolParser needs to fulfill:

Snipplet: public class CustomProtocolParser implements ProtocolParser{}
 protected MessageBase msg;
 
 protected ByteBuffer byteBuffer;
 
  public void startBuffer(ByteBuffer byteBuffer) {

  }
  public boolean hasNextMessage() {

  }
  public boolean isExpectingMoreData() {

  }
  public MessageBase getNextMessage() {

  }
  public boolean releaseBuffer() {
  }

Let's discuss two scenarios which might happen when parsing bytes:

  1. Connection has bytes ready to be read and these bytes would makes up several packets
  2. Connection has bytes ready to be read but not enough bytes for a full packet

Scenario 1 : Connection has bytes ready to be read and these bytes would make up several protocol packets
Action CustomProtocolParser explanation
newProtocolParser() Because CustomProtocolParser has been set up as a statefull ProtocolParser a new Instance CustomProtocolParser will be created
isExpectingMoreData() returns true and ParserProtocolFilter reads in the bytes into WorkerThread's Bytebuffer. So now the ByteBuffer consists of several protocol packets
startBuffer(ByteBuffer) ParserProtocolFilter signals that bytes have been read into the given byteBuffer
hasNextMessage() Tries to parse an protocol header

  • check if protocol magic is ok
    • if not construct am MessageError and return true
  • parse packet-size,request-id, message-type
    • construct depending on parsed type an RequestMessage, ReplyMessage, FragmentMessage
  • check if Bytebuffer contains the packet-size amount of bytes
    • here in scenario 1 finds out yes and returns true
getNextMessage()
  • Create a slice of the current ByteBuffer and add it to the Message constructed in hasNextMessage().
    • the sliced ByteBuffer is a shared copy of WorkThread's ByteBuffer which holds the actual payload of the send packet (data without protocol header)
    • CustomProtocolParser marks the payload endposition so that the next parsing of WorkThread's ByteBuffer will start at endposition + 1
  • Now return the Message
Next filter in ProtocolChain
will be executed
Grizzly will store the Message in the current ProtocolChain's Context. The example's MessageDispatcher will retrieve this Message and invoke the example's buisnesslogic
hasMoreBytesToParse() returns true because there is another packet in the current ByteBuffer
isExpectingMoreData() returns false because there is still another packet in the current ByteBuffer
hasNextMessage() Tries to parse the next protocol header



Scenario 2 : Connection has bytes ready to be read but not enough bytes for a full packet
newProtocolParser() same as in Scenario 1
isExpectingMoreData()
startBuffer(ByteBuffer)
hasNextMessage() first part same as in Scenario 1

  • parse packet-size,request-id, message-type
    • construct depending on parsed type an RequestMessage, ReplyMessage, FragmentMessage
  • check if Bytebuffer contains the packet-size amount of bytes
    • here in scenario 2 finds out that byteBuffer contains less bytes than needed to complete Message
    • now checks if ByteBuffer has a enough capacity left to receive all missing bytes
    • If not the current WorkThread's ByteBuffer will be given to the constructed Message and the current WorkThread is allocated a new empty ByteBuffer which has enough capacity to receive all missing bytes
    • Now returns false because Message is not yet ready
hasMoreBytesToParse() returns false because current ByteBuffer is empty
isExpectingMoreData() returns true
releaseBuffer() Because this CustomProtocolParser holds an uncompleted Message releaseBuffer must return true. Grizzly will then make sure that when bytes get read in on the current connection this same instance of CustomProtocolParser will be invoked
isExpectingMoreData() returns true
ReadFilter
will be executed
Tries to read in the missing Bytes
startBuffer(ByteBuffer) ParserProtocolFilter signals to our saved CustomProtocolParser (see releaseBuffer) that bytes have been read into the given byteBuffer
hasNextMessage() returns true because Bytebuffer contains now the missing packet-size amount of bytes.
See Scenario 1 for next Actions

Client sending a Request

Sending data to the server with the example client is very similar to standard Socket IO :
 RemoteCall call= customProtocolClient.callRemote();
 OutputStream out=call.getOutputStream();
 // write something to out
 InputStream in =call.getInputStream();
 // read something in
}
If a Client wants to send data to its server it fetches an RemoteCall Object by calling callRemote on CustomProtocolClient. RemoteCall provides an RemoteInputStream and a ProtocolOutputStream. Now when client writes out bytes the ProtocolOutputStream automatically adds an new protocol header to the stream if the client either explicitly closes the stream or the number of written bytes exceed (8192 - (Protocol header)). The InputStream is actually an RemoteInputStream that streams the reply of an ReplyMessage to the client. The InputStream simply blocks until bytes start to arrive.

Server receiving a Request

The server plugs into message processing by subclassing the example MessageDispatcher class. By implementing (1) and (2) server can execute its businesslogic and use Context to reply on the connection.

Snipplet: public abstract class MessageDispatcher implements ProtocolFilter{}

 abstract public void onRequestMessage(RequestMessage msg, Context ctx); // see (1) 

 abstract public void onMessageError(MessageError msg, Context ctx); // see (2)

 private void dispatch(final Message msg, final Context ctx) { 
       ctx.incrementRefCount(); // see (3)
	   
        executorService.execute(new Runnable() {
            public void run() {
                switch (msg.getMessageType()) {
                    case Message.Message_Request:
                        onRequestMessage((RequestMessage) msg, ctx);
                        break;
                    case Message.Message_Error:
                        onMessageError((MessageError) msg, ctx);
                        break;  
                }
                workerCtx.getController().returnContext(ctx); // see (4)
            }
        });
    }

WorkerThread invokes MessageDispatcher on the ProtocolChain. It will then call MessageDispatcher.dispatch(..) with the ProtocolChain's Context Object. Briefly a Context has all kinds of abilities Filters might need to communicate with the Grizzly Framework.

Here are just a couple of Context's features:

In the above code the server's businesslogic (1) and (2) will be executed on a new Thread. But Context is a Grizzly pooled resource. To prevent the WorkerThread from releasing Context.incrementRefCount (3) must be called. This ensures that Context won't be recycled. In (4) businesslogic Thread then marks Context to be released.

Note: Since example client also registers a MessageDispatcher on the ProtocolChain it also acts as a server that can handle RequestMessages.

top

Summary


By touching some of the main points the reader should now have a kickstart into understanding the Grizzly framework filter example. Readers interested in the example should just check out from subversion the latest Grizzly version and execute com.sun.grizzly.standalone.framework.Main. Hopefully this tutorial helps to understand the following points a little better:

top


See Also


Here are some further links.