Showing posts with label compress. Show all posts
Showing posts with label compress. Show all posts

Sunday, January 9, 2011

Servlet-GSON vs JSONME-JavaME III

This is the third post about JSON and JavaME. In previous releases: first we introduced JSON for JavaMe (J2ME), then we improved the example by adding compression/decompression support. Now, we are going to introduce Base64 format support for sending images in our JSON example.
Although this is not a best practice, because you should send the image's URL  instead of sending the image itself in the JSON String, sometimes you just need to do exactly that.

Requirements:

Servlet-GSON vs JSONME-JavaME: Contains the original classes. More info:
http://www.java-n-me.com/2010/11/servlet-gson-vs-jsonme-javame.html

Servlet-GSON vs JSONME-JavaME II: Second part of the saga. Adds compression/decompression support. More info:
http://www.java-n-me.com/2010/12/servlet-gson-vs-jsonme-javame-ii.html

Base64 encode-decode in JavaMe: Explains how to use Base64 format in JavaMe (J2ME). More info:
http://www.java-n-me.com/2010/12/base64-encode-decode-in-javame.html


You should check the previous requirements if you haven't, because in this post we use some of the classes defined there.

The image we are going to send is in PNG format:



OK, let's begin, first we are going to show you the changes in some of the original classes. As usual, let's start with the server side:

import java.awt.Image;
public class Client {

   /** Name of the client*/
    private String name;

    /** Last name of the client*/
    private String lastName;

    /** Identification of the client*/
    private String id;

    /** Photo of the client*/
    private transient Image photo;


    /** Photo of the client in Base 64 format*/
    private  String photoBase64;


    //Getters and Setters...
}

Our Client class has now two new attributes for the photograph of the client. The java.awt.Image attribute binds to the photograph of the client directly. The String attribute photoBase64 contains the Base64 format of the photograph that is going to be sent in the JSON String. Notice that the attribute photo is marked as transient, meaning that GSON won't use this attribute in the JSON String.

Following is the servlet class which will create a client with a photograph, will get the JSON String representation of the client and his photograph and will compress the data before sending the response to the mobile client. The servlet uses the GSON API, the java.util.zip.GZIPOutputStream and the JSE sun.misc.BASE64Encoder class:


import com.google.gson.Gson;
import java.io.*;
import java.net.URL;
import java.util.Vector;
import java.util.zip.GZIPOutputStream;
import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.*;
import javax.swing.ImageIcon;
import model.Client;
import sun.misc.BASE64Encoder;

public class ClientsServlet extends HttpServlet {
   ...

    /**
     * Handles the HTTP <code>POST</code> method.
     * @param request servlet request
     * @param response servlet response
     * @throws ServletException if a servlet-specific error occurs
     * @throws IOException if an I/O error occurs
     */
    protected void doPost(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        //sets the type of response and the charset used
        //Be careful, in your mobile client, read the response using the same charset
        //if sending the response as text. If you sent it as binary data,
        //there is no problem
        response.setContentType("application/json; charset=UTF-8");

        //create some clients and add them to the vector
        Vector vClients = new Vector();

        //used to convert Image to Base64 format
        BASE64Encoder encoder = new BASE64Encoder();

        Client clientOne = new Client();
        clientOne.setName("Alexis");
        clientOne.setLastName("López Ñ");
        clientOne.setId("123456");
        clientOne.setPhoto(new ImageIcon("images/duke.png").getImage());
        //get the image bytes. Read it as an URL
        byte[] imgBytes =
          getImageBytes("http://localhost:8080/GSON_Servlet/images/duke.png");
        String imageCoded = encoder.encode(imgBytes);
        clientOne.setPhotoBase64(imageCoded);

        //... add more clients...
      
        vClients
           .add(clientOne);

        //convert the clients vector to JSON using GSON, very Easy
        Gson gson = new Gson();
        String jsonOutput = gson.toJson(vClients);

        System.out.println("*****JSON STRING TO RESPONSE*****");
        System.out.println(jsonOutput);
        System.out.println("*********************************");

        System.out.println("Length of String in bytes = "
                                    + jsonOutput.getBytes().length);
        //compress de data
        byte[] compressed = compress(jsonOutput);
        System.out.println("Length of compressed bytes = "
                                    + compressed.length);

        //send the binary response
        ServletOutputStream out = response.getOutputStream();
        out.write(compressed, 0, compressed.length);
        out.flush();
        out.close();
    }
    ...
}

The servlet uses two important methods. One is the +compress(String):byte[] that we have analized in the previous post of this saga. The other one is the +getImageBytes(String):byte[] which is shown next:

/** * Returns the binary representation of the image passed as an URL * @param imgURL URL String of the image * @return array of binary data representing the image * @throws IOException */ public static byte[] getImageBytes(String imgURL) throws IOException { URL imgUrl = new URL(imgURL); ByteArrayOutputStream bout = new ByteArrayOutputStream(); InputStream in = imgUrl.openStream(); int i = 0; while ((i = in.read()) != -1) { bout.write(i); } in.close(); bout.close(); byte[] bytes = bout.toByteArray(); return bytes; }

The last method reads the image from the URL and gets its binary representation. One important thing to keep in mind, is that you should use PNG Images as it is the default format for JavaMe (J2ME) MIDP applications, not doing so may arise exceptions on the mobile client when parsing the Image data. When the servlet is run and it receives a POST request, the following message is shown on the console output:

INFO: *****JSON STRING TO RESPONSE***** 9/01/2011 06:11:23 PM INFO: [{"name":"Alexis","lastName":"López Ñ","id":"123456","photoBase64":"iVBORw0KGgoAAAANSUhEUgAAAP8AAADICAMAAAAQqcftAAAAAXNSR0IArs4c6QAAAwBQTFRFDAAA\r\nCgQKGAAAEAgMGAAIGAAQGAgIGAgQEBAIABAQCBAQEBAQEBAYEBgQEBgYECEYGAgYGBAIGBAQGBAY\r\nGBgIGBgQGBgYGBghGCEYIQAAIQAIIQAQIQgQIQgYIRAQIRAYKQAMKQgQKQAYIRAhIRgQIRgYIRgh\r\nISEYMQAILgUVNwQSQgAYKRAUKRgYKSEYPA0YISEhKRAhKRghKRgpMRQhMRgpPQghPRQhKSEhKSkh\r\nISEpKSYrMSUlNi4pOScvOzU1SBwpRkI9WgYgbBk1RkJKTkdHVkhNVFJMUlJaWlRUXlhaZVtdb2Fk\r\ncWxne2lwf3h5yBUsqDBU1Bcx0SRJ3BU53xs53yJK4TJjin5"
... not all data shown ... "}] INFO: *********************************



INFO: Length of String in bytes = 17425
INFO: Length of compressed bytes = 12890

As you can see, the JSON representation of the client now has the photograph in Base64 format in order to send the image data to the mobile client as part of the JSON String. Also notice that there is some compression in the response to the client.

On the mobile client side, there are also changes in the Client class:

import java.util.Vector;
import javax.microedition.lcdui.Image;
import org.bouncycastle.util.encoders.Base64;
import org.json.me.JSONArray;
import org.json.me.JSONException;
import org.json.me.JSONObject;

public class Client {


    /** Name of the client*/
    private String name;

    /** Constant name of the attribute name*/
    private static final String ATT_NAME = "name";

    /** Last name of the client*/
    private String lastName;

    /** Constant name of the attribute last name*/
    private static final String ATT_LAST_NAME = "lastName";

    /** Identification of the client*/
    private String id;

    /** Constant name of the attribute id*/
    private static final String ATT_ID = "id";

    /** Photo of the client*/
    private  Image photo;

    /** Photo of the client in Base 64 format*/
    private  String photoBase64;


     /** Constant name of the attribute photoBase64*/
    private static final String ATT_PHOTO_BASE_64 = "photoBase64";

    //Getters and Setters...

    /**
     * This method should be used by this class only, that's why it is private.
     * Allows to get a JSONObject from the Client passed as parameter
     * @param client Client Object to convert to JSONObject
     * @return JSONObject representation of the Client passed as parameter
     */
    private static JSONObject toJSONObject(Client client) {
        JSONObject json = new JSONObject();
        try {
            json.put(ATT_NAME, client.getName());
            json.put(ATT_LAST_NAME, client.getLastName());
            json.put(ATT_ID, client.getId());
            //only if the client has a photo associated send it as JSON
            if(client.getPhotoBase64() != null)
            {
                json.put(ATT_PHOTO_BASE_64, client.getPhotoBase64());
            }
        } catch (JSONException ex) {
            ex.printStackTrace();
        }
        return json;
    }

    /**
     * Allows to get a Client Object from a JSON String
     * @param jsonText JSON String to convert to a Client Object
     * @return Client Object created from the String passed as parameter
     */
    public static Client fromJSON(String jsonText) {
        Client client = new Client();
        try {
            JSONObject json = new JSONObject(jsonText);

            //check if the JSON text comes with the attribue Name
            if (json.has(ATT_NAME)) {
                //asign the String value of the attribute name to the client
                client.setName(json.getString(ATT_NAME));
            }

            //check if the JSON text comes with the attribue last name
            if (json.has(ATT_LAST_NAME)) {
                client.setLastName(json.getString(ATT_LAST_NAME));
            }

            //check if the JSON text comes with the attribue id
            if (json.has(ATT_ID)) {
                client.setId(json.getString(ATT_ID));
            }

             //check if the JSON text comes with the attribue photo
            if (json.has(ATT_PHOTO_BASE_64)) {
                client.setPhotoBase64(json.getString(ATT_PHOTO_BASE_64));


                //once the photo in base 64 format has been read,
                //convert it to an Image. IMPORTANT: PNG image format or
                //you may get an Exception
                byte[] decoded = Base64.decode(client.getPhotoBase64());
                Image img = Image.createImage(decoded, 0, decoded.length);
                client.setPhoto(img);
            }
        } catch (JSONException ex) {
            ex.printStackTrace();
        }
        return client;
    }
}

You can find the changes in bold. The important changes are the adition of the attributes for managing the photograph and the modification of the methods for parsing the JSON String to and from an Object. Notice the use of the org.bouncycastle.util.encoders.Base64 class to decode the Base64 format data of the photograph. You can find the original source code of the Client class (mobile client side) in the first post of this saga.

Next is the MIDlet that runs the code:

import com.tinyline.util.GZIPInputStream;
import java.io.*;
import java.util.Vector;
import javax.microedition.io.*;
import javax.microedition.lcdui.Display;
import javax.microedition.lcdui.Form;
import javax.microedition.midlet.*;


public class JSONMIDlet extends MIDlet {


    private Form f = null;


    public JSONMIDlet() {
        //form for the photos
        f = new Form("Photo testing");
    }


    public void pauseApp() {
    }


    public void destroyApp(boolean unconditional) {
        notifyDestroyed();
    }


    public void startApp() {
        try {
            //ask for the clients
            Vector clients = getClients();


            //show information of the clients
            System.out.println("*****CLIENT INFORMATION*****");
            for (int i = 0; i < clients.size(); i++) {
                Client cl = (Client) clients.elementAt(i);


                System.out.println("Client " + (i + 1) 
                                                + " name = " + cl.getName());
                System.out.println("Client " + (i + 1) 
                                                + " last name = " + cl.getLastName());
                System.out.println("Client " + (i + 1) 
                                                + " ID = " + cl.getId());
                System.out.println("Client " + (i + 1) 
                                                + " Photo Base64 = " + cl.getPhotoBase64());
                System.out.println("");


                //if the client has a photo, append it to the form
                if (cl.getPhoto() != null) {
                    f.append(cl.getPhoto());
                }
            }




            Display dsp = Display.getDisplay(this);
            dsp.setCurrent(f);




        } catch (IOException ex) {
            //manage the exception
            ex.printStackTrace();
        }
    }


...//other methods


}

The method +getClients():Vector has not changed and can be found in the last post of this saga (as there is compression/decompression support). Finally the emulator shows the following:


Well, that's it for today. Quite a large post, I hope I was clear enaugh, but if you have any questions do not hesitate to contact me.

see ya soon!


References:

TinyLine Utils. 2010. TinyLine [online].
Available on Internet: http://www.tinyline.com/utils/index.html
[accessed on December 11 2010].

Using JavaScript Object Notation (JSON) in Java ME for Data Interchange. Agosto 2008. Oracle [online].
Available on Internet: http://java.sun.com/developer/technicalArticles/javame/json-me/
[accessed on October the 26th 2010].

meapplicationdevelopers: Subversion. 2007. Oracle [online].
Available on Internet: https://meapplicationdevelopers.dev.java.net/source/browse/meapplicationdevelopers/demobox/mobileajax/lib/json/
[accessed on October the 27th 2010].

The Legion of the Bouncy Castle. bouncycastle.org [online].
Available on Internet: http://www.bouncycastle.org/
[accessed on December 28 2010].

Saturday, December 25, 2010

Servlet-GSON vs JSONME-JavaME II

This is the third and last part of our compression saga in Java Me (J2ME). Right until now, we have seen two libraries: TinyLine Utils (Decompress only) and JAZZLIB (Compress/Decompress). You can use whichever you want based on the requirements of your project.

In this post, we are going to improve our Servlet-GSON vs JSONME-JavaME example. We are adding compression support to the server side and decompression support on mobile client side. We will calculate the size of the data to be sent over the network before and after compression. This is helpful, when you don't have unlimited data connection on the mobile client and you need to reduce costs.

So, let's begin with the server side, the following code is the compression method used in our servlets project:

import java.util.zip.GZIPOutputStream;
...
    /**
     * Compress the String parameter
     * @param message
     * @return byte array of compressed data
     * @throws IOException if any error
     */
    public static byte[] compress(String message) throws IOException {
        ByteArrayOutputStream out = new ByteArrayOutputStream();
        GZIPOutputStream gz = new GZIPOutputStream(out);

        gz.write(message.getBytes());
        gz.flush();
        gz.close();

        out.flush();
        byte[] array = out.toByteArray();
        out.close();

        return array;

    }
...


As we are on server side, we can use the java.util.zip.GZIPOutputStream class in order to compress the data. Next, is the servlet code we have seen before but now it uses the compress method:

...
    /**
     * Handles the HTTP <code>POST</code> method.
     * @param request servlet request
     * @param response servlet response
     * @throws ServletException if a servlet-specific error occurs
     * @throws IOException if an I/O error occurs
     */
    protected void doPost(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        //sets the type of response and the charset used
        //Be careful, on your mobile client, read the response using the same charset
        response.setContentType("application/json; charset=UTF-8");

        //create some clients and add them to the vector
        Vector vClients = new Vector();

        Client clientOne = new Client();
        clientOne.setName("Alexis");
  //Note de special characters to be transmitted
        clientOne.setLastName("López Ñ");
        clientOne.setId("123456");

        vClients.add(clientOne);

        Client clientTwo = new Client();
        clientTwo.setName("Second");
        clientTwo.setLastName("Client");
        clientTwo.setId("987534");

        vClients.add(clientTwo);

        Client clientThree = new Client();
        clientThree.setName("Colombia");
        clientThree.setLastName("JUG");
        clientThree.setId("555555");

        vClients.add(clientThree);

        //convert the clients vector to JSON using GSON, very Easy
        Gson gson = new Gson();
        String jsonOutput = gson.toJson(vClients);

        System.out.println("*****JSON STRING TO RESPONSE*****");
        System.out.println(jsonOutput);
        System.out.println("*********************************");

  System.out.println("Length of String in bytes = " 
                                    + jsonOutput.getBytes().length);
        //compress de data
        byte[] compressed = compress(jsonOutput);
        System.out.println("Length of compressed bytes = " 
                                    + compressed.length);

        //send the binary response
        ServletOutputStream out = response.getOutputStream();
        out.write(compressed, 0, compressed.length);
        out.flush();
        out.close();
    }
...


As you can see, is the same method used in the first version of our Servlet-GSON vs JSONME-JavaME example, but now it uses the compress method before writing the response. Another difference, is that this time we are writing binary data so we use the javax.servlet.ServletOutputStream. As we are writing binary data, there is no need to be careful with the charset when reading the response on the mobile client side. Following is the output from the previous code:


INFO: *****JSON STRING TO RESPONSE*****
INFO: [{"name":"Alexis","lastName":"López Ñ","id":"123456"},{"name":"Second","lastName":"Client","id":"987534"},{"name":"Colombia","lastName":"JUG","id":"555555"}]
INFO: *********************************

INFO: Length of String in bytes = 157
INFO: Length of compressed bytes = 114


From the previous output, you can see that there is some compression. To be more accurate you can use a sniffer or protocol analizer (such as wireshark) in order to detect the packages sent over the network and to get the exact amount of bytes sent from the server to the mobile client.

Now let's check the decompression method on the mobile client side:

import com.tinyline.util.GZIPInputStream;
//You can also use net.sf.jazzlib.GZIPInputStream;
...
   /**
     * Decompress the data received in the Stream.
     * @param in <code>InputStream</code> of the connection where the
     * compressed data is received.
     * @return <code>String</code> representing the data decompressed
     * @throws IOException if there is any error during reading
     */
    public static String deCompress(InputStream in) throws IOException {
        StringBuffer sb = new StringBuffer();

        GZIPInputStream gz = null;
        try {
            gz = new GZIPInputStream(in);
            int c = 0;
            while ((c = gz.read()) != -1) {
                sb.append((char) c);
            }
        } finally {
            if (gz != null) {
                gz.close();
            }
        }

        return sb.toString();
    }
...

As we are only decompressing data, you can use any of the two libraries we have evaluated before. Next is the method that connects to the servlet, opens the InputStream and maps JSON Strings to Objects:

...
   /**
     * Connects to the server in order to obtain information of the clients
     * @return Vector of <code>Client</code> objects
     * @throws IOException if errors with the connections
     */
    public Vector getClients() throws IOException {
        //This is my servlet’s URL
        String URL = "http://localhost:8080/GSON_Servlet/ClientsServlet";
        Vector vClients = new Vector();

        HttpConnection http = null;
        InputStream in = null;
        try {
            //Open HttpConnection using POST
            http = (HttpConnection) Connector.open(URL);
            http.setRequestMethod(HttpConnection.POST);
            //Content-Type is must to pass parameters in POST Request
            http.setRequestProperty("Content-Type",
                                                 "application/x-www-form-urlencoded");

            //decompress the data
            in = http.openInputStream();


            //IMPORTANT: As the servlet wrote the data using a binary stream,
            //there is no need to use a special charset
            String jSonString = deCompress(in);

            System.out.println("*****JSON STRING RECEIVED*****");
            System.out.println(jSonString);
            System.out.println("******************************");

            //Parse the JSON String to obtain a Vector of clients
            vClients = Client.fromJSONs(jSonString);

        } //whatever happens, close the streams and connections
        finally {
            if (in != null) {
                in.close();
            }

            if (http != null) {
                http.close();
            }
        }

        return vClients;
    }
...

As said, the previous code connects to the server, reads the response, decompress the response and maps JSON Strings to Objects. The output is shown next, check that the special characters are ok even if we didn't used a special charset when reading the response from the server:


*****JSON STRING RECEIVED*****
[{"name":"Alexis","lastName":"López Ñ","id":"123456"},{"name":"Second","lastName":"Client","id":"987534"},{"name":"Colombia","lastName":"JUG","id":"555555"}]
******************************
*****CLIENT INFORMATION*****
Client 1 name = Alexis
Client 1 last name = López Ñ
Client 1 ID = 123456

Client 2 name = Second
Client 2 last name = Client
Client 2 ID = 987534

Client 3 name = Colombia
Client 3 last name = JUG
Client 3 ID = 555555


Ok, that's it for this post. I hope I was clear enough, if you have any comments or questions do not hesitate to contact me or to leave a comment.

see ya soon!


References:

A pure java implementation of java.util.zip library. 2010. SourceForge [online].
Available on Internet: http://jazzlib.sourceforge.net/
[accessed on December 20 2010].

Jazzlib Java Me. November 2010. STAFF [online].
Available on Internet: http://code.google.com/p/staff/downloads/list?q=jazzlib
[accessed on December 20 2010].

TinyLine Utils. 2010. TinyLine [online].
Available on Internet: http://www.tinyline.com/utils/index.html
[accessed on December 11 2010].

Using JavaScript Object Notation (JSON) in Java ME for Data Interchange. Agosto 2008. Oracle [online].
Available on Internet: http://java.sun.com/developer/technicalArticles/javame/json-me/
[accessed on October the 26th 2010].

meapplicationdevelopers: Subversion. 2007. Oracle [online].
Available on Internet: https://meapplicationdevelopers.dev.java.net/source/browse/meapplicationdevelopers/demobox/mobileajax/lib/json/
[accessed on October the 27th 2010].

Monday, December 20, 2010

Compression in JavaMe II

This is the second part of our compression saga in Java Me. In the previous post, we used the TinyLine GZIPInputStream to decompress data from an InputStream. It is useful when your application receives more information than it sends. But, what happens if you still need to compress data on the mobile client side before sending it to the server?

There is a library called JAZZLIB which is an implementation of the JSE java.util.zip package that uses pure Java, no native code. You can get more information about this JSE library in the following link:


The last link takes you to the JSE JAZZLIB library, for a Java Me implementation that comes with the GZIPInputStream and the GZIPOutputStream streams to decompress or compress data, follow this link:


Once you have imported tha library in your project, you can start compressing or decompressing directly on the mobile client side. The example of this post will focus only on two methods: One for the compression and the other for the decompression.

The following code represents the decompression method, you can realize that is the same decompression method used in the previous post, the only change is that this time we import the net.sf.jazzlib.GZIPInputStream class instead of the com.tinyline.util.GZIPInputStream class, but the method's java code doesn't change.

import net.sf.jazzlib.GZIPInputStream;
...
    /**
     * Decompress the data received in the Stream.
     * The stream is not closed in this method.
     * @param in <code>InputStream</code> of the connection where the
     * compressed data is received.
     * @return <code>String</code> representing the data decompressed
     * @throws IOException if there is any error during reading
     */
    public static String deCompress(InputStream in) throws IOException {
        StringBuffer sb = new StringBuffer();

        GZIPInputStream gz = new GZIPInputStream(in);
        int c = 0;
        while ((c = gz.read()) != -1) {
            sb.append((char) c);
        }

        gz.close();

        return sb.toString();
    }
...

The InputStream parameter could be one of:

  • +HttpConnection.openInputStream():InputStream
  • +SocketConnection.openInputStream():InputStream
  • +InputConnection.openInputStream():InputStream

Another approach to decompress data is to receive a compressed array of bytes instead of a stream as the following:

import net.sf.jazzlib.GZIPInputStream;
...
   /**
     * Decompress the data received in the array of bytes.
     * @param bytes data array compressed
     * @return <code>String</code> representing the data decompressed
     * @throws IOException if there is any error during reading
     */
    public static String deCompress(byte[] bytes) throws IOException {
        StringBuffer sb = new StringBuffer();

        ByteArrayInputStream in = new ByteArrayInputStream(bytes);
        GZIPInputStream gz = new GZIPInputStream(in);
        int c = 0;
        while ((c = gz.read()) != -1) {
            sb.append((char) c);
        }

        gz.close();

        return sb.toString();
    }
...

Notice that the net.sf.jazzlib.GZIPInputStream decompress on the fly, you don't have to invoke any other method than +read():int.

Now we want to compress data using the net.sf.jazzlib.GZIPOutputStream class. It is not difficult either check the following code:

import net.sf.jazzlib.GZIPOutputStream;
...
   /**
     * Compress the data received as a String message.
     * @param message <code>String</code> to be compressed
     * @return <code>String</code> representing the data compressed
     * @throws IOException if there is any error during writing
     */
    public static String compress(String message) throws IOException {
        ByteArrayOutputStream out = new ByteArrayOutputStream();
        GZIPOutputStream gz = new GZIPOutputStream(out);
        byte[] bytes = message.getBytes();
        gz.write(bytes, 0, bytes.length);

        //IMPORTANT, acording to the JAZZLIB API documentation,
        //the +close():void method:
        //Writes remaining compressed output data to the output stream and closes it.
        //so you BETTER invoke it, before returning the compressed array data
        gz.close();

        byte[] array = out.toByteArray();
        out.close();

        return new String(array);
    }
...

For compression we use the net.sf.jazzlib.GZIPOutputStream class. It does compression on the fly, you just have to invoke the +write(byte[],int,int):void method in order to get the data compressed. Notice that you have to invoke the +close():void method before you can get the data compressed or you will have an incomplete compressed data. Another approach to the previous code is the following:

import net.sf.jazzlib.GZIPOutputStream;
...
   /**
     * Compress the data received as a String message.
     * @param message <code>String</code> to be compressed
     * @param out OutputStream to write compressed data to.
     *  It is not closed in this method.
     * @throws IOException if there is any error during writing
     */
    public static void compress(String message, OutputStream out)
    throws IOException {
        GZIPOutputStream gz = new GZIPOutputStream(out);
        byte[] bytes = message.getBytes();
        gz.write(bytes, 0, bytes.length);

        //IMPORTANT, acording to the JAZZLIB API documentation,
        //the +close():void method:
        //Writes remaining compressed output data to the output stream and closes it.
        //so you BETTER invoke it, before returning the compressed array data
        gz.close();
    }
...

In this case, we receive the message to be compressed and the stream where the compressed data is written to. The stream can be one of:

  • +HttpConnection.openOutputStream():OutputStream
  • +SocketConnection.openOutputStream():OutputStream
  • +OutputConnection.openOutputStream():OutputStream.

Ok, so that's it for this post. Now we have seen two libraries to decompress data on the mobile client and one of them can be used to compress data as well. In the next post, we will integrate compression of data to the Servlet-GSON vs JSONMe-JavaMe example to optimize the amount of data sent over the network.

see you soon!


References:

A pure java implementation of java.util.zip library. 2010. SourceForge [online].
Available on Internet: http://jazzlib.sourceforge.net/
[accessed on December 20 2010].

Jazzlib Java Me. November 2010. STAFF [online].
Available on Internet: http://code.google.com/p/staff/downloads/list?q=jazzlib
[accessed on December 20 2010].

Saturday, December 11, 2010

Compress in JavaMe

In scenarios where the amount of transmitted data is relevant, you should probably need to do some data compression on server side before sending information to the mobile client.

On server side, there is no problem to compress data. You can use java.util.zip.GZIPInputStream or java.util.zip.GZIPOutputStream to read compressed data from a stream or to write data and compress it to a stream respectively.

On mobile client side (JavaMe), there is not such package as java.util.zip and therefore we are going to need some third party libraries in order to handle the compression. In this post we are going to use TinyLine GZIPInputStream which is part of the TinyLine Utils, you can find the binaries in the following link:


The library is about 9KB and it has only the GZIPInputStream class, there is no support for the GZIPOutputStream. But if you think about it, the major data transmition occurs from the server to the mobile client so having an InputStream is more important than having an OuputStream on the mobile client side.

Once you have added the library to your mobile application, you can make use of com.tinyline.util.GZIPInputStream class in order to receive the compressed data form the server. As I mentioned on my last post, I'm just showing the relevant code so the post can be easier to read:


...

    /**
     * Decompress the data received in the Stream. The stream is not closed in this method.
     * @param in <code>InputStream</code> of the connection where the
     * compressed data is received.
     * @return <code>String</code> representing the data decompressed
     * @throws IOException if there is any error during reading
     */
    public static String deCompress(InputStream in) throws IOException {
        StringBuffer sb = new StringBuffer();

        GZIPInputStream gz = new GZIPInputStream(in);
        int c = 0;
        while ((c = gz.read()) != -1) {
            sb.append((char) c);
        }

        gz.close();

        return sb.toString();
    }
...


It is pretty simple and it works OK. There is another library, JAZZLIB, wich implements both GZIPInputStream and GZIPOutputStream, but it takes about 41KB. If you think that your application sends as much data as it receives from the server, maybe it will be worth it to compress the data on the mobile client side too. In the next post we will make an example using this library and then we will improve our JSON example to use compression.

see ya soon!


References:

TinyLine Utils. 2010. TinyLine [online].
Available on Internet: http://www.tinyline.com/utils/index.html
[accessed on December 11 2010].