Monday, December 20, 2010

Compression in JavaMe II

This is the second part of our compression saga in Java Me. In the previous post, we used the TinyLine GZIPInputStream to decompress data from an InputStream. It is useful when your application receives more information than it sends. But, what happens if you still need to compress data on the mobile client side before sending it to the server?

There is a library called JAZZLIB which is an implementation of the JSE java.util.zip package that uses pure Java, no native code. You can get more information about this JSE library in the following link:


The last link takes you to the JSE JAZZLIB library, for a Java Me implementation that comes with the GZIPInputStream and the GZIPOutputStream streams to decompress or compress data, follow this link:


Once you have imported tha library in your project, you can start compressing or decompressing directly on the mobile client side. The example of this post will focus only on two methods: One for the compression and the other for the decompression.

The following code represents the decompression method, you can realize that is the same decompression method used in the previous post, the only change is that this time we import the net.sf.jazzlib.GZIPInputStream class instead of the com.tinyline.util.GZIPInputStream class, but the method's java code doesn't change.

import net.sf.jazzlib.GZIPInputStream;
...
    /**
     * Decompress the data received in the Stream.
     * The stream is not closed in this method.
     * @param in <code>InputStream</code> of the connection where the
     * compressed data is received.
     * @return <code>String</code> representing the data decompressed
     * @throws IOException if there is any error during reading
     */
    public static String deCompress(InputStream in) throws IOException {
        StringBuffer sb = new StringBuffer();

        GZIPInputStream gz = new GZIPInputStream(in);
        int c = 0;
        while ((c = gz.read()) != -1) {
            sb.append((char) c);
        }

        gz.close();

        return sb.toString();
    }
...

The InputStream parameter could be one of:

  • +HttpConnection.openInputStream():InputStream
  • +SocketConnection.openInputStream():InputStream
  • +InputConnection.openInputStream():InputStream

Another approach to decompress data is to receive a compressed array of bytes instead of a stream as the following:

import net.sf.jazzlib.GZIPInputStream;
...
   /**
     * Decompress the data received in the array of bytes.
     * @param bytes data array compressed
     * @return <code>String</code> representing the data decompressed
     * @throws IOException if there is any error during reading
     */
    public static String deCompress(byte[] bytes) throws IOException {
        StringBuffer sb = new StringBuffer();

        ByteArrayInputStream in = new ByteArrayInputStream(bytes);
        GZIPInputStream gz = new GZIPInputStream(in);
        int c = 0;
        while ((c = gz.read()) != -1) {
            sb.append((char) c);
        }

        gz.close();

        return sb.toString();
    }
...

Notice that the net.sf.jazzlib.GZIPInputStream decompress on the fly, you don't have to invoke any other method than +read():int.

Now we want to compress data using the net.sf.jazzlib.GZIPOutputStream class. It is not difficult either check the following code:

import net.sf.jazzlib.GZIPOutputStream;
...
   /**
     * Compress the data received as a String message.
     * @param message <code>String</code> to be compressed
     * @return <code>String</code> representing the data compressed
     * @throws IOException if there is any error during writing
     */
    public static String compress(String message) throws IOException {
        ByteArrayOutputStream out = new ByteArrayOutputStream();
        GZIPOutputStream gz = new GZIPOutputStream(out);
        byte[] bytes = message.getBytes();
        gz.write(bytes, 0, bytes.length);

        //IMPORTANT, acording to the JAZZLIB API documentation,
        //the +close():void method:
        //Writes remaining compressed output data to the output stream and closes it.
        //so you BETTER invoke it, before returning the compressed array data
        gz.close();

        byte[] array = out.toByteArray();
        out.close();

        return new String(array);
    }
...

For compression we use the net.sf.jazzlib.GZIPOutputStream class. It does compression on the fly, you just have to invoke the +write(byte[],int,int):void method in order to get the data compressed. Notice that you have to invoke the +close():void method before you can get the data compressed or you will have an incomplete compressed data. Another approach to the previous code is the following:

import net.sf.jazzlib.GZIPOutputStream;
...
   /**
     * Compress the data received as a String message.
     * @param message <code>String</code> to be compressed
     * @param out OutputStream to write compressed data to.
     *  It is not closed in this method.
     * @throws IOException if there is any error during writing
     */
    public static void compress(String message, OutputStream out)
    throws IOException {
        GZIPOutputStream gz = new GZIPOutputStream(out);
        byte[] bytes = message.getBytes();
        gz.write(bytes, 0, bytes.length);

        //IMPORTANT, acording to the JAZZLIB API documentation,
        //the +close():void method:
        //Writes remaining compressed output data to the output stream and closes it.
        //so you BETTER invoke it, before returning the compressed array data
        gz.close();
    }
...

In this case, we receive the message to be compressed and the stream where the compressed data is written to. The stream can be one of:

  • +HttpConnection.openOutputStream():OutputStream
  • +SocketConnection.openOutputStream():OutputStream
  • +OutputConnection.openOutputStream():OutputStream.

Ok, so that's it for this post. Now we have seen two libraries to decompress data on the mobile client and one of them can be used to compress data as well. In the next post, we will integrate compression of data to the Servlet-GSON vs JSONMe-JavaMe example to optimize the amount of data sent over the network.

see you soon!


References:

A pure java implementation of java.util.zip library. 2010. SourceForge [online].
Available on Internet: http://jazzlib.sourceforge.net/
[accessed on December 20 2010].

Jazzlib Java Me. November 2010. STAFF [online].
Available on Internet: http://code.google.com/p/staff/downloads/list?q=jazzlib
[accessed on December 20 2010].

19 comments:

  1. Hi! I wrote an app that compresses and decompresses data in Java ME using jazzlib. The problem is when I try to run it on a Blackberry it gives a "Symbol 'Calendar.set' not found" error. Any idea why? Thanks

    ReplyDelete
  2. Hey! Can u please give a complete sample code of J2ME implementation of the jazzlib class

    ReplyDelete
  3. Hi Anonymous, sure, send me your email and I will reply with the complete example.

    ReplyDelete
  4. Hi, I get the following error when I add the jar file in my j2me project, can you please suggest me about it?
    Error preverifying class java.util.jar.Attributes
    java/lang/NoClassDefFoundError: java/lang/Cloneable

    Thanks, Regards. -Ashraf

    ReplyDelete
  5. Hi Ashraf, I think you are trying to use a JavaSE class in a JavaME project. If this happens once you add the jazzlib jar, please verify that you have downloaded the JavaME version, here is the link:

    http://code.google.com/p/staff/downloads/list?q=jazzlib

    Regards

    ReplyDelete
    Replies
    1. Hi Alexis, Thanks a million for such prompt reply, I highly appreciate your help. Yes, not the error is gone :)

      Actually, I am reading GZIP compressed data from web service developed in c#, and trying to decompress using jazzlib.

      It is not working, showing "Error in GZIP header, second by does not match". Have you ever faced something like that.

      Thank you for your help. your blog is good. I should read new posts regularly.

      Regards,

      Ashraf
      http://www.ashrafur.com

      Delete
    2. Hi Ashraf. I sent you an email with some information, please confirm.

      Regards

      Delete
  6. This comment has been removed by the author.

    ReplyDelete
  7. Hi when I am Compressing data and Decompress data it gives .. java.io.IOException: Error in GZIP header, first byte doesn't match
    Error

    ReplyDelete
    Replies
    1. Hi Rahul. Please let me see the code you are using. One of my readers had that same problem once, it was because he was sending the compressed data over HTTP (using a web service) without encoding it.

      Hope this helps.

      Delete
    2. hii Alexis can u help me out . i have the same problem while reading string in decompress method it is throwing different exception depending upon length of string..it is very urgentt..plzz help me out

      Delete
    3. Please send me some of the code you are using. Regards.

      Delete
  8. hii, I have the same problem while using the jazzlib in j2me
    i am compressing a string using gzip the compression works fine but when i try to decompress the string it was giving a header error .but i made some changes in the code and now it is giving a unable to add Code length error
    plzz help

    ReplyDelete
    Replies
    1. Mahesh, Hi. Sure let me help you, show me some of the code you are using. Regards.

      Delete
    2. hi Alexis,
      the code is same as the one mentioned above.

      import java.io.*;
      import javax.microedition.*;
      import java.util.*;
      import net.sf.jazzlib.*;

      public class Compress
      {



      public Compress()
      {



      }

      public static String decompress(String str) throws IOException
      {
      StringBuffer sb = new StringBuffer();
      byte[] bytes=str.getBytes();
      ByteArrayInputStream in = new ByteArrayInputStream(bytes);
      GZIPInputStream gz = new GZIPInputStream(in);
      int c = 0;
      while ((c = gz.read()) != -1) {
      sb.append((char) c);
      }

      gz.close();

      return sb.toString();
      }


      public static String compres(String message) throws IOException {
      ByteArrayOutputStream out = new ByteArrayOutputStream();
      GZIPOutputStream gz = new GZIPOutputStream(out);
      byte[] bytes = message.getBytes();
      gz.write(bytes, 0, bytes.length);

      //IMPORTANT, acording to the JAZZLIB API documentation,
      //the +close():void method:
      //Writes remaining compressed output data to the output stream and closes it.
      //so you BETTER invoke it, before returning the compressed array data
      gz.close();

      String array = out.toString();
      out.close();

      return array;
      }




      }

      It is throwing an exception in while reading the characters in decompress method(crc mismatch and unable to add code length exception
      plz help

      Delete
    3. Hi Mahesh, your code looks ok. Please confirm the following:

      1. You downloaded the JavaMe version of the librari (jazzlib-j2me-0.0.7.jar) as described in my post.

      2. You are NOT receiving the String you want to decompress over Http. If the String you want to decompress is received over Http, you should be aware of encoding.

      Regards.

      Delete
    4. hi Alexis thankx for replying
      But my problem is not in compression method .the problem is in decompression
      During the while statement it is throwing an exception::
      java.io.IOException: GZIP crc sum mismatch, theirs "7440bb1c" and ours "fbca4b5b
      I have checked it out .this exception is from GzipInputStream and occurs while computing the footer part.
      and what i am doing is i am just providing the string(that i have to compress) from textbox to compres method and providing its result to decompress method
      As I am Completely New to j2me .So i dont know about the http thing u are talking about.
      Can u plz help me with some sample code

      my email id :nairmahesh23@gmail.com

      Delete
    5. OK, I sent you an email asking you to show me the code you are using to invoke the class that compress/decompress and the parameters you are using. Regards.

      Delete
  9. can any one give the information about GZIP...?
    how it actually works ..?

    ReplyDelete