Java and ME: 2010

Tuesday, December 28, 2010

Base64 encode-decode in JavaMe

I want to start this post asking: why do you think you need to encode data? At first, I didn't have a specific answer, so I googled and found some interesting answers and now I can tell you: You want to encode, because you need to transfer binary data through a channel that is designed to deal with textual data. A common example is when you need to transfer an Image inside a JSON String or in a XML message.

We will use the Bouncy Castle library to Encode/Decode binary data in Base64 format. Although this library has a lot more and can be used for Encryption/Decryption using different methods, today we will focus on the Encoding/Decoding part. You can download the library using the following link:

bouncy castle library

Make sure you download the J2ME version. When you download this version, you'll get the full source code and when you build it, you get a 1.7MB library... which may be too much for a mobile application. Anyway, with the source code you can make a build of some classes and not of all of them. That's how I made a build with only the encoders packages and classes and is just about 14KB. If you need it, ~~just ask for it~~ you can download it using this link.

To work with Base64 in JavaMe (J2ME) the bouncy castle library comes with the org.bouncycastle.util.encoders.Base64 class. Next is how you can use it in your MIDlet:

import org.bouncycastle.util.encoders.Base64;

...
   /**
   * As an example, encodes a String and then it decodes it.
   * Prints to console the result of the encode/decode.
   */
   public void encodeDecodeExample() {

   String word = "New word to encode using Base64 also special "
   + "chars like Ñ and ó";

   System.out.println("Encoding: ");
   System.out.println(word);

   byte[] coded = Base64.encode(word.getBytes());
   String strCoded = new String(coded);

   //prints the encoded word
   System.out.println("Result: ");
   System.out.println(strCoded);

   System.out.println("Decoding: ");
   System.out.println(strCoded);

   byte[] decoded = Base64.decode(strCoded);
   String strDecoded = new String(decoded);

   //prints the decoded word
   System.out.println("Result: ");
   System.out.println(strDecoded);
   }
...

When the previous code is run, the console output shows the following:

Encoding:
New word to encode using Base64 also special chars like Ñ and ó

Result:
TmV3IHdvcmQgdG8gZW5jb2RlIHVzaW5nIEJhc2U2NCBhbHNvIHNwZWNpYWwgY2hhcnMgbGlrZSDRIGFuZCDz

Decoding:
TmV3IHdvcmQgdG8gZW5jb2RlIHVzaW5nIEJhc2U2NCBhbHNvIHNwZWNpYWwgY2hhcnMgbGlrZSDRIGFuZCDz

Result:
New word to encode using Base64 also special chars like Ñ and ó

You can see that first it prints the String varible called 'word' encoded, after that it decodes the encoded word and prints the decoded word. You only need to have the byte array of data and invoke the proper methods.

This library is pretty simple to use and it gives you the power of transmit binary data as text. As we mentioned before, if you need to send, for example, an Image as text (JSON String? XML?).

This is a small part of the bouncy castle library. It has many encryption/decryption method to be used in your mobile applications. In future posts we will explore some of these methods to add security to our mobile applications.

This concludes our introduction to Base64 in JavaMe (J2ME), wait for the next post where we will use it to send images from the server to the mobile client using, once again, JSON Strings.

see ya soon!

References:

Base64. 2010. Wikipedia [online].
Available on Internet: http://en.wikipedia.org/wiki/Base64
[accessed on December 28 2010].

The Legion of the Bouncy Castle. bouncycastle.org [online].
Available on Internet: http://www.bouncycastle.org/
[accessed on December 28 2010].

Saturday, December 25, 2010

Servlet-GSON vs JSONME-JavaME II

This is the third and last part of our compression saga in Java Me (J2ME). Right until now, we have seen two libraries: TinyLine Utils (Decompress only) and JAZZLIB (Compress/Decompress). You can use whichever you want based on the requirements of your project.

In this post, we are going to improve our Servlet-GSON vs JSONME-JavaME example. We are adding compression support to the server side and decompression support on mobile client side. We will calculate the size of the data to be sent over the network before and after compression. This is helpful, when you don't have unlimited data connection on the mobile client and you need to reduce costs.

So, let's begin with the server side, the following code is the compression method used in our servlets project:

import java.util.zip.GZIPOutputStream;
...
   /**
   * Compress the String parameter
   * @param message
   * @return byte array of compressed data
   * @throws IOException if any error
   */
   public static byte[] compress(String message) throws IOException {
   ByteArrayOutputStream out = new ByteArrayOutputStream();
   GZIPOutputStream gz = new GZIPOutputStream(out);

   gz.write(message.getBytes());
   gz.flush();
   gz.close();

   out.flush();
   byte[] array = out.toByteArray();
   out.close();

   return array;

   }
...

As we are on server side, we can use the java.util.zip.GZIPOutputStream class in order to compress the data. Next, is the servlet code we have seen before but now it uses the compress method:

...
   /**
   * Handles the HTTP <code>POST</code> method.
   * @param request servlet request
   * @param response servlet response
   * @throws ServletException if a servlet-specific error occurs
   * @throws IOException if an I/O error occurs
   */
   protected void doPost(HttpServletRequest request, HttpServletResponse response)
   throws ServletException, IOException {
   //sets the type of response and the charset used
   //Be careful, on your mobile client, read the response using the same charset
   response.setContentType("application/json; charset=UTF-8");

   //create some clients and add them to the vector
   Vector vClients = new Vector();

   Client clientOne = new Client();
   clientOne.setName("Alexis");
//Note de special characters to be transmitted
   clientOne.setLastName("López Ñ");
   clientOne.setId("123456");

   vClients.add(clientOne);

   Client clientTwo = new Client();
   clientTwo.setName("Second");
   clientTwo.setLastName("Client");
   clientTwo.setId("987534");

   vClients.add(clientTwo);

   Client clientThree = new Client();
   clientThree.setName("Colombia");
   clientThree.setLastName("JUG");
   clientThree.setId("555555");

   vClients.add(clientThree);

   //convert the clients vector to JSON using GSON, very Easy
   Gson gson = new Gson();
   String jsonOutput = gson.toJson(vClients);

   System.out.println("*****JSON STRING TO RESPONSE*****");
   System.out.println(jsonOutput);
   System.out.println("*********************************");

System.out.println("Length of String in bytes = "
   + jsonOutput.getBytes().length);
   //compress de data
   byte[] compressed = compress(jsonOutput);
   System.out.println("Length of compressed bytes = "
   + compressed.length);

   //send the binary response
   ServletOutputStream out = response.getOutputStream();
   out.write(compressed, 0, compressed.length);
   out.flush();
   out.close();
   }
...

As you can see, is the same method used in the first version of our Servlet-GSON vs JSONME-JavaME example, but now it uses the compress method before writing the response. Another difference, is that this time we are writing binary data so we use the javax.servlet.ServletOutputStream. As we are writing binary data, there is no need to be careful with the charset when reading the response on the mobile client side. Following is the output from the previous code:

INFO: *****JSON STRING TO RESPONSE*****

INFO: [{"name":"Alexis","lastName":"López Ñ","id":"123456"},{"name":"Second","lastName":"Client","id":"987534"},{"name":"Colombia","lastName":"JUG","id":"555555"}]

INFO: *********************************

INFO: Length of String in bytes = 157

INFO: Length of compressed bytes = 114

From the previous output, you can see that there is some compression. To be more accurate you can use a sniffer or protocol analizer (such as wireshark) in order to detect the packages sent over the network and to get the exact amount of bytes sent from the server to the mobile client.

Now let's check the decompression method on the mobile client side:

import com.tinyline.util.GZIPInputStream;
//You can also use net.sf.jazzlib.GZIPInputStream;
...
   /**
   * Decompress the data received in the Stream.
   * @param in <code>InputStream</code> of the connection where the
   * compressed data is received.
   * @return <code>String</code> representing the data decompressed
   * @throws IOException if there is any error during reading
   */
   public static String deCompress(InputStream in) throws IOException {
   StringBuffer sb = new StringBuffer();

   GZIPInputStream gz = null;
   try {
   gz = new GZIPInputStream(in);
   int c = 0;
   while ((c = gz.read()) != -1) {
   sb.append((char) c);
   }
   } finally {
   if (gz != null) {
   gz.close();
   }
   }

   return sb.toString();
   }
...

As we are only decompressing data, you can use any of the two libraries we have evaluated before. Next is the method that connects to the servlet, opens the InputStream and maps JSON Strings to Objects:

...
   /**
   * Connects to the server in order to obtain information of the clients
   * @return Vector of <code>Client</code> objects
   * @throws IOException if errors with the connections
   */
   public Vector getClients() throws IOException {
   //This is my servlet’s URL
   String URL = "http://localhost:8080/GSON_Servlet/ClientsServlet";
   Vector vClients = new Vector();

   HttpConnection http = null;
   InputStream in = null;
   try {
   //Open HttpConnection using POST
   http = (HttpConnection) Connector.open(URL);
   http.setRequestMethod(HttpConnection.POST);
   //Content-Type is must to pass parameters in POST Request
   http.setRequestProperty("Content-Type",
   "application/x-www-form-urlencoded");

   //decompress the data
   in = http.openInputStream();

   //IMPORTANT: As the servlet wrote the data using a binary stream,
   //there is no need to use a special charset
   String jSonString = deCompress(in);

   System.out.println("*****JSON STRING RECEIVED*****");
   System.out.println(jSonString);
   System.out.println("******************************");

   //Parse the JSON String to obtain a Vector of clients
   vClients = Client.fromJSONs(jSonString);

   } //whatever happens, close the streams and connections
   finally {
   if (in != null) {
   in.close();
   }

   if (http != null) {
   http.close();
   }
   }

   return vClients;
   }
...

As said, the previous code connects to the server, reads the response, decompress the response and maps JSON Strings to Objects. The output is shown next, check that the special characters are ok even if we didn't used a special charset when reading the response from the server:

*****JSON STRING RECEIVED*****

[{"name":"Alexis","lastName":"López Ñ","id":"123456"},{"name":"Second","lastName":"Client","id":"987534"},{"name":"Colombia","lastName":"JUG","id":"555555"}]

******************************

*****CLIENT INFORMATION*****

Client 1 name = Alexis

Client 1 last name = López Ñ

Client 1 ID = 123456

Client 2 name = Second

Client 2 last name = Client

Client 2 ID = 987534

Client 3 name = Colombia

Client 3 last name = JUG

Client 3 ID = 555555

Ok, that's it for this post. I hope I was clear enough, if you have any comments or questions do not hesitate to contact me or to leave a comment.

see ya soon!

References:

A pure java implementation of java.util.zip library. 2010. SourceForge [online].
Available on Internet: http://jazzlib.sourceforge.net/
[accessed on December 20 2010].

Jazzlib Java Me. November 2010. STAFF [online].
Available on Internet: http://code.google.com/p/staff/downloads/list?q=jazzlib
[accessed on December 20 2010].

TinyLine Utils. 2010. TinyLine [online].
Available on Internet: http://www.tinyline.com/utils/index.html
[accessed on December 11 2010].

Using JavaScript Object Notation (JSON) in Java ME for Data Interchange. Agosto 2008. Oracle [online].
Available on Internet: http://java.sun.com/developer/technicalArticles/javame/json-me/
[accessed on October the 26th 2010].

meapplicationdevelopers: Subversion. 2007. Oracle [online].
Available on Internet: https://meapplicationdevelopers.dev.java.net/source/browse/meapplicationdevelopers/demobox/mobileajax/lib/json/
[accessed on October the 27th 2010].

Monday, December 20, 2010

Compression in JavaMe II

This is the second part of our compression saga in Java Me. In the previous post, we used the TinyLine GZIPInputStream to decompress data from an InputStream. It is useful when your application receives more information than it sends. But, what happens if you still need to compress data on the mobile client side before sending it to the server?

There is a library called JAZZLIB which is an implementation of the JSE java.util.zip package that uses pure Java, no native code. You can get more information about this JSE library in the following link:

http://jazzlib.sourceforge.net/

The last link takes you to the JSE JAZZLIB library, for a Java Me implementation that comes with the GZIPInputStream and the GZIPOutputStream streams to decompress or compress data, follow this link:

jazzlib-j2me-0.0.7.jar

Once you have imported tha library in your project, you can start compressing or decompressing directly on the mobile client side. The example of this post will focus only on two methods: One for the compression and the other for the decompression.

The following code represents the decompression method, you can realize that is the same decompression method used in the previous post, the only change is that this time we import the net.sf.jazzlib.GZIPInputStream class instead of the com.tinyline.util.GZIPInputStream class, but the method's java code doesn't change.

import net.sf.jazzlib.GZIPInputStream;
...
   /**
   * Decompress the data received in the Stream.
   * The stream is not closed in this method.
   * @param in <code>InputStream</code> of the connection where the
   * compressed data is received.
   * @return <code>String</code> representing the data decompressed
   * @throws IOException if there is any error during reading
   */
   public static String deCompress(InputStream in) throws IOException {
   StringBuffer sb = new StringBuffer();

   GZIPInputStream gz = new GZIPInputStream(in);
   int c = 0;
   while ((c = gz.read()) != -1) {
   sb.append((char) c);
   }

   gz.close();

   return sb.toString();
   }
...

The InputStream parameter could be one of:

+HttpConnection.openInputStream():InputStream
+SocketConnection.openInputStream():InputStream
+InputConnection.openInputStream():InputStream

Another approach to decompress data is to receive a compressed array of bytes instead of a stream as the following:

import net.sf.jazzlib.GZIPInputStream;
...
   /**
   * Decompress the data received in the array of bytes.
   * @param bytes data array compressed
   * @return <code>String</code> representing the data decompressed
   * @throws IOException if there is any error during reading
   */
   public static String deCompress(byte[] bytes) throws IOException {
   StringBuffer sb = new StringBuffer();

   ByteArrayInputStream in = new ByteArrayInputStream(bytes);
   GZIPInputStream gz = new GZIPInputStream(in);
   int c = 0;
   while ((c = gz.read()) != -1) {
   sb.append((char) c);
   }

   gz.close();

   return sb.toString();
   }
...

Notice that the net.sf.jazzlib.GZIPInputStream decompress on the fly, you don't have to invoke any other method than +read():int.

Now we want to compress data using the net.sf.jazzlib.GZIPOutputStream class. It is not difficult either check the following code:

import net.sf.jazzlib.GZIPOutputStream;
...
   /**
   * Compress the data received as a String message.
   * @param message <code>String</code> to be compressed
   * @return <code>String</code> representing the data compressed
   * @throws IOException if there is any error during writing
   */
   public static String compress(String message) throws IOException {
   ByteArrayOutputStream out = new ByteArrayOutputStream();
   GZIPOutputStream gz = new GZIPOutputStream(out);
   byte[] bytes = message.getBytes();
   gz.write(bytes, 0, bytes.length);

   //IMPORTANT, acording to the JAZZLIB API documentation,
   //the +close():void method:
   //Writes remaining compressed output data to the output stream and closes it.
   //so you BETTER invoke it, before returning the compressed array data
   gz.close();

   byte[] array = out.toByteArray();
   out.close();

   return new String(array);
   }
...

For compression we use the net.sf.jazzlib.GZIPOutputStream class. It does compression on the fly, you just have to invoke the +write(byte[],int,int):void method in order to get the data compressed. Notice that you have to invoke the +close():void method before you can get the data compressed or you will have an incomplete compressed data. Another approach to the previous code is the following:

import net.sf.jazzlib.GZIPOutputStream;
...
   /**
   * Compress the data received as a String message.
   * @param message <code>String</code> to be compressed
   * @param out OutputStream to write compressed data to.
   * It is not closed in this method.
   * @throws IOException if there is any error during writing
   */
   public static void compress(String message, OutputStream out)
   throws IOException {
   GZIPOutputStream gz = new GZIPOutputStream(out);
   byte[] bytes = message.getBytes();
   gz.write(bytes, 0, bytes.length);

   //IMPORTANT, acording to the JAZZLIB API documentation,
   //the +close():void method:
   //Writes remaining compressed output data to the output stream and closes it.
   //so you BETTER invoke it, before returning the compressed array data
   gz.close();
   }
...

In this case, we receive the message to be compressed and the stream where the compressed data is written to. The stream can be one of:

+HttpConnection.openOutputStream():OutputStream
+SocketConnection.openOutputStream():OutputStream
+OutputConnection.openOutputStream():OutputStream.

Ok, so that's it for this post. Now we have seen two libraries to decompress data on the mobile client and one of them can be used to compress data as well. In the next post, we will integrate compression of data to the Servlet-GSON vs JSONMe-JavaMe example to optimize the amount of data sent over the network.

see you soon!

References:

A pure java implementation of java.util.zip library. 2010. SourceForge [online].
Available on Internet: http://jazzlib.sourceforge.net/
[accessed on December 20 2010].

Jazzlib Java Me. November 2010. STAFF [online].
Available on Internet: http://code.google.com/p/staff/downloads/list?q=jazzlib
[accessed on December 20 2010].

Saturday, December 11, 2010

Compress in JavaMe

In scenarios where the amount of transmitted data is relevant, you should probably need to do some data compression on server side before sending information to the mobile client.

On server side, there is no problem to compress data. You can use java.util.zip.GZIPInputStream or java.util.zip.GZIPOutputStream to read compressed data from a stream or to write data and compress it to a stream respectively.

On mobile client side (JavaMe), there is not such package as java.util.zip and therefore we are going to need some third party libraries in order to handle the compression. In this post we are going to use TinyLine GZIPInputStream which is part of the TinyLine Utils, you can find the binaries in the following link:

http://www.tinyline.com/utils/index.html

The library is about 9KB and it has only the GZIPInputStream class, there is no support for the GZIPOutputStream. But if you think about it, the major data transmition occurs from the server to the mobile client so having an InputStream is more important than having an OuputStream on the mobile client side.

Once you have added the library to your mobile application, you can make use of com.tinyline.util.GZIPInputStream class in order to receive the compressed data form the server. As I mentioned on my last post, I'm just showing the relevant code so the post can be easier to read:

...

   /**
   * Decompress the data received in the Stream. The stream is not closed in this method.
   * @param in <code>InputStream</code> of the connection where the
   * compressed data is received.
   * @return <code>String</code> representing the data decompressed
   * @throws IOException if there is any error during reading
   */
   public static String deCompress(InputStream in) throws IOException {
   StringBuffer sb = new StringBuffer();

   GZIPInputStream gz = new GZIPInputStream(in);
   int c = 0;
   while ((c = gz.read()) != -1) {
   sb.append((char) c);
   }

   gz.close();

   return sb.toString();
   }
...

It is pretty simple and it works OK. There is another library, JAZZLIB, wich implements both GZIPInputStream and GZIPOutputStream, but it takes about 41KB. If you think that your application sends as much data as it receives from the server, maybe it will be worth it to compress the data on the mobile client side too. In the next post we will make an example using this library and then we will improve our JSON example to use compression.

see ya soon!

References:

TinyLine Utils. 2010. TinyLine [online].
Available on Internet: http://www.tinyline.com/utils/index.html
[accessed on December 11 2010].

Sunday, November 28, 2010

Servlet-GSON vs JSONME-JavaME

In this post I’m going to show you a simple example of data transmition between a mobile client using Java Me and a Server using Servlets. The data format to use in the transmition is JSON.

Requirements:

Servlets: On the server side, servlets will wait for requests. More info:
http://www.oracle.com/technetwork/java/index-jsp-135475.html

Java Me: Used to create mobile applications using Java. More info:
http://www.oracle.com/technetwork/java/javame/overview/index.html

JSON: Data interchange format. More info:
http://www.java-n-me.com/2010/10/json-without-fear-of-friday-13th.html

GSON: Google's library to create JSON Strings from and to objetos. More info:
http://www.java-n-me.com/2010/11/gson-json-in-just-two-lines-of-code.html

JSON Me: Library to work with JSON in Java ME. More info:
http://www.java-n-me.com/2010/11/java-me-and-json.html

Following is the Client class that is used on the server side which we will use to create the Objects to pass through the network in JSON format:

package model;

public class Client {

   /** Name of the client*/
   private String name;

   /** Last name of the client*/
   private String lastName;

   /** Identification of the client*/
   private String id;

   //Getters and Setters...
}

As shown, there is nothing strange. Just a simple class with some attributes. This is because on the server side we have GSON. On the mobile client side, this class has a little more lines of code as we shall see later.

The servlet will respond to POST requests, will create 3 Client objects, convert them to JSON Strings (using GSON) and send the response to the client:

package servlets;

import com.google.gson.Gson;
import java.io.*;
import java.util.Vector;
import javax.servlet.ServletException;
import javax.servlet.http.*;
import model.Client;

public class ClientsServlet extends HttpServlet {

   protected void doPost(HttpServletRequest request, HttpServletResponse response)
   throws ServletException, IOException {

   //sets the type of response and the charset used
   //Be careful, in your mobile client, read the response using the same charset
   response.setContentType("application/json; charset=UTF-8");

   //create some clients and add them to the vector
   Vector vClients = new Vector();

   Client clientOne = new Client();
   clientOne.setName("Alexis");
   clientOne.setLastName("Lopez");
   clientOne.setId("123456");

   vClients.add(clientOne);

   Client clientTwo = new Client();
   clientTwo.setName("Second");
   clientTwo.setLastName("Client");
   clientTwo.setId("987534");

   vClients.add(clientTwo);

   Client clientThree = new Client();
   clientThree.setName("Colombia");
   clientThree.setLastName("JUG");
   clientThree.setId("555555");

   vClients.add(clientThree);

   //convert the clients vector to JSON using GSON, very easy!
   Gson gson = new Gson();
   String jsonOutput = gson.toJson(vClients);

   System.out.println("*****JSON STRING TO RESPONSE*****");
   System.out.println(jsonOutput);
   System.out.println("*********************************");

   //print the response
   PrintWriter out = response.getWriter();
   out.println(jsonOutput);
   out.flush();
   out.close();
   }
}

When the servlet is run and it receives a POST request, the following message is shown on the console:

INFO: *****JSON STRING TO RESPONSE*****

INFO: [{"name":"Alexis","lastName":"Lopez","id":"123456"},{"name":"Second","lastName":"Client","id":"987534"},{"name":"Colombia","lastName":"JUG","id":"555555"}]
INFO: *********************************

As you can see, is the JSON representation of the newly created Vector of Client objects. Note that the servlet defines the type of response as JSON ("application/json") and also uses the "UTF-8" charaset. This is important to remember, because when the mobile client is reading the answer, you must use the same charset or you could get strange characters when reading accents or 'Ñ' letter, etc. You may also notice how easy it is to implement GSON to convert objects to JSON Strings. Only two lines of code.

That's it on the server side. Now we will see the Client class on the mobile side and will notice that it has more lines of code than the Client class on the server side. As mentioned before, this is because in Java ME there is NO Java Reflection API and therefore we can not use GSON but we use JSON ME and we have to manually map the attributes of the classes. For more information about the next class, check this previous post: http://www.java-n-me.com/2010/11/java-me-and-json.html

import java.util.Vector;
import org.json.me.JSONArray;
import org.json.me.JSONException;
import org.json.me.JSONObject;
public class Client {

   /** Name of the client*/
   private String name;

   /** Constant name of the attribute name*/
   private static final String ATT_NAME = "name";

   /** Last name of the client*/
   private String lastName;

   /** Constant name of the attribute last name*/
   private static final String ATT_LAST_NAME = "lastName";

   /** Identification of the client*/
   private String id;

   /** Constant name of the attribute id*/
   private static final String ATT_ID = "id";

   /**
   * Allows to get the JSON String from the Client passed as parameter
   * @param client Client object to convert to JSON
   * @return JSON String of the Client passed as parameter
   */
   public static String toJSON(Client client)
   {
   return toJSONObject(client).toString();
   }

   /**
   * This method should be used by this class only, that's why it is private.
   * Allows to get a JSONObject from the Client passed as parameter
   * @param client Client Object to convert to JSONObject
   * @return JSONObject representation of the Client passed as parameter
   */
   private static JSONObject toJSONObject(Client client) {
   JSONObject json = new JSONObject();
   try {
   json.put(ATT_NAME, client.getName());
   json.put(ATT_LAST_NAME, client.getLastName());
   json.put(ATT_ID, client.getId());
   } catch (JSONException ex) {
   ex.printStackTrace();
   }
   return json;
   }

   /**
   * Allows to get JSON String from a Vector of Clients
   * @param clients Vector of clients to convert to JSON
   * @return JSON String of the Vector of clients
   */
   public static String toJSONs(Vector clients) {
   JSONArray clientsArray = new JSONArray();
   for (int i = 0; i < clients.size(); i++) {
   Client client = (Client)clients.elementAt(i);

   JSONObject jsonObject = toJSONObject(client);
   clientsArray.put(jsonObject);
   }
   return clientsArray.toString();
   }

   /**
   * Allows to get a Client Object from a JSON String
   * @param jsonText JSON String to convert to a Client Object
   * @return Client Object created from the String passed as parameter
   */
   public static Client fromJSON(String jsonText) {
   Client client = new Client();
   try {
   JSONObject json = new JSONObject(jsonText);

   //check if the JSON text comes with the attribue Name
   if (json.has(ATT_NAME)) {
   //asign the String value of the attribute name to the client
   client.setName(json.getString(ATT_NAME));
   }

   //check if the JSON text comes with the attribue last name
   if (json.has(ATT_LAST_NAME)) {
   client.setLastName(json.getString(ATT_LAST_NAME));
   }

   //check if the JSON text comes with the attribue id
   if (json.has(ATT_ID)) {
   client.setId(json.getString(ATT_ID));
   }
   } catch (JSONException ex) {
   ex.printStackTrace();
   }
   return client;
   }

   /**
   * Allows to get a Vector of Clients from a JSON String
   * @param jsonArray String that contains Clients JSON Text
   * @return Vector of Clients from the String passed as parameter
   */
   public static Vector fromJSONs(String jsonArray) {
   Vector clients = new Vector();
   try {
   JSONArray jsonArr = new JSONArray(jsonArray);
   for (int i = 0; i < jsonArr.length(); i++) {
   //get a single client JSON Object
   JSONObject json = jsonArr.getJSONObject(i);

   //pass the JSON String to the method in order
   //to get the Client Object
   Client client = fromJSON(json.toString());

   clients.addElement(client);
   }
   } catch (JSONException ex) {
   ex.printStackTrace();
   }
   return clients;
   }

   //Getters and Setters...
}

Finally we'll see the MIDlet that runs and creates an HTTP connection to access the servlet. The connection is established using POST method and we read the data using the same charset that was used when creating the http response to avoid getting strange characters. Then the console displays the values of the attributes of the clients received:

import java.io.*;
import java.util.Vector;
import javax.microedition.io.*;
import javax.microedition.midlet.*;

public class JSONMIDlet extends MIDlet {

   public void pauseApp() {
   }

   public void destroyApp(boolean unconditional) {
   notifyDestroyed();
   }

   public void startApp() {
   try {

   //ask for the clients
   Vector clients = getClients();

   //show information of the clients
   System.out.println("*****CLIENT INFORMATION*****");
   for (int i = 0; i < clients.size(); i++) {
   Client cl = (Client) clients.elementAt(i);

   System.out.println("Client " + (i + 1) + " name = " + cl.getName());
   System.out.println("Client " + (i + 1) + " last name = " + cl.getLastName());
   System.out.println("Client " + (i + 1) + " ID = " + cl.getId());
   System.out.println("");
   }
   } catch (IOException ex) {
   //manage the exception
   }
   //close the application
   destroyApp(true);
   }

   /**
   * Connects to server in order to obtain information of the clients
   * @return Vector of <code>Client</code> objects
   * @throws IOException if errors with the connections
   */
   public Vector getClients() throws IOException {
   //This is my servlet’s URL
   String URL = "http://localhost:8080/GSON_Servlet/ClientsServlet";
   Vector vClients = new Vector();

   HttpConnection http = null;
   DataInputStream din = null;
   try {
   //Open HttpConnection using POST
   http = (HttpConnection) Connector.open(URL);
   http.setRequestMethod(HttpConnection.POST);
   //Content-Type is must to pass parameters in POST Request
   http.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");

   //Open InputStream to read the response
   din = new DataInputStream(http.openInputStream());
   int i = 0;
   StringBuffer str = new StringBuffer();
   while ((i = din.read()) != -1) {
   str.append((char) i);
   }

   //IMPORTANT: Read the response in the same charset that it was written
   String jSonString = new String(str.toString().getBytes(), "UTF-8");

   System.out.println("*****JSON STRING RECEIVED*****");
   System.out.println(jSonString);
   System.out.println("******************************");

   //Parse the JSON String to obtain a Vector of clients
   vClients = Client.fromJSONs(jSonString);

   } //whatever happens, close the streams
   finally {
   if (din != null) {
   din.close();
   }

   if (http != null) {
   http.close();
   }
   }

   return vClients;
   }
}

When we run the MIDlet the console prints the following lines:

*****JSON STRING RECEIVED*****
[{"name":"Alexis","lastName":"Lopez","id":"123456"},{"name":"Second","lastName":"Client","id":"987534"},{"name":"Colombia","lastName":"JUG","id":"555555"}]
****************************** *****CLIENT INFORMATION***** Client 1 name = Alexis Client 1 last name = Lopez Client 1 ID = 123456 Client 2 name = Second Client 2 last name = Client Client 2 ID = 987534 Client 3 name = Colombia Client 3 last name = JUG Client 3 ID = 555555

Obviously, in a business application the servlet would connect to a database in order to get the clients information, then the servlet would send that information to the mobile client using JSON. The mobile client would have a graphical interface that would allow the user to view the information received from the server... and so many other improvements. But this simple example, shows you the possibilities offered by JSON for communication between server and mobile client in a standard way without having to invent new formats for data exchange.

This concludes the communication Servlets/GSON vs JSON Me/Java Me. I know there have been a lot of code, it may be difficult to read. For the following posts I will emphasize on the really important parts of the code and let the entire source code to be downloaded.

see you soon!

References:

Friday, November 12, 2010

Java ME and JSON

In previous posts, we saw how JSON has been gaining ground as data transmission format. We also introduced GSON, Google’s API to create JSON text from an Object and Objects from JSON text in a fairly simple way, but that can’t be used in Java ME because of the Java Reflection API.

In this post we are going to stand on the mobile client side using JSON ME as application library to create JSON text. The library has disappeared from the JSON’s official web site (http://www.json.org), I'm not sure why, but the project is in the repository of java.net. The steps needed to obtain the source code from java.net SVN to NetBeans can be found in the following link:

http://www.java-n-me.com/2010/11/how-to-import-source-code-from-svn-to.html

If you don’t want to build the source code, contact me so I can send you the library. Once you have the JSON ME .jar file, you are able to implement JSON in your mobile application. Although you have to write more lines of code than when you are using GSON, it is still easy to use and the benefit in terms of ease of data transmission between mobile client and server is worth it!

As a example, we will develop a simple MIDlet that does not have UI, but only the necessary classes from Java ME to be able to generate JSON text from Objects and Objects from JSON text.

Next is the source code of the MIDlet:

import java.util.Vector;
import javax.microedition.midlet.*;
public class JSONMIDlet extends MIDlet {
   public void startApp() {
   Vector clients = new Vector();
   Client client = new Client();
   client.setName("Mr Developer");
   client.setLastName("level 1");
   client.setId("123456");

   //shows the client JSON String
   String jsonOne = Client.toJSON(client);
   System.out.println("***** FIRST OBJECT AS JSON ******");
   System.out.println(jsonOne);
   //add the client to the vector
   clients.addElement(client);

   client = new Client();
   client.setName("Mrs Developer");
   client.setLastName("level 2");
   client.setId("987654");

   //shows the client JSON String
   String jsonTwo = Client.toJSON(client);
   System.out.println("***** SECOND OBJECT AS JSON ******");
   System.out.println(jsonTwo);
   //add the client to the vector
   clients.addElement(client);

   //Now get the Clients Vector JSON String
   String jsons = Client.toJSONs(clients);
   System.out.println("***** ALL OBJECTS AS JSON ******");
   System.out.println(jsons);

   System.out.println("");

   //now map objectos from JSON Strings
   Client newClient = Client.fromJSON(jsonOne);
   System.out.print("***** NAME OF THE NEW FIRST CLIENT = ");
   System.out.println(newClient.getName());

   newClient = Client.fromJSON(jsonTwo);
   System.out.print("***** NAME OF THE NEW SECOND CLIENT = ");
   System.out.println(newClient.getName());

   Vector newClients = Client.fromJSONs(jsons);
   System.out.print("***** SIZE OF THE NEW CLIENT VECTOR = ");
   System.out.println(newClients.size());

   destroyApp(true);
   }

   public void pauseApp() {
   }

   public void destroyApp(boolean unconditional) {
   notifyDestroyed();
   }
}

As you can see, the MIDlet is creating Client Objects with defaults values. Surely in a real application, these data are coming from text fields in a UI. The MIDlet output to the console JSON Strings from these Objects and some attributes of Objects created from JSON strings.

Most of the work is done in the classes you wish to convert to JSON text. The following classes are the key to work with JSON ME:

org.json.me.JSONObject: Contains key / value pairs with which the JSON text is built. It also provides the method for obtaining the JSON text.
org.json.me.JSONArray: Ordered sequence of values that are separated by commas. These values can be JSONObjects. Practically, it can be understood as an array of JsonObject Objects.

Below is the Client class which implements the methods that convert to and from JSON and makes use of the previously named classes:

The Client class contains methods that allow us to get a Client Object from JSON text and get the JSON text from a Client Object. As good practice, you can create constants with the attribute names so they can be used in the methods avoiding hard coding. So you can see the big difference with GSON, given that Java ME can’t use the Reflection API, it is necessary to assign values to attributes manually.

Here is the output text that is displayed by the console, once the MIDlet has been run:

***** FIRST OBJECT AS JSON ****** {"lastName":"level 1","name":"Mr Developer","id":"123456"}
***** SECOND OBJECT AS JSON ****** {"lastName":"level 2","name":"Mrs Developer","id":"987654"}
***** ALL OBJECTS AS JSON ****** [{"lastName":"level 1","name":"Mr Developer","id":"123456"},{"lastName":"level 2","name":"Mrs Developer","id":"987654"}] ***** NAME OF THE NEW FIRST CLIENT = Mr Developer ***** NAME OF THE NEW SECOND CLIENT = Mrs Developer ***** SIZE OF THE NEW CLIENT VECTOR = 2

If many classes of your application need to be converted to JSON text, you may create an Interface that defines methods for all classes wishing to implement JSON. A good example of this practice can be found on the following link:

http://jimmod.com/blog/2010/03/java-me-j2me-json-implementation-tutorialsample/

OK that's it for this post, I hope I was clear enough. Any further explanation do not hesitate to contact me.

In the next post we will integrate the pairs Servlet / GSON vs JSON ME / Java ME, so we can communicate mobile clients with servers and vice versa.

See you soon!

References:

Java ME (J2ME) JSON Implementation Tutorial/Sample. March 2010. Jimmy’s Blog [online].
Available on Internet: http://jimmod.com/blog/2010/03/java-me-j2me-json-implementation-tutorialsample/
[accessed on November 09 2010].

Sunday, November 7, 2010

GSON: JSON in just two lines of code, second part

As I mentioned in my last post, in this post we'll make a little more complicated example than the previous one. For those who have not yet revised the previous post, you can do it now using this link:

http://www.java-n-me.com/2010/11/gson-json-in-just-two-lines-of-code.html

Well, now that we have reviewed the above example, we will modify the classes. Find the changes in bold:

import java.util.*;

public class Book

{

/** Name of the book*/

private String name;

/** number of pages*/

private Integer numPages;

/** How much the book costs*/

private Double cost;

/**authors's names*/

private String[] authors;

/** Publication Date*/

private Date published;

/** Pages of the book*/

private List<page> pages;

//Constructor, Getters and Setters...

}

public class Page

{

/** Number of the page*/

private int numPage;

/** Text of the page*/

private String text;

//Constructor, Getters and Setters...

}

import com.google.gson.Gson;
import java.util.*;
public class Control
{
public static void main(String[] args)
{
//create a book
   Book myBook = new Book();
   myBook.setName("Java and JSON");
   myBook.setNumPages(2);
   myBook.setCost(1.5);
   myBook.setAuthors(new String[]{"Alexis López", "Duke"});
   myBook.setPublished(new Date());

     //Create the pages
   Vector<Page> pages = new Vector();
   Page pOne = new Page();
   pOne.setNumPage(1);
   pOne.setText("This text is from page one");

   Page pTwo = new Page();
   pTwo.setNumPage(2);
   pTwo.setText("This text is from page two");

   pages.add(pOne);
   pages.add(pTwo);

   //Add the pages to the book
   myBook.setPages(pages);

//Convert myBook to JSON using GSON
   Gson gson = new Gson();
   String json = gson.toJson(myBook);

//print the result
System.out.println("*** MyBook as JSON ***");
   System.out.println(json);

   //Map JSON text to a new book
   Book newBook = gson.fromJson(json, Book.class);

   //print some attributes of the new book
System.out.println("*** NewBook from JSON ***");
   System.out.println("Name = " + newBook.getName());
   System.out.println("Authors = " + Arrays.toString(newBook.getAuthors()));
   System.out.println("Number of pages = " + newBook.getNumPages());
   System.out.println("Publication date = " + newBook.getPublished());
   System.out.println("Page One = " +
   ((Page)(newBook.getPages().get(0))).getText());
   System.out.println("Page two = " +
   ((Page)(newBook.getPages().get(1))).getText());
   //...
}
}

Note that we used java.util.List and not a java.util.Vector for the book's pages Collection. This is because GSON, when we are mapping the new Object, gson will try to use a java.util.LinkedList, and therefore it will not be compatible with java.util.Vector nor with java.util.ArrayList, producing the following exception:

Exception in thread "main" java.lang.IllegalArgumentException: Can not set java.util.Vector field Book.pages to java.util.LinkedList
   at sun.reflect.UnsafeFieldAccessorImpl.throwSetIllegalArgumentException(UnsafeFieldAccessorImpl.java:146)
   at sun.reflect.UnsafeFieldAccessorImpl.throwSetIllegalArgumentException(UnsafeFieldAccessorImpl.java:150)
   at sun.reflect.UnsafeObjectFieldAccessorImpl.set(UnsafeObjectFieldAccessorImpl.java:63)
   at java.lang.reflect.Field.set(Field.java:657)
   at com.google.gson.FieldAttributes.set(FieldAttributes.java:182)
   at com.google.gson.JsonObjectDeserializationVisitor.visitFieldUsingCustomHandler
(JsonObjectDeserializationVisitor.java:118)
   at com.google.gson.ObjectNavigator.navigateClassFields(ObjectNavigator.java:158)

//...

For this reason we use java.util.List. The following console output is shown when you run the Control class:

*** MyBook as JSON ***

{"name":"Java and JSON","numPages":2,"cost":1.5,"authors":["Alexis López","Duke"],"published":"7/11/2010 10:10:19 AM","pages":[{"numPage":1,"text":"This text is from page one"},{"numPage":2,"text":"This text is from page two"}]}

*** NewBook from JSON ***

Name = Java and JSON

Authors = [Alexis López, Duke]

Number of pages = 2

Publication date = Sun Nov 07 10:10:19 COT 2010

Page One = This text is from page one

Page two = This text is from page two

Thus, we can have Objects with relationships to another Objects and convert them to and from JSON.

In a future post, we will review examples with Java ME/JSON ME.

See you soon!

References:

google-gson. August 19 de 2010. Google Code [online]. Available from Internet:http://code.google.com/p/google-gson/ [accessed on October the 26th 2010].

Wednesday, November 3, 2010

GSON: JSON in just two lines of code

The increased use of JSON for data exchange has resulted in the emergence of several libraries to handle it. GSON, google's API, allows the production of JSON strings from Objects and Objects from JSON strings quite easily. However, additional steps must be performed when the Object implements Collections of items that are not primitive.

In this post we will have a simple example with no Collections. We'll convert Object to JSON and then JSON to Object.

GSON can be downloaded from:

http://code.google.com/p/google-gson/

It's a .jar file wich you can add to your project. GSON uses the Java Refelction API in order to generate the JSON Strings in just two lines of code! two lines!

Note: Due to the use of the Java Reflection API, GSON can't be used in Java ME, but it is a very good choice if you want to produce JSON Strings of an Object at server side and then send it to a mobile client.

The Java Classes we will use during the example are:

import java.util.Date;

public class Book

{

/** Name of the book*/

private String name;

/** number of pages*/

private Integer numPages;

/** How much the book costs*/

private Double cost;

/**authors's names*/

private String[] authors;

/** Publication Date*/

private Date published;

//Constructor, Getters and Setters...

}

import com.google.gson.Gson;
import java.util.Date;
public class Control
{
public static void main(String[] args)
{
//create a book
   Book myBook = new Book();
   myBook.setName("Java and JSON");
   myBook.setNumPages(2);
   myBook.setCost(1.5);
   myBook.setAuthors(new String[]{"Alexis López", "Duke"});
   myBook.setPublished(new Date());

//Convert myBook to JSON using GSON
   Gson gson = new Gson();
   String json = gson.toJson(myBook);

//print the result
System.out.println("*** MyBook as JSON ***");
   System.out.println(json);

   //Map JSON text to a new book
   Book newBook = gson.fromJson(json, Book.class);

   //print some attributes of the new book
System.out.println("*** NewBook from JSON ***");
   System.out.println("Name = " + newBook.getName());
   System.out.println("Authors = " + Arrays.toString(newBook.getAuthors()));
   System.out.println("Number of pages = " + newBook.getNumPages());
   System.out.println("Publication date = " + newBook.getPublished());

   //...
}
}

Once the Control class has been run, you will notice in the console the JSON String of the Object myBook:

{"name":"Java and JSON","numPages":2,"cost":1.5,"authors":["Alexis López","Duke"],"published":"3/11/2010 09:42:06 AM"}

Also, you will notice the information of the new book (newBook) mapped from the MyBook JSON String:

Name = Java and JSON

Authors = [Alexis López, Duke]

Number of pages = 2

Publication date = Wed Nov 03 09:42:06 GMT-05:00 2010

As I mentioned at the beginning of the post, when the object in question contains among its attributes a Collection, you should consider this:

- Producing the JSON string is the same as we saw in the example above.
- Mapping Objects from the JSON String requires additional steps, especially because it is not possible to recognize the type of Generics used for collection from the plain text (JSON).

In a future post, we will extend the example to clarify this.

See you soon!

References:

google-gson. August 19 de 2010. Google Code [online]. Available from Internet:http://code.google.com/p/google-gson/ [accessed on October the 26th 2010].