Wednesday, October 31, 2012

Java 7: File Filtering using NIO.2 - Part 2

Hello all. This is Part 2 of the File Filtering using NIO.2 series. For those of you who haven't read Part 1, here's a recap.

NIO.2 is a new API for I/O operations included in the JDK since Java 7. With this new API, you can perform the same operations performed with java.io plus a lot of great functionalities such as: Accessing file metadata and watching for directory changes, among others. Obviously, the java.io package is not going to disappear because of backward compatibility, but we are encouraged to start using NIO.2 for our new I/O requirements. In this post, we are going to see how easy it is to filter the contents of a directory using this API. There are 3 ways in order to do so, we already review one way in Part 1 and now we are going to see  another approach.

What you need
NetBeans 7+ or any other IDE that supports Java 7

Filtering content of a directory is a common task in some applications and NIO.2 makes it really easy. The classes and Interfaces we are going to use are described next:
  • java.nio.file.Path: Interface whose objects may represent files or directories in a file system. It's like the java.io.File but in NIO.2. Whatever I/O operation you want to perform, you need an instance of this interface.
  • java.nio.file.PathMatcher: Interface that allows objects to perform match operations on paths.
  • java.nio.file.DirectoryStream: Interface whose objects iterate over the content of a directory.
  • java.nio.file.Files: Class with static methods that operates on files, directories, etc.

The way we are going to filter the contents of a directory is by using objects that implement the java.nio.file.PathMatcher interface. We can get one of these objects with the help of the java.nio.file.Files class, using the method +getPathMatcher(String):PathMatcher. This method supports both "glob" and "regex" patterns. You can check Part 1 of File Filtering using NIO.2 for more information about "glob" and for "regex" visit the java.util.regex.Pattern class. The pattern is matched against the name of the files, directories, etc. That live inside the directory. This is important to remember, using this method you can only filter by the name of the file, directory, etc.

For example, if you want to filter .png and .jpg images, you should use one of the following syntax and pattern (notice the colon between the syntax and the pattern):
  • "glob:*.{png,jpg}"
  • "regex:([^\s]+(\.(?i)(png|jpg))$)"

Of course, "glob" syntax is much simpler, but you have the option of using regular expressions for a more detailed match. Anyway, you may be wondering why you should use this approach if the java.nio.files.DirectoryStream interface allows you to filter directly using "glob"... Well, let's suppose that you already have a filter, but you need to perform more than one filtering operation, that's when you need to use this approach.

The following piece of code defines a method which scans a directory using different patterns:

//in a class...
    
    /**
     * Scans the directory using the patterns passed 
     * as parameters. 
     * Only 3 patterns will be used.
     * @param folder directory to scan
     * @param patterns The first pattern will be used
     * as the glob pattern for the DirectoryStream.     
     */
    private static void scan(String folder, String... patterns) {
        //obtains the Images directory in the app directory
        Path dir = Paths.get(folder);
        //the Files class offers methods for validation
        if (!Files.exists(dir) || !Files.isDirectory(dir)) {
            System.out.println("No such directory!");
            return;
        }
        //validate at least the glob pattern
        if (patterns == null || patterns.length < 1) {
            System.out.println(
                "Please provide at least the glob pattern.");
            return;
        }

        //obtain the objects that implements PathMatcher
        PathMatcher extraFilterOne = null;
        PathMatcher extraFilterTwo = null;
        if (patterns.length > 1 && patterns[1] != null) {
            extraFilterOne = FileSystems.getDefault().
                                 getPathMatcher(patterns[1]);
        }
        if (patterns.length > 2 && patterns[2] != null) {
            extraFilterTwo = FileSystems.getDefault().
                                 getPathMatcher(patterns[2]);
        }

        //Try with resources... so nice!
        try (DirectoryStream ds = 
                  Files.newDirectoryStream(dir, patterns[0])) {
            //iterate over the content of the directory and apply 
            //any other extra pattern
            int count = 0;
            for (Path path : ds) {
                System.out.println(
                          "Evaluating " + path.getFileName());

                if (extraFilterOne != null && 
                    extraFilterOne.matches(path.getFileName())) {
                    System.out.println(
                                  "Match found Do something!");
                }

                if (extraFilterTwo != null && 
                    extraFilterTwo.matches(path.getFileName())) {
                    System.out.println(
                             "Match found Do something else!");
                }

                count++;
            }
            System.out.println();
            System.out.printf(
                 "%d Files match the global pattern\n", count);
        } catch (IOException ex) {
            ex.printStackTrace();
        }
    }

You can try invoking the last method with the following parameters:

  • C:\Images or /Images depending on your OS.
  • ?_*.jpg This pattern specifies that you want all .jpg images whose name starts with one digit followed by an underscore.
  • glob:2_* Specifies another filter (using glob syntax) where you only want items whose name starts with the number two followed by an underscore.
  • glob:3_* Specifies another filter (using glob syntax) where you only want items whose name starts with the number three followed by an underscore.

Having several filters allows you to take different actions for matched items.
Following is the result of the execution on my windows machine:



And on my Linux virtual machine:



Again, Write once, run everywhere! However, notice that the ordering of the items is system dependent, so do not ever hardcode the position of a file or directory.

I hope you enjoyed this post, there is another more powerful way to filter the content of a directory and we'll explore it in Part 3.

Click here to download the source code.


See ya!

References:

Reese Richard and Reese Jennifer (2012). Java 7 New Features Cookbook. United Kingdom: Packt Publishing Ltd.

Saturday, October 27, 2012

Book Review: Java EE 6 Cookbook for Securing, Tuning, and Extending Enterprise Applications

Book cover from
http://www.packtpub.com
Java EE 6 is the latest specification for building Enterprise Applications with Java. It is a set of APIs that will simplify your development and if you come from Java EE 5, then you will notice the difference. Fortunately, Java EE 6 is been around for almost three years now and is widely adopted, which improves the quality of the documentation and training you may find about it.

In this post, I'll be reviewing the book "Java EE 6 Cookbook for Securing, Tuning, and Extending Enterprise Applications" written by Mick Knutson and published by PACKT. I have to say that this book is not for begginers in the Java EE field, you need some Java EE background in order to better understand the recipes. It is aimed at Java developers and programmers who want to secure, tune and, extend their Java EE applications

I liked the way chapter one updates you not only with the new features added to the specification, but also with the outgoing ones. It explains why some JSRs are pruned from this release and then it starts with the new: Context Dependency Injection (CDI), EJB 3.1, JPA 2.0, JAX-RS 1.1, Servlets 3.0, JSF 2.0, etc. Again, this is just a small review as cited on the book "... This chapter is not a tutorial or primer on the various specifications..." But is good enough to see what's new in Java EE 6. This chapter contains no recipes.

Chapter two dives into the implementation of some of the new features of the JPA 2.0 spec. The recipes are written in the form "Getting ready - How to do it - How it works - There's more..." which allows you to start using the new features very fast and avoid pitfalls. There's a tutorial in this chapter that helps you in profiling and testing JPA Operations, really useful.

Chapter three is about security using the Java EE built-in features and it also explains how to use Spring Security for a more fine-grained security implementation. This chapter is all about security, I wasn't expecting recipes about configuring Linux firewall rules or obfuscating Java byte-code... well done!

So far, so good. I haven't finished the book, but looking at the table of contents I can see really interesting chapters ahead:
  • Chapter 4: Enterprise Testing Strategies. Inlcuding Testing JPA with DBUnit,  Testing JAX-WS and JAX-RS with soupUI, among others.
  • Chapter 5: Extending Enterprise Applications. Inlcudes integrating Scala, Groovy and Jython into Enterprise Applications. I think this is an important chapter, specially because JSR 292 Invoke Dynamics is now part of the JDK 7.
  • Chapter 6: Enterprise Mobile Device Integration. Inlcudes an evaluation of Mobile-Web frameworks: jQuery Mobile, Sencha Touch and Modernizer. Really interesting and updated. There is also information about development and distribution considerations for iOS and Android.
  • Chapter 7: Deployment and Configuration. Inlcudes recipes that cover issues and solutions to application configuration with the use of Java Management Extensions (JMX).This chapter also cover tools to aid in  rapid and hot-deployment of Java EE applications such as Apache Ant and Apache Maven.
  • Chapter 8: Performance and Debugging. Inlcudes recipes that will help you in improving  the performance of your Enterprise applications with: jVisualVM, jstatd, among others.
There are so many recipes about topics I wasn't expecting that it really surprised me. After reviewing this book I realize that many of the topics will show you tools that you can leverage when tuning and extending your Java EE applications, so you can make the difference in your team. This is a must read if you are a architect/leadDevelper/consultant.

One more thing, the book is published in a lot of formats: printed book, kindle, PDF, ePub, so you have many options to read it. I have the kindle edition and it looks great on my kindle, paragraphs are well formatted and the source code is easy to read.

For more information about this book go to:
http://www.packtpub.com/java-ee6-securing-tuning-extending-enterprise-applications-cookbook/book

See ya!

Friday, October 19, 2012

Java 7: File Filtering using NIO.2 - Part 1

Hello all. NIO.2 is a new API for I/O operations included in the JDK since Java 7. With this new API, you can perform the same operations performed with java.io plus a lot of great functionalities such as: Accessing file metadata and watching for directory changes, among others. Obviously, the java.io package is not going to disappear because of backward compatibility, but we are encouraged to start using NIO.2 for our new I/O requirements. In this post, we are going to see how easy it is to filter the contents of a directory using this API. There are 3 ways in order to do so, that's why this post is Part 1.

What you need
NetBeans 7+ or any other IDE that supports Java 7
JDK 7+

Filtering content of a directory is a common task in some applications and NIO.2 makes it really easy. The classes and Interfaces we are going to use are described next:
  • java.nio.file.Path: Interface whose objects may represent files or directories in a file system. It's like the java.io.File but in NIO.2. Whatever I/O operation you want to perform, you need an instance of this interface.
  • java.nio.file.DirectoryStream: Interface whose objects iterate over the content of a directory.
  • java.nio.file.Files: Class with static methods that operates on files, directories, etc.

The way we are going to filter the contents of a directory is by using glob patterns, which are like regular expressions but simpler. The pattern is matched against the name of the files, directories, etc. That live inside the directory. This is important to remember, using this method you can only filter by the name of the file, directory, etc.
For more information about globbing, check this wiki. Also, there is some documentation about globbing patterns in the Java Docs.

So, let's suppose that we have a directory called Images, and we need to iterate over the files inside this directory, but we only need the .png files. We have to follow this steps in order to do so:
  1. Obtain a java.nio.file.Path instance which points to the directory Images.
  2. Open a new java.nio.file.DirectoryStream using the java.nio.file.Files class and passing the directory and the pattern (*.png) as parameters.
  3. Iterate over the content of the directory using the java.nio.file.DirectoryStream instance.
Next is the source code of a method that scans a directory using the pattern passed as parameter:

//in a class...
    
    /**
     * Scans the directory using the glob pattern passed 
     * as parameter. 
     * @param folder directory to scan
     * @param pattern glob pattern (filter)
     */
    private static void scan(String folder, String pattern) {
        //obtains the Images directory in the app directory
        Path dir = Paths.get(folder);
        //the Files class offers methods for validation
        if (!Files.exists(dir) || !Files.isDirectory(dir)) {
            System.out.println("No such directory!");
        }
        //Try with resources... so nice!
        try (DirectoryStream<path> ds = 
                    Files.newDirectoryStream(dir, pattern)) {
            //iterate over the content of the directory
            int count = 0;
            for (Path path : ds) {
                System.out.println(path.getFileName());
                count++;
            }
            System.out.println();
            System.out.printf("%d Files match the pattern"
                                                     , count);
        } catch (IOException ex) {
            ex.printStackTrace();
        }
    }


Following is the result of the execution on my windows machine:



And on my Linux virtual machine:


That's something I love from Java, Write once, run everywhere! I hope you enjoyed this post, there are more ways to filter the content of a directory and we'll explore them in future posts.

Click here to download the complete source code.

See ya!

References:

Reese Richard and Reese Jennifer (2012). Java 7 New Features Cookbook. United Kingdom: Packt Publishing Ltd.

Sunday, October 14, 2012

Java 7: Meet the Fork/Join Framework

Free clip art from
http://www.freepik.com
JSR-166(y) is the official name of this new feature which is included in Java 7. If you notice there is a "y" in the name, this is because JSR-166 (Concurrency Utilities) is being added since Java 5, but it wont stop here as there are already plans to add new classes in Java 8 under the JSR-166(e). Check this page maintained by Doug Lea, the creator of JSR-166, for more information.

According to Wikipedia, Parallelism is the "simultaneous execution of some combination of multiple instances of programmed instructions and data on multiple processors" and Java has classes and interfaces to achieve this (sort of...) since DAY 1. You may know them as: java.lang.Thread, java.lang.Runnable, etc... What Concurrency Utilities (java.util.concurrent package) does is simplify the way we code concurrent tasks, so our code is much simpler and cleaner. As developers we haven't had to do anything when running our applications in machines with higher processing resources, obviously, the performance of our applications will improve, but are we really using the processing resources to the maximum? The answer is big NO.

This post will show you how the Fork/Join framework will help us in using the processing resources to the maximum when dealing with problems that can be divided into small problems and all the solutions to each one of those small problems produce the solution of the big problem (like recursion, divide and conquer).

What you need
NetBeans 7+ or any other IDE that supports Java 7
JDK 7+
Blur on an image, example from Oracle

The Basics
The Fork/Join framework focuses on using all the processing resources available in the machine to improve the performance of the applications. It was designed to simplify parallelism in Divide and Conquer algorithms. The magic behind the Fork/Join framework is its work-stealing algorithm in which work threads that are free steal tasks from other busy threads, so all threads are working at all times. Following are the basics you should know in order to start using the framework:
  • Fork means splitting the task into subtasks and work on them.
  • Join means merging the solution of every subtask into one general solution.
  • java.lang.Runtime use this class in order to obtain the number of processors available to the Java virtual machine. Use the method +availableProcessors():int in order to do so.
  • java.util.concurrent.ForkJoinPool Main class of the framework, is the one that implements the work-stealing algorithm and is responsible for running the tasks.
  • java.util.concurrent.ForkJoinTask Abstract class for the tasks that run in a java.util.concurrent.ForkJoinPool. Understand a task as a portion of the whole work, for example, if you need to to do something on an array, one task can work on positions 0 to n/2 and another task can work on positions (n/2) +1 to n-1, where n is the length of the array.
    • java.util.concurrent.RecursiveAction Subclass of the abstract task class, use it when you don't need the task to return a result, for example, when the task works on positions of an array, it doesn't return anything because it worked on the array. The method you should implement in order to do the job is compute():void, notice the void return.
    • java.util.concurrent.RecursiveTask Subclass of the abstract task class, use it when your tasks return a result. For example, when computing Fibonacci numbers, each task must return the number it computed in order to join them and obtain the general solution. The method you should implement in order to do the job is compute():V, where V is the type of return; for the Fibonacci example, V may be java.lang.Integer.

When using the framework, you should define a flag that indicates whether it is necessary to fork/join the tasks or whether you should compute the work directly. For example, when working on an array, you may specify that if the length of the array is bigger than 500_000_000 you should fork/join the tasks, otherwise, the array is small enough to compute directly. In essence, the algorithm you should follow is shown next:

if(the job is small enough)
{
   compute directly
}
else
{
   split the work in two pieces (fork)
   invoke the pieces and join the results (join)
}

OK, too much theory for now, let's review an example.

The Example
Blurring an image requires to work on every pixel of the image. If the image is big enough we are going to have a big array of pixels to work on and so we can use fork/join to work on them and use the processing resources to the maximum. You can download the source code from the Java™ Tutorials site.

Once you download the source code, open NetBeans IDE 7.x and create a new project:



Then select Java Project with Existing Sources from the Java category in the displayed pop-up window:



Select a name and a project folder and click Next >



Now, select the folder where you downloaded the source code for the Blur on an image example:



And select the file ForkBlur.java then click finish:



The source code will be imported and a new project will be created. Notice that the new project is shown with erros, this is because Java 7 is not enable for default:



To fix this, right click on the project name and select the option Properties. On the pop-up dialog, go to Libraries and select JDK 1.7 from the Java Platform ComboBox:



Now, go to the option Sources and select JDK 7 from the Source/Binary Format ComboBox:



Last but not least, increase the memory assigned to the virtual machine when running this application as we'll be accessing a 5 million positions array (or more). Go to the option Run and insert -Xms1024m -Xmx1024m on the VM Options TextBox:


Click OK and your project should be compiling with no errors. Now, we need to find an image bigger enough so we can have a large array to work on. After a while, I found some great images (around 150 MB) from planet Mars, thanks to the curiosity robot, you can download yours from here. Once you download the image, past it on the project's folder.

Before we run the example, we need to modify the source code in order to control when to run it using the Fork/Join framework. In the ForkBlur.java file, go to line 104 in order to change the name of the image that we are going to use:

//Change for the name of the image you pasted 
//on the project's folder.
String filename = "red-tulips.jpg";

Then, replace lines 130 to 136 with the following piece of code:

ForkBlur fb = new ForkBlur(src, 0, src.length, dst);
        boolean computeDirectly = true;

        long startTime = System.currentTimeMillis();
        if (computeDirectly) {
            fb.computeDirectly();
        } else {
            ForkJoinPool pool = new ForkJoinPool();
            pool.invoke(fb);
        }
        long endTime = System.currentTimeMillis();

Notice the computeDirectly flag. When true, we'll NOT be using the fork/Join Framework, instead we will compute the task directly. When false, the fork/join framework will be used.

The compute():void method in the ForkBlur class implements the fork/join algorithm. It's based on the length of the array, when the length of the array is bigger than 10_000, the task will be forked, otherwise, the task will be computed directly.

Following you can see my 2 processors when executing the Blur on an image example without using the Fork/Join framework (computeDirectly = true), it took about 14s to finish the work:



You can see that the processors are working, but not to the maximum. When using the Fork/Join framework (computeDirectly = false) you can see them working at 100% and it took almost 50% less time to finish the work:



This video shows the complete process:




I hope you can see how useful this framework is. Of course, you cannot use it all around your code, but whenever you have a task that can be divided into small tasks then you know who to call.

For more tutorials and information about concurrency in Java, check the Java™ Tutorials site.

see ya!


References:

The Java™ Tutorials. Oracle [online].
Available on Internet: http://docs.oracle.com/javase/tutorial/essential/concurrency/
[accessed on October 10 2012].

PHOTOJOURNAL. NASA Jet Propulsion Laboratory [online].
Available on Internet: http://photojournal.jpl.nasa.gov/catalog/PIA16101
[accessed on October 10 2012].

Monday, October 8, 2012

ADF Essentials

Hello all. Today I'm going to present you a great framework for bulding Rich Web applications with Java. It is called ADF Essentials and for those of you who haven't heard about it, let me tell you its benefits:
  • 150 Ajax-enabled JSF components.
  • Support for Skinning.
  • Visual and declarative development.
  • Free to develop and deploy.
  • Glassfish server officially certified, check the ADF Essentials product certification page.
  • Includes ADF Faces, ADF Data Visualization Tools (DVT), ADF Controller, ADF Model, ADF Share, and ADF Business Components. Check this page for a complete list of components included.
  • For more information and other benefits visit the ADF Essentials product home page.

The framework is built over JSF 2 so if you have previous experience in building Web applications using JSF you are likely to start developing with ADF Essentials pretty fast. On the contrary, if you are just beggining with Web development, the visual and declarative development this framework offers will let you build the application in a fast pace. Either case, there are tons of documentation online: books, tutorials and  videos to help you.

What you will need:

I've been using ADF Faces (which is the complete version of the framework) for the last 3 years and I can tell how easy it is to develop RIA (Rich Internet Applications) with it. In future posts I will show you how to deploy a Web application developed with ADF Essentials on Glassfish server and even on Tomcat (which is not yet officially supported).

Two things before we finish:
  • ADF Faces is the complete framework (which is free to develop but not free to deploy) and ADF Essentials is a subset of ADF Faces and is free to develop and deploy.
  • Check the following video from ADF Insider Essentials's channel on youtube so you can see the framework in action:




see ya!


References:

Oracle ADF Essentials. Oracle [online].
Available on Internet: http://www.oracle.com/technetwork/developer-tools/adf/overview/adfessentials-1719844.html
[accessed on October 06 2012].

Friday, October 5, 2012

My "Make the Future Java" Tool Kit arrived!

Hello all. I want to share with you that my "Make the Future Java" Tool Kit just arrived! It took about 15 days to travel from Redwood, CA to Colombia, but it's finally here and I'm going to show you what's inside.

Make the Future Java motivates each of us to share our knowledge and to inspire future technologists to learn more about Java. You can order your Tool Kit too, is FREE. For more information, please visit the program website.

So what's inside? According to Oracle: "The Make the Future Java Tool Kit contains both learning resources and Java branded materials to give your outreach project more impact". It comes with a nice T-Shirt, postcards, posters and DVDs with the following:


All the tools, tutorials and HOL (Hands On Lab) are available online as well. I'll be posting about our JUG meetings and events where we Make the Future Java. Remember, You can help too, order your Tool Kit and inspire others!

see ya!!