Nisheeth Joshi [Nisheeth Joshi] - Hands-On Artificial Intelligence with Java for Beginners

We'll be developing a very simple, decision tree-based classifier, using the weka.classifiers package. For decision tree classification, we'll use the J48 algorithm, which is a very popular algorithm. To develop a classifier, we'll set two flags, as follows:

• -C: Sets the confidence threshold for pruning. Its default value is 0.25.
• -M: Sets the maximum number of instances for developing a decision tree classifier. Its default value is 2.

All of the other classifiers can be developed based on similar methods, which we'll incorporate while developing our decision tree classifier. We'll develop one more classifiera Naive Bayes classifierbased on the same mechanism that we will follow to develop our decision tree classifier.

Let's get to the code and see how to do it. We'll start by importing the following classes:

import weka.core.Instances;
import weka.core.converters.ConverterUtils.DataSource;
import weka.classifiers.trees.J48;

Now, let's move on to the following code:

public static void main(String[] args) {
// TODO code application logic here
try{
DataSource src = new DataSource("/Users/admin/Documents/NetBeansProjects/DevelopClassifier/vote.arff");
Instances dt = src.getDataSet();
dt.setClassIndex(dt.numAttributes()-1);
String[] options = new String[4];
options[0] = "-C";
options[1] = "0.1";
options[2] = "-M";
options[3] = "2";
J48 tree = new J48();
tree.setOptions(options);
tree.buildClassifier(dt);
System.out.println(tree.getCapabilities().toString());
System.out.println(tree.graph());
//NaiveBayes nb = new NaiveBayes();
}
catch(Exception e){
System.out.println("Error!!!!\n" + e.getMessage());
}

This time, we're using a vote.arff dataset, because it has a very large amount of data. It is the 1984 United States Congressional Voting Records database, which has many tuples. It includes attributes such as whether a member is handicapped. Based on these attributes, it makes a prediction on whether a person is a Democrat or a Republican.

First, we'll create an object for our dataset by using a DataSource class. Then, we'll create an Instances object, and we'll put the dataset into the Instances object. Once we have opened our dataset, we will have to tell Weka which attribute is a class attribute (which one will be used for classification). As you can see in the list of attributes in the preceding code, the class attribute is at the end. Therefore, we'll take setClassIndex; and, since the -1 attribute is the class attribute, (dt.numAttributed()-1) will get the index of that particular attribute.

We'll then create an array of Strings; and, since we need to set -C and -M, we'll initialize our String array with four elements. The first element will be -C, the second will be the threshold value, the third will be -M, and the fourth will be the number of iterations it should take. Then, we will create an object for J48. Once we have created an object for J48, we'll assign the options to J48 by using setOptions. Then, we will have to build a classifier using the dataset.

So, we will take our J48 object with its buildClassifier method, and we'll supply it with our dataset. This will create a classifier for the tree object.

Once we have done this, we can print its capabilities with the toString method. This will print the types of attributes that it can classify. Once we have done so, we can print its graph. This will give us the exact decision tree graph that it has developed, which it has also trained.

Running the code will provide the following output:

Since the first print statement was getCapabilities, that has been printed. The classifier has been trained, and it can incorporate Nominal, Binary, Unary, and a list of some more attributes that it can train itself on. digraph J48Tree in the output is the tree that was generated with those attributes. That is how we can develop a classifier.

Suppose that we want to train one more classifier, using Naive Bayes; first, we will have to incorporate the NaiveBayes class that is available in the bayes package of the weka.classifiers class:

import weka.classifiers.bayes.NaiveBayes;

Next, we will create an object, nb, for NaiveBayes, and pass the dt dataset to the buildClassifier method of nb:

NaiveBayes nb = new NaiveBayes();
nb.buildClassifier(dt);
System.out.println(nb.getCapabilities().toString());

When this has finished, the classifier will be trained, and we will be able to print its capabilities.

Run the code again to get the following output:

In the preceding screenshot, you can see that the Naive Bayes classifier has been trained, and it has provided the attributes upon which the classifier can be trained.