ABLE Functional Agent Beans

The following Able Agents which perform specific functions are provided with the ABLE package.
 

Genetic Search Agent

  The genetic search agent uses the genetic search object to perform optimizations.

Neural Classifier Agent

Neural Clustering Agent

Neural Prediction Agent
 

5.0 Using the ABLE Framework

To develop a neural network application agent or to add neural network processing to a Java application using the ABLE framework, perform the following steps:
  1. Develop a functional specification describing the requirements for your application.
  2. Develop a high-level design.
  3. Determine which functions can or should be neural networks.
  4. For each neural network agent, perform the following steps:
    1. Define the problem to be solved by the neural network bean or agent.
    2. Define the source data.
    3. Generate or create train-and-test data sets.
    4. Generate or create a neural network.
    5. Define acceptable performance levels.
    6. Train the neural network using the Able Agent Editor.
    7. Test the neural network to ensure acceptable performance.
  5. Integrate the agent or bean with an application, if desired.
The Able Agent Editor performs many of the steps listed above. The following sections list the functions in the Able Editor and where they fit into the application development process. The following sections also describe the other application development steps in more detail. 

Defining the Problem Type

The neural network core beans supported by the ABLE framework are capable of solving many types of problems. In some cases, you can use several different models to solve the same type of problem, so it is important to recognize the basic task you are trying to perform. The type of task, along with information about the source data, can guide you in selecting the correct neural network model and architecture to use.

ABLE provides a set of function-specific agents which greatly simplify the process of constructing a neural network agent. However, in order to use them, you must first decide the type of problem you are trying to solve.

Classification
The most common type of problem. Examples are credit scoring, defect classification, and image recognition.
Modeling
The neural network approximates the mathematical function used to convert a set of input data to a set of output data.
Forecasting
A method of using historical data to make predictions about the future. Examples of time-series forecasting problems are inventory control, labor forecasting, and commodity trading.
Reconstruction
A type of classification problem where the input and output patterns are the same. The object is to match the input pattern to the closest matching training pattern and to output the original training pattern. The effect is to recognize a "noisy" version of a pattern and to output a "clean" version.
Clustering
A type of analysis performed on data when there is reason to believe that patterns exist in the data. An example is analysis of market response to products or services.
Routing
A type of problem similar to the traveling salesman problem, where the object is to visit a number of destinations starting and ending at a common point, that uses the self-organizing routing network. The goal is to minimize the distance traveled. Routing long-distance trucks or airplanes is an example of this type of problem.

Neural Model Capabilities

Figure 12 provides an overview of the capabilities of each of the neural network models. For information on each model refer to "Neural Network Models".

Figure 12. Neural Network Model Capabilities
Problem type  Back Propagation  Adaptive Resonance  Feature Map  Routing  Radial Basis  Recurrent  Temporal Difference 
Classification 
Modeling 
Forecasting 
Reconstruction 
Clustering 
Routing 

If the problem you are trying to solve matches one of these problem types, the ABLE neural network agents can generate a complete agent  to solve your problem. See "Application Generation" for details.

If the problem or data does not fit into one of the problem types already defined, then you can create your own model. To construct your own model for an application that is not supported by the problem types already defined, you need to understand the capabilities of each neural network model. See "Neural Network Models" for details regarding the neural network models supported by the ABLE framework. See the examples for details on creating your own models. 


Defining the Source Data

ABLE can read data from files with several different formats or file types. To define the source data, create a data Import bean and specify that it is a file Import object. Select one of the supported file types and supply any additional parameters that are required.

When the Able agent is stepped, the next record is read from the source data file associated with the active Import bean. Regardless of the file type or the format of the data in the file, the data is read, converted to the data type specified as the Import buffer data type, and placed in the Import buffer.

Supported File Types

ABLE supports the following file types:
Text
You can explicitly define the number of fields in a record and how they are delimited. NNU defaults to space-delimited fields, with one record per line of text. Fixed-width fields are not supported if they are not separated by at least one delimiter character. Fixed-width files without field delimiters can be loaded into a database and processed.
Database (not supported)
You must specify the database file. ABLE reads the data definition and extracts the field information.
Spreadsheet (not supported)
The ABLE framework reads the most common type of spreadsheet files, the .wks format. ABLE loads the field name and data type information.

Data Representation

One of the key issues in developing neural networks is the presentation of the training data to the network. Most neural network models require your raw data to be converted to a range of 0.0 to 1.0, or, for symmetric networks, -0.5 to +0.5. Data representation influences network accuracy, network size, and the time required to train the network. You can select a data representation to increase the influence of a parameter you realize is significantly more important than others in your data.

Following an import object, a translation object is normally used to convert your raw data into a format the network can best utilize. As an example, assume your input data contains a discrete variable with five possible states called A, B, C, D, and E. Figure 13 shows several alternate data representations which could be selected.

Figure 13. Data Representation Comparison
Category  Numeric  One-of-N Code  Thermometer Code  Binary Code 
0.1  0 0 0 0 1  0 0 0 0 1  0 0 0 
0.3  0 0 0 1 0  0 0 0 1 1  0 0 1 
0.5  0 0 1 0 0  0 0 1 1 1  0 1 0 
0.7  0 1 0 0 0  0 1 1 1 1  0 1 1 
0.9  1 0 0 0 0  1 1 1 1 1  1 0 0 

The numeric representation uses only one value which is the minimum possible. This keeps the network size small, but it will be more difficult for the network to learn and consequently training time will be longer. If there are relationships between the categories, a numeric representation implies that the categories are sorted according to those relationships. If the parameter to be represented is of relatively low importance and network size is a concern, the numeric representation could be used.

The one-of-n code representation is shown with a length of five since there are five categories. It will create larger networks, since there are now five network inputs compared to one with the numeric data type. More inputs mean that this parameter will have more connections in the network and enhance the importance of this parameter in the results obtained. Also, the network will learn more easily, and train more quickly. Note, though, that four of these inputs are always zero so use this data type for categories where there are no relationships between the values. For example, A-E could represent broad occupational categories such as Professional/technical, Management, Sales, Manufacturing, and Administrative.

The thermometer code representation is similar to the one-of-n but implies a relationship between the categories. For example, A-E could represent a scale from best to worst.

The binary code is shown with a length of 3 as that is the smallest size needed to represent five separate categories. Up to eight categories could be represented with a length of 3, 16 with a length of 4, and so on. If there are more than ten categories, use a binary representation rather than thermometer or one-of-n. In this example, it saves two network inputs.

Just as your raw data is translated for input to the network, the network's output may need to be translated back from its output range to symbols. Typically this is the mirror image of the translation used to provide the input. If you want your answers back in the categories A through E and selected the numeric representation, for example, define a translation that maps 0.1 to A, 0.3 to B, and so on. Use the threshold function to convert numbers from 0.0 to 0.2 to an integer 1, 0.2 to 0.4 to an integer 2, and so on; then define a translate table to convert the integers to alphabetic categories.

You may wish to artificially increase the number of inputs to emphasize a parameter such as gender. While a binary field of length one could be used to represent male and female, increasing the length will create more network nodes and connections between this and other parameters in your data. Making the length four, and mapping male to 0000 and female to 1111, quadruples the number of connections and provides a stronger distinction between records which differ in this value.

Data representation is a key consideration in designing neural network applications. Some experimentation should be expected to determine the best representation based on the results from both the network output and training experience. 


Creating the Training and Testing Data Sets

When using neural networks, an important consideration is the proper selection of training data. This includes deciding which subset of examples to use for training and which to use for testing (they should be distinct). The selection of training data is a critical factor in determining whether and how quickly your network converges (trains). Neural networks may relieve you of some algorithm development, but they may also place a heavy burden on training data selection and representation.

The Able Editor provides several features that aid in working with data sets. The source data can be in a text file, database file, or spreadsheet format file. To use the Able editor to convert the file to a different type, create an Import bean and an Export bean, and then connect them. Open the Import object on your source file, and open the Export object on an output file of the desired format. Save the agent and use the Cycle button where steps per cycle is set to the number of source records.   Able reads each record from the source file and writes it to the Export (output) file in the new format.

In a similar manner, you can add a Filter bean between the Import and Export objects to process or modify the data set records before they are written out. You can write a custom AbleBean to process the data however you desire. Or, you can use the Translate filter to convert data using the most common neural network data types. These include symbols, numbers, binary codes, one of N codes, and thermometer codes. See "Translate Editor" for details on using the Translate editor, and "Data Types" for a list of supported data types.

The Able function-specific beans also provide the ability to read a single source data file and generate a complete agent for neural classification, clustering, or prediction.  


Creating an Agent

Creating an Able Agent is equivalent to constructing an application. The input data comes through an Import object, event, or User-defined Function, is processed by one or more AbleBeans, and finally is passed on to Export beans or User-defined Functions, or sent as events.

To create an agent, perform the following steps:

  1. Create a container agent by selecting File->New->  and selecting the type of agent
  2. Create a set of Import, Network, Fuzzy, Submodule, Filter, and Export beans.
  3. Configure each bean by giving it a name and setting parameters.
  4. Connect the beans together with data flow connections, event connections, and property connections.
You can perform these steps in almost any sequence you desire. For example, you may wish to create an AbleBean and then configure it completely. Then you can add another bean and connect it to the first object. You may start training a Network bean and later decide to add another Network bean to the agent.

The only restrictions are that you must have two objects to create a buffer, event, or property connection and that you must completely configure a bean before it can be turned ON to process data flow. When an object is OFF, no data flows through it to other connected beans. You can see this graphically because OFF beans are grayed out.

In addition to constructing an agent from scratch, you can use the Able Editor function-specific beans to automatically generate an agent based on a problem type and source data definition. This may be sufficient to solve your application problem or you can then modify the generated agent. 


Creating a Neural Network Bean

To create a neural network bean, you must select the model type from the Learning icon palette and then specify the architecture parameters for the neural network by using the bean Customizer dialog.  The neural network bean is added to the container agent. However, you must save either the agent or the neural network bean explicitly to make a permanent copy of the network bean. 

Training the Neural Network

After the agent and the training data set are created, you are ready to train the neural network. Depending on the type of neural network, you must select training parameters in such a way as to achieve desired results in the training process. Also, you must determine how to measure when the training process is complete.

The training requirements vary widely for each neural network model. See "Neural Model Capabilities" for details about the recommended training procedure for each supported neural network model.

In general, training a neural network bean using the Able Editor requires the following steps:

  1. Reset the neural network weights.
  2. Present a series of training patterns to the neural network.
  3. Use Inspectors to monitor the state of the neural network.
  4. Modify training parameters as required.
Resetting the network weights effectively erases all prior learning in the neural network. Reset places most neural networks in a state with random connection weights. These connection weights are adjusted during the training process, controlled by the learning algorithm associated with each neural network model. The Able Editor provides the capability to change all of the neural model training parameters. Sometimes the neural network must be trained a number of times before acceptable performance is attained. Often, the training parameters must be adjusted to obtain an acceptable level of neural network performance.

The Inspectors provided allow you to view the neural network parameters and arrays in both text and graphic modes. The Inspectors are critical in allowing you to evaluate whether the training regimen is on a track to producing a working neural network application.
The Able Editor provides several levels of control over the training process. A step presents a single pattern to each AbleBean in the agent. A cycle presents a number of patterns based on the environment-variable steps per cycle in the Able Editor properties. When you run the agent, each bean is presented with patterns until a breakpoint is reached. Breakpoints allow you to specify stopping conditions based on neural network parameters. See "Breakpoints" for details.
 

Testing the Neural Network

In traditional software development, you define functional specifications and test the software to ensure that it meets those requirements. The neural network development process is similar. Regardless of how you specify acceptable network performance, you must test the system to verify that it meets your functional specifications.

When the completion criteria have been met, the network is trained. At this point you can try the test cases against the network to see whether the network truly functions in the desired manner. Use the Able Editor to do this type of preliminary testing. Select a data set file of test cases, and then run the test cases through the neural network. Create an Inspector on the Export bean and enable data logging. This provides a record of the neural network (and module) performance for later analysis.

Able agents allow you to set up training and test Import beans and then switch between them by simply setting the desired Import bean to the data flow ON state. This turns the other bean to data flow OFF.

Using Breakpoints to Control Training

<< Note: breakpoints are not supported yet in the Able Editor >>

A breakpoint defines a condition that causes the AbleDefaultAgent to halt the processing of a module. When a breakpoint is reached, a Cycle or Run command stops the processing and returns control to the user. Breakpoint conditions are checked at the end of each agent step, whether the processing command is Step, Cycle, or Run.

A breakpoint has the following form:

    <Object> <Parameter> <Condition> <Value>
You can set multiple breakpoints on a single neural network parameter. This effectively allows you to set ranges on parameters. The safest way to set a breakpoint is with the >= or <= conditions. If you set exact equality conditions and equality does not occur, training proceeds. Because at least one module processing step is performed before a breakpoint is reported, a breakpoint condition that is true before a step or cycle may not be true after the step, when the breakpoint conditions are checked.

Breakpoints often are used to limit the number of training epochs for a neural network. Another common approach is to set a breakpoint on a neural network error parameter. When the error rate falls to the specified level, the neural network performance can be evaluated by switching to an Import object with test data. If necessary, neural network parameters can be modified, and the breakpoint can be cleared and set at a new, lower, error rate. Breakpoints can be set interactively from the interactive development environment or automatically from control scripts.
 
Topics    Index   APIs


Last modified: Mon Jul 26 09:37:15 CDT 1999