Custom Java filters

This section describes when and how to create a custom hook file for Java method filtering.


When recording a Java Record Replay script, the recorder selects which methods are recorded in the Vuser script and which are left out. VuGen uses hooking to filter the methods to include in the script.

VuGen comes with built-in support for the RMI, JMS, and JDBC protocols. The built-in filters for these protocols are designed to record only the server-related traffic relevant to your testing goals.

If your protocol is not supported, you must define your own custom hook file. Custom Java protocols, proprietary enhancements and extensions to the default protocols, and data abstraction all require a custom filter definition.

Note: If no hook file is defined for your protocol, your Vuser script will be empty.

Creating a custom hook file demands planning and a good understanding of the protocols your application uses. When hooking is implemented correctly, the Vuser script should be well correlated and ready for compilation. For more details on how to select which methods and classes to hook, see Determine which elements to include.

When you record a method, the methods that are called from the recorded method—whether directly or indirectly—are not recorded.

To record a method, VuGen must recognize the object upon which the method is invoked, along with the method's arguments. VuGen recognizes an object if it is returned by another recorded method provided that:

  • The construction method of that object is hooked

  • The object is a primitive or a built-in object

  • The object supports a serializable interface

You can create a custom filter to exclude unwanted methods. When recording a Java application, your script may include calls to methods that do not affect the server, such as calls to a local utility or the GUI interface. These calls are usually not relevant to your testing goals, and it would be correct to filter them out.

Before creating a test, we recommend that you become familiar with your application and determine its primary classes and methods, so that you know which ones to include in your recording.

If you are not familiar with your application's classes, VuGen allows you to record with a stack trace that logs all of the methods that were called by your application. To record with stack trace set the log level to Detailed.

Back to top

Determine which elements to include

When designing a custom filter, we recommend that you start by choosing the appropriate built-in filter as a base filter. You can then customize the filter using one of the following approaches:

Top-down Include the relevant package and exclude specific classes that are not part of the client-server activity. This is recommended if you are familiar with your application and you can identify a well-defined layer that implements all client-server activity without involving any GUI elements.
Bottom-up Use the default filter and refine it by adding individual methods or classes. Use this approach if you cannot identify a well-defined layer or if you are not familiar with your application. Do not add all AUT packages and then try to remove extra component one by one.

Include or exclude elements as follows:

  • If, as a result of your including a class, your script has many unrelated method calls, try modifying the filter to exclude the irrelevant methods.

  • If you identify a non-client/server call in your script, exclude its method in the filter.

  • During recording, VuGen may detect an unknown input argument, for example, an argument whose construction it had never encountered before. If this argument supports serialization, VuGen serializes it by saving it to a file in a special format. During replay, VuGen reconstructs the argument by deserializing it.

  • VuGen serializes objects passed as arguments that were not included by the filter. We recommend that you include this object in the filter in order to track its construction and activity instead of using it in its serialized form. You can identify serialized objects in the script by searching for calls to the lr.deserialize() method in your script. For more information see Java scripts correlation - serialization.

  • Exclude all activity that involves GUI elements.

  • Add classes for utilities that may be required for the script to be compiled.

Back to top

Create a custom Java filter

When preparing a script, you may need to customize the filter several times in order to achieve the optimal filter. An optimal filter records the relevant methods without introducing a large number of irrelevant calls to the script.

Note: Rerecording a script after modifying the filters will overwrite all manual changes. Therefore, if you plan to add manual code to your script such as control flow or message statements, make sure to do so after you have a functional script that runs inside VuGen.

To define a custom hook file:

  1. Create a user.hooks file in the VuGen installation classes folder, typically <LoadRunner root>\Virtual User Generator\classes. VuGen automatically searches for this file when recording. For structural details about the hook file, see Hook file structure.

  2. Open the Recording Options dialog box (Ctrl+F7) and select the Log Options node. For Log Level, select Detailed.

  3. Record your application. Click Start Record (Ctrl + R) to begin and Stop (Ctrl + F5) to end.

  4. View the script's steps. If you can determine the business logic from the steps and apply correlation, you may not need to create custom filters. If, however, the script is very long or hard to maintain and correlate, you should customize the script's filter.

  5. Try to identify the high-level method in the call that captures or wraps one or more client server calls. You can do this by opening the AUT source files (if they are available) or by viewing a Stack Trace of the script.

  6. Set the filter to include the relevant methods. For more information, see Determine which elements to include.

  7. Record the application again. You should always rerecord the application after modifying the filter.

  8. Repeat steps 4 through 7 until you get a simple script which can be maintained and correlated.

  9. Correlate the script. In order for your test to run properly, you may need to insert a correlation to capture a value and use it at a later point in the script. For more information about the built-in correlation mechanism, see Java scripts correlation - serialization.


  • Adding custom hooks to the default recorder is a complicated task and should be considered carefully as it has both functional and performance consequences.
  • Do not modify any of the other .hook files as this might damage the VuGen recorder.
  • Incorrect hooking definitions can lead to incorrect scripts, slow recording, and application freeze-up.

Back to top

Hook file structure

The following section describes the structure of a typical hook file used when creating a custom filter. For details, see Create a custom Java filter.

The hook files are structured as .ini files where each section represents a hook definition. Regular expressions are supported in some of the entries. Any entry that uses a regular expression must start with an exclamation point (!).

Note: When you use a filter such as !.*, the ! indicates the beginning of a RegExp—not a Regexp negation.

class      = MyPackage.MyClass
method     = MyMethod
signature  = ()V
ignore_cl =
ignore_mtd = 
ignore_tree = 
cb_class   = mercury.ProtocolSupport
cb_mtd = 
general_cb = true
deep_mode = soft | hard
make_methods_public = true | false
lock = true | false

Structure description


Specifies the name of this section in the hook file. Hook-Name must be unique across all hook files. A good practice is to give the fully qualified class name and method.

For example:



A fully qualified class name.

Regular expressions can be used to include several classes from the same package, a whole package, several packages, or any class that matches a name.

The following example filters for any class that starts with javax.jms and is followed by at least one character:

class = !javax\.jms\.*


The simple name of the method to include.

Regular expressions can be used to include more than one method from the class.

For example:

method = getQueueName


The standard Java internal type signature of the method.

To determine the signature of a method, run the command

javap -s class-name

where class name is the fully qualified name of the class.

Regular expressions can be used to include several methods with the same name, but with different arguments.

For example:

  • signature = !.* will match any possible signature, thus causing any method in this class to be recorded into the script regardless of signature.
  • signature = !\(Lorg/omg/CORBA/ORB;\).* will match any signature starting with (Lorg/omg/CORBA/ORB;).

ignore_cl (optional)

A specific class to ignore from the classes that match this hook.

This can be a list of comma-separated class names. Each item in the list can contain a regular expression. If an item in the list contains a regular expression, prepend an exclamation point (!) to the class name.

For example:

Ignore_cl = !com.comp.jms.Queue,!com\.comp\..*

ignore_mtd (optional)

A specific method to ignore.

When the loaded class method matches this hook definition, this method will not be hooked. The method name must be the simple method name followed by the signature (as explained above).

To ignore multiple methods, list them in a comma-separated list.

To use a regular expression, prepend an exclamation point (!) to the method name.

For example:

ignore_cl = open, close

ignore_tree (optional)

A specific tree to ignore.

When the name of the class matches the ignore tree expression, any class that inherits from it will not be hooked, if it matches this hook definition.

To ignore multiple trees, list them in a comma-separated list.

To use a regular expression, prepend an exclamation point (!) to the class name.

This option is relevant only for hooks that are defined as deep.


The callback class that gets the call from the hooked method. It should always be set to mercury.ProtocolSupport.

The built-in hook definitions may use other classes, such as mercury.jms.JMSSupport.

cb_mtd (optional)

A method in the callback class that gets the call from the hooked method. If omitted, it uses the default, general_rec_func.

For cases where you only need to lock the subtree of calls, use general_func instead.


The general callback method.

This value should always be set to true.


Deep mode refers to classes and interfaces that inherit or implement the class or interface that the hook is listed for. The inherited classes are hooked according to the type of hook: Hard, Soft, or Off.

  • Hard. Hooks the current class and any class that inherits from it. If regular expressions exist, they are matched against every class that inherits from the class in the hook definition. Interface inheritance is treated the same way as class inheritance.

  • Soft. Hooks the current class and any class that inherits from it, only if the methods are overridden in the inheriting class.

    If the hook lists an interface, then if a class implements this interface those methods are hooked. If they exist in classes that directly inherit from that class they are also hooked.

    However, if the hook lists an interface and a class implements a second interface that inherits from this interface, the class will not be hooked.

    Note: Regular expressions are not inherited but converted to actual methods.

  • Off. Only the class listed in the hook definition and the direct inheriting class are hooked. If the hook lists an interface, only classes that directly implement it are hooked.

make_methods_public (optional)

Any method that matches the hook definition is converted to public.

This is useful for custom hooks or for locking a sub tree of calls from a non-public method.

Note: Applies only during record. During replay, the method uses the original access flags. In the case of non-public methods, it will throw java.lang.VerifyError.

lock (optional)

When set to true (default), this locks the sub tree and prevents the calling of any method originating from the original method.

When set to false, this unlocks the sub tree, records any method originating from the current method (if it is hooked), and invokes the callback.

Back to top

See also: