---

Measurement Technology

Individual aspects of broker performance are measured by dedicated suites of tests. All suites employ the same measurement framework. To form a specific test, the framework is augmented by an action to be measured on the client side and servants to be instantiated on the server side.

The concept of measurement framework is employed in both the C++ and Java environments. The implementations in both languages are functionally equivalent, except for short explicit garbage collection loop in Java client, which is aimed at ensuring similar starting conditions for each measurement.

Measurement Framework

The measurement framework carries out a measurement in five stages.

Warmup

In the warmup stage, the framework instantiates threads that carry out the action to be measured. When the threads start, the framework waits for a random time from 5 to 20 seconds to make sure the system is under a stable load before the measurement progresses.

Client Resource Usage

In the client resource usage stage, the framework collects information about client resource usage in 60 samples taken over the period of 60 seconds. For each resource, the minimum, maximum and average values are kept.

Server Resource Usage

In the server resource usage stage, the framework collects information about server resource usage, same as in the client resource usage stage.

Solo Measurement

In the solo measurement stage, the framework records 5000 individual measurements observed by a randomly selected thread. Information such as statistical distribution and quartiles is calculated from the individual measurements.

Shared Measurement

In the shared measurement stage, the framework records minimum, maximum and average values of measurements observed by all threads over the period of 60 seconds. Information such as overall minima, maxima and averages is calculated from the shared measurements.

Each measurement is repeated 5 times.

Acquiring Time Information

Solaris

On Solaris, the gethrtime () system call is used to obtain the time information. The resolution of thus obtained time is 1 ns.

Windows

On Windows, the QueryPerformanceCounter () system call is used to obtain the time information. The resolution of thus obtained time is the same as the frequency of the processor clock.

Acquiring Resource Usage Information

Solaris

The memory usage information on Solaris is collected from the /proc/self/psinfo file, where the resident and swapped application memory sizes are stored, from the kstat structures, where the kernel memory size is stored, from the sysconf () system call, which reports the total physical memory size, and from the swapctl () system call, which reports the total swapped memory size.

The network usage information on Solaris is collected from the kstat structures, where the network traffic information is stored. The network usage information does not include the traffic over the loopback interface in bytes, which is not reported by the operating system.

The processor usage information on Solaris is collected by hooking the thr_create () and pthread_create () system calls, where the application thread count is kept, and from the kstat structures, where the total processor usage is stored.

Windows

On Windows, most information is collected from performance data accessible under HKEY_PERFORMANCE_DATA key in the registry. The data is not actually stored in the registry; the registry functions cause the system to collect the data from its source. The data come in form of performance objects which represent various resources found in a system. Each object has a name and a set of named counters associated with it. The counters associated with an object can come in several instances for entities where it makes sense, eg. processes, network interfaces, etc.

The memory usage information is collected from Working Set and Page File Bytes counters of the corresponding instance of Process object, where the resident and swapped application memory sizes are stored, from Pool Paged Bytes and Pool Nonpaged Bytes counters of the Memory object, where the kernel memory sizes are stored and from the GlobalMemoryStatus () system call, which reports the total physical and total swapped memory sizes.

The network usage information is collected from Bytes Sent/sec, Bytes Received/sec, Packets Sent/sec and Packets Received/sec counters of all instances of Network Interface object.

The processor usage information is collected from Thread Count counter of the corresponding instance of Process object, where the application thread count is kept, and from % User Time, % Processor Time and % Privileged Time counters of all instances of Processor object, where the total processor usage is stored.

Java

In Java, the information is collected using native code methods contained in platform dependent dynamic shared libraries, which utilize the above mentioned methods. The only exception is the application thread count, which is determined using pure Java methods and represents the number of active threads in the root thread group.