The HammerDB workload is based on and intended to be as close as possible to the example published in the TPC-C specification. As such the implementation is intentionally non-optimized or biased towards any particular database implementation or system hardware (being open source you are free to inspect all of the HammerDB source code). The crucial element is to reiterate the point made in the previous section that the HammerDB workloads are designed to be reliable, scalable and tested to produce accurate, repeatable and consistent results. In other words HammerDB is designed to measure relative as opposed to absolute database performance between systems. What this means is if you run a test against one particular configuration of hardware and software and re-run the same test against exactly the same configuration you will get exactly the same result within the bounds of the random selection of transactions which will typically be within 1%. Results Any differences between results are directly as a result of changes you have made to the configuration (or management overhead of your system such as database checkpoints or user/administrator error). Testing has proven that HammerDB tests re-run multiple times unattended (see the autopilot feature) on the same reliable configuration produce performance profiles that will overlay each other almost identically. The Figure below illustrates an example of this consistency and shows the actual results of 2 sequences of tests run unattended one after another against one of the supported databases with the autopilot feature from 1 to 144 virtual users to test modifications to a WAL (Write Ahead Log File). In other words HammerDB will give you the same results each time, if your results vary you need to focus entirely on your database, OS and hardware configuration.
Consequently when you begin to make changes you are able to quantify the impact of these changes. Examples include modifying database parameters, changing database software or operating system or upgrading or modifying hardware. Taking a baseline of a database system with a HammerDB workload is an ideal method to ensure that optimal database configurations are engineered put into production.