Arguably, the most common beginning errors with database benchmarking is for a user to select a single point of utilisation (usually overconfigured) and then extrapolate conclusions about system performance from this single point. Instead, HammerDB has always encouraged the building of performance profiles to fully understand how a database system behaves.
HammerDB v4.11 automates this practice by introducing the concept of performance profiles for TPROC-C workloads. This functionality available with both GUI and CLI enables you to group a number of related TPROC-C workloads together to build a profile across multiple benchmarks.
Typically, this functionality will be used with an automated workload where you have defined an increasing sequence of Virtual Users to identify the levels of peak performance on a system or compare different software and hardware configurations. The performance profile allows you to group these related TPROC-C workloads together with a single profile ID.
By default, the performance profile ID is set to 0 meaning that a particular HammerDB job is not related to other jobs and by setting this profile ID to a positive integer then groups all jobs with the same number together.
Using Performance Profiles with Autopilot
In the GUI, when jobs are enabled, there is the new option of performance profile ID. To activate performance profiles set this ID to a positive value with the Job Options dialog.
As this setting is related to automated workloads the same setting is also available in the Autopilot options.
Then run an autopilot sequence exactly as before.
When the run has completed, view your results with the HammerDB web service and you will see a new section called Performance Profiles and a profile listed under the ID you have created. The profile lists the number of Jobs grouped under that ID and pinpoints the job with the maximum NOPM and TPM for that job and the number of Active Virtual Users.
Clicking on the profile ID will show a graph of both NOPM and TPM (both of which can be interactively deselected) allowing you to view the performance profile for your grouped jobs.
Using Performance Profiles with the CLI
In advanced server environments, most users will use scripts to run a performance profile sequence. The equivalent of setting the Performance Profile ID in the GUI is done with the jobs profileid command. Without an argument it will report the current profile ID, by default 0 for no profile and with an integer argument it will set a Profile ID.
hammerdb>jobs profileid Performance profile id set to 0
hammerdb>jobs profileid 1 Setting performance profile id to 1
hammerdb>jobs profileid Performance profile id set to 1
If running a sequence of tests as follows, setting the Profile ID requires just one command before any of the workload is run.
#!/bin/tclsh
# maintainer: Pooja Jain
set tmpdir $::env(TMP)
puts "SETTING CONFIGURATION"
jobs profileid 1
dbset db maria
dbset bm TPC-C
giset timeprofile xt_gather_timeout 1200
giset commandline keepalive_margin 1200
diset connection maria_host 127.0.0.1
diset connection maria_port 3306
diset connection maria_socket /tmp/mariadb.sock
#
diset tpcc maria_user root
diset tpcc maria_pass maria
diset tpcc maria_dbase tpcc
diset tpcc maria_driver timed
diset tpcc maria_rampup 2
diset tpcc maria_duration 5
diset tpcc maria_no_stored_procs false
diset tpcc maria_allwarehouse false
diset tpcc maria_timeprofile true
diset tpcc maria_purge true
#start CPU
metstart
puts "TEST STARTED"
foreach z { 1 2 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88
92 96 100 104 108 112 116 120 } {
loadscript
vuset vu $z
vuset logtotemp 1
vucreate
tcstart
tcstatus
vurun
tcstop
vudestroy
}
metstop
puts "TEST COMPLETE"
Running the webservice will then show the performance profile
and clicking on the Profile ID the graph of the NOPM and TPM for all of the jobs under that ID.
The CLI also adds a jobs profile command that will report the results of jobs with the profile ID specified.
hammerdb>job profile 1
{
"66686B0D61A903E233236323": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 08:19:41",
"activevu": "1",
"nopm": "16254",
"tpm": "37709"
},
"66686CB661A903E243430373": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 08:26:46",
"activevu": "2",
"nopm": "35434",
"tpm": "82497"
},
"66686E6161A903E263536313": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 08:33:53",
"activevu": "4",
"nopm": "71549",
"tpm": "166405"
},
"6668700E61A903E283834363": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 08:41:02",
"activevu": "8",
"nopm": "135774",
"tpm": "316012"
},
"666871C161A903E203230333": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 08:48:17",
"activevu": "12",
"nopm": "196216",
"tpm": "455867"
},
"6668737A61A903E283131383": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 08:55:38",
"activevu": "16",
"nopm": "258999",
"tpm": "601697"
},
"6668753861A903E283437363": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 09:03:04",
"activevu": "20",
"nopm": "310608",
"tpm": "721658"
},
"666876FB61A903E293532383": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 09:10:35",
"activevu": "24",
"nopm": "358359",
"tpm": "832467"
},
"666878C361A903E203836353": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 09:18:11",
"activevu": "28",
"nopm": "399416",
"tpm": "928993"
},
"66687A8E61A903E213737353": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 09:25:50",
"activevu": "32",
"nopm": "438920",
"tpm": "1018825"
},
"66687C5E61A903E263536313": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 09:33:34",
"activevu": "36",
"nopm": "471456",
"tpm": "1095396"
},
"66687E2961A903E263635343": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 09:41:13",
"activevu": "40",
"nopm": "522600",
"tpm": "1213621"
},
"66687FF761AA03E293130303": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 09:48:55",
"activevu": "44",
"nopm": "548843",
"tpm": "1275641"
},
"666881CE61AA03E283137393": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 09:56:46",
"activevu": "48",
"nopm": "569173",
"tpm": "1322266"
},
"666883A861AA03E293734373": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 10:04:40",
"activevu": "52",
"nopm": "591268",
"tpm": "1373413"
},
"6668858561AA03E223038303": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 10:12:37",
"activevu": "56",
"nopm": "609316",
"tpm": "1415349"
},
"6668877161AA03E243737363": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 10:20:49",
"activevu": "60",
"nopm": "628853",
"tpm": "1462056"
},
"6668896861AA03E253531393": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 10:29:12",
"activevu": "64",
"nopm": "642036",
"tpm": "1491259"
},
"66688B6861AA03E203133313": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 10:37:44",
"activevu": "68",
"nopm": "656450",
"tpm": "1524772"
},
"66688D7161AA03E293331363": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 10:46:25",
"activevu": "72",
"nopm": "661515",
"tpm": "1536318"
},
"66688F8361AA03E203331333": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 10:55:15",
"activevu": "76",
"nopm": "670828",
"tpm": "1557467"
},
"6668919E61AA03E263931383": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 11:04:14",
"activevu": "80",
"nopm": "672855",
"tpm": "1563938"
},
"666893BF61AA03E213631303": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 11:13:19",
"activevu": "84",
"nopm": "680334",
"tpm": "1580944"
},
"666895E761AA03E253330363": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 11:22:31",
"activevu": "88",
"nopm": "685096",
"tpm": "1590859"
},
"6668981461AA03E213832303": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 11:31:48",
"activevu": "92",
"nopm": "680334",
"tpm": "1580664"
},
"66689A4661AA03E213938343": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 11:41:10",
"activevu": "96",
"nopm": "679016",
"tpm": "1577343"
},
"66689C7E61AA03E213730313": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 11:50:38",
"activevu": "100",
"nopm": "678694",
"tpm": "1577827"
},
"66689EB561AA03E233739373": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 12:00:05",
"activevu": "104",
"nopm": "674706",
"tpm": "1566707"
},
"6668A0F461AA03E243230353": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 12:09:40",
"activevu": "108",
"nopm": "670643",
"tpm": "1557979"
},
"6668A33361AA03E293836323": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 12:19:15",
"activevu": "112",
"nopm": "665293",
"tpm": "1545308"
},
"6668A57A61AA03E203433363": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 12:28:58",
"activevu": "116",
"nopm": "661495",
"tpm": "1536360"
},
"6668A7CF61AA03E253833343": {
"db": "MariaDB",
"bm": "TPC-C",
"tstamp": "2024-06-11 12:38:55",
"activevu": "120",
"nopm": "655745",
"tpm": "1523566"
}
}
Performance Profiles Summary
With the new performance profile, functionality HammerDB enables you to group related TPROC-C jobs under an ID and use it to compare performance across benchmarks at different levels of utilisation and different configurations.
The profile ID chart will show performance for each individual profile.
And you can then quickly pinpoint specific jobs and drill down into their performance characteristics.
If you are not already building performance profiles manually, then this functionality makes it easier than ever to fully analyze your database performance.