Why both TPM and NOPM Performance Metrics?

The reason for reporting both TPM and NOPM performance metrics is for both historical and performance reasons. On the historical side HammerDB started out as Hammerora and only supported Oracle so it only needed to report Oracle TPM. However as other databases were added they all record TPM in a slightly different way so it wasn’t possible to compare TPM between different database for example Oracle and SQL Server.  Therefore NOPM was added.

NOPM is a close relation of the official tpmC statistic recording only new orders per minute, however it is strictly forbidden to use the term tpmC unless you are doing an official fully audited and approved TPC-C benchmark and therefore the term NOPM signifies this difference.  Nevertheless because it comes from the schema itself that means that NOPM can be used to compare different databases whereas TPM cannot.

There should always be a close ratio between TPM and NOPM for example with Oracle NOPM is close to  1/3rd of TPM.  This also brings additional benefit that if the values diverge it may indicate errors in the testing. In particular because TPM records all transactions, both commits and rollbacks during the measuring period on the database whether HammerDB induced transactions or not.

Therefore one question is why not just use NOPM? There are 2 reasons. Firstly for database performance engineering studies TPM relates directly to the metrics captured from the database. For example  the TPM  value is the same number multiplied by 60 as the transactions per second value in the load profile of an Oracle AWR report (the SQL Statement actually gets the number from the AWR repository). Secondly and most importantly the TPM values for all databases are from in-memory tables again using the Oracle example v$sysstat (All v$ tables are memory based) and therefore selecting from these does not impact the test – this means we can then run the graphical transaction counter and select from this table without affecting the test results. However as NOPM is selected from the district table if we ran this during the test at the same frequency it would impact the results by introducing locking and therefore we take this value at the start and end of the test only.

Therefore this is why we have 2 values of TPM and NOPM and both are as valid as each other (graphs for TPM and NOPM should look identical) that can be summed up in a simple explanation. If you are comparing one database only then TPM is better as you can relate this directly to all of the database performance metrics you have captured however if you are comparing different databases then you should use NOPM instead when reporting results.

Author