The Grand Prix of Database Management Systems

Benchmarks for these products are defined by the "Transaction Processing Performance Council" (TPC). The TPC-C benchmark, for example, models an inventory control system. Comparisons are made based on the number of transactions per minute (TpmC) accumulated by each DBMS.

The benchmarking results appear prominently in each DBMS's printed ads. For example, by late 1994 Informix announced a triunfant 3,118.2 TpmC rating. Sometime later, Oracle took over, announcing 5,369 TpmCs. The following day, Sybase reported 4,545 TpmC, which still trailed Oracle but showed a similar level of performance.

As in all performance tests, each contestant presents only part of the truth. For example, Oracle might have been ahead- but the products used for benchmarking will not be available until February of 1996. And they used 3.5 gigabytes... of main memory! To complement what you read in the ads, read the full TPC reports.

Furthermore, noone mentions the "proprietary" databases: Tandem made a whopping 20,918.03 TmpCs with their NonStop SQLMP. IBM's DB2 is second (behind Sybase) in cost performance. And Digital's Rdb was the absolute winner in (now obsolete) TPC-A benchmarks.

These companies invest hundreds of thousands of dollars in expensive hardware configurations and in low-level programming to accelerate their systems in a way no friendly fourth-generation language will ever be able to. Their final applications and your typical Guatemalan application have as much in common as a Formula 1 racing car and a small pickup truck!

The only way to evaluate a DBMS correctly, is to design your own performance tests, based on the expected transaction volumes. Remember that, for these companies, the grand prix is, in the end, to win your business.