OTLPBench
The OTLPBench Testbed
Overview
Benchmarking is an important, yet often overlooked, aspect of any database management system (DBMS) research and development effort. Despite several advancements over the last decades, the deployment of a comprehensive testing platform with a diverse set of data sets and workloads is still non-trivial. In many cases, researchers and developers are limited to a small number of workloads to evaluate the performance characteristics of their work. This is due to the lack of a universal benchmarking infrastructure, and because it is typically hard to gain access to real data and workloads due to privacy and security concerns. This results in many unnecessary engineering efforts, limits availability of reference workloads, and makes scientific results difficult to compare.
To remedy these problems, we present OLTP-Bench, an extensible “batteries included” DBMS benchmarking testbed that is tailored for on-line transaction processing (OLTP) and Web-oriented workloads.
Some of the key features are:
- Precise rate control (allows to define and change over time the rate at which requests are submitted)
- Precise transactional mixture control (allows to define and change over time % of each transaction type)
- Access Distribution control (allows to emulate evolving hot-spots, temporal skew, etc..)
- Support trace-based execution (ideal to handle real data)
- Extensible design
- Support for statistics collection (microseconds latency and throughput precision, seconds precision for OS resource utilization)
- Automatic rendering via Javascript (and many available and gnuplot matlab scripts)
- Elegant management of SQL Dialect translations (to target various DBMSs)
- Targeting all major relational DBMSs and DBaaS offerings via the standard JDBC interface (tested on MySQL, Postgres, Oracle, SQLServer, DB2, HSQLDB, Amazon RDS MySQL, Amazon RDS Oracle, SQL Azure)
- Store-Procedure friendly architecture
Research papers:
Systems:
- Fork OLTPBench the project on Github.
- Visit the dedicated website http://www.oltpbenchmark.com.