OSDL-DBT-1



An On-Line Transaction Processing Test Kit



Similarities and Differences to the TPC-W™



Version 1.0



June 20, 2002

Table of Contents

1Similarities and Differences to the TPC-W™ Specification 3

1.1Components 3

1.2Definition of Terms 3

1.2.1RBE vs. Driver 3

1.2.2EB vs. EU 3

1.2.3Web Interaction vs. Interaction 3

1.2.4Transaction Management Server 4

1.3Implementation Rules 4

1.3.1The Driver 4

1.3.2 The Database 5

1.3.2.1Database Entities, Relationships, and Characteristics 5

1.3.2.2Scaling and Database Population 5

1.3.2.3ACID Properties 5

1.3.2.4Integrity Rules 6

1.3.2.5Checkpoints 6

1.3.2.6Isolation Level 6

1.3.2.7180-Day Space Requirement 6

1.3.2.8Foreign Keys 6

1.3.2.9Stored Procedure 7

1.4The TPC 8

1Similarities and Differences to the TPC-WSpecification

1.1Components

TPC-W is composed of a Remote Browser Emulator (RBE), a Payment Gateway Emulator (PGE) and the System Under Test (SUT). The SUT includes network components, web servers, application servers and a database server.


OSDL-DBT-1 has only a driver, a transaction management server and a database server. The is no PGE used to simulate credit card authorizations.

1.2Definition of Terms

Some terms defined in the TPC-W do not apply to the OSDL-DBT-1 test kit and some have different meanings. The TPC-W terms will be used if they apply to both cases. Where terms differ, they will be defined for the OSDL-DBT-1 test kit.

1.2.1RBE vs. Driver

The Remote Browser Emulator (RBE) is the software component that drives the TPC-W workloads.1 It creates and manages the Emulated Browsers (EB). The OSDL-DBT-1 driver controls the DBT-1 workloads. It creates and manages the Emulated Users (EU).

1.2.2EB vs. EU

The Emulated Browser (EB) refers to the entity (e.g. a process or thread) that emulates a user communicating via a browser for HTML content. Packets are sent through HTTP and TCP/IP protocols over a network connection to the SUT.2 An Emulated User refers to the entity (e.g. a thread) communicating with the transaction management server by sending request input and receiving output data via TCP/IP sockets and thus eliminates the need for HTTP servers in the kit.

1.2.3Web Interaction vs. Interaction

The term web interaction in the TPC-W benchmark refers to a complete cycle of communication between the EB and the SUT. The exchange of messages may include HTTP requests, cookies, HTML pages, image files or other web objects.3 There are no web objects in the OSDL-DBT-1 test. An interaction refers to a cycle of communication between an EU and the transaction management server. The exchange of messages includes request input and output data.

1.2.4Transaction Management Server

While the TPC-W benchmark requires web servers to serve web objects, the OSDL-DBT-1 test kit contains a transaction management server that performs the transaction management functions.


In the OSDL-DBT-1 test kit, the transaction management server is the middle tier connecting the driver and the database. It performs transaction management by creating and managing database connections and a transaction queue.


Web logging4 does not apply to the OSDL-DBT-1 test.


Other terms, such as "user session", "customer", "shopping session", "database transaction", "randomly selected with [x..y]", and "non-uniform random function (NURand)", are the same in both cases. Refer to the TPC-W Specification Clause 2.1 for the definition of these terms.

1.3Implementation Rules

1.3.1The Driver

The driver has the same responsibilities as the RBE5, except for providing security for some web interactions. In the TPC-W benchmark, the secure web interactions are: Buy Request, Buy Confirm, Order Inquiry, and Order Display. In secure web interactions, all communications between the EB and the SUT must be encrypted and communicated with SSL version 3 or with TLS using SSL_RSA_WITHRC4_128_MD5 as the cipher suite.6 The OSDL-DBT-1 driver does not provide any security mechanisms.


Like the TPC-W benchmark, the OSDL-DBT-1 driver uses TCP/IP sockets for all network communication.7

However, the TPC-W Specification requires that a new socket must be created for each user session and user sessions can not share a socket connection.8 The OSDL-DBT-1 driver creates a socket connection for each EU, and keeps that socket open for the duration of the test.


The EU sends input following the rules defined in the TPC-W Specification Clause 2.3 "Input Requirements" with some exceptions. Due to limitations in the SAP DB, some of the stored procedures have the number of input and output parameters reduced. For a description of stored procedure limitations, refer to Section 1.3.2.9.


In the TPC-W benchmark, each web interaction results in the presentation of a page to the EB.9 After receiving the page, the EB parses that page, and selects the web interaction to be requested next. In the OSDL-DBT-1 test, each interaction results in the output data being sent directly to the EU. After receiving the data, the EU selects the next interaction to request. The EU chooses the navigation options in the same way as the EB.10 The driver maintains the interaction mix defined in the TPC-W Specification Clause 5.2.1.


After each interaction, the EU simulates a user thinking (think time) by sleeping for a certain period of time. Think time is generated randomly according to the TPC-W Specification Clause 5.3.


The EU ends a user session when the conditions defined in the TPC-W Specification Clause 6.2.2.1 are met or when the duration of the test expires. The TPC-W Specification requires the EBs to close the network connection to the web servers when the user session ends, and start a new session for a new EB.11 The OSDL-DBT-1 driver keeps a socket open for the duration of the test.

1.3.2 The Database

1.3.2.1Database Entities, Relationships, and Characteristics

The OSDL-DBT-1 database consists of 8 individual base tables as defined in the TPC-W™ Specification Clause 1.3.1. Table layouts are defined in the TPC-W™ Specification Clause 1.4.

1.3.2.2Scaling and Database Population

The OSDL-DBT-1 data generator mostly follows the guidelines in the TPC-W Specification. The OSDL-DBT-1 data generator generates dates starting from the year 1900 while the TPC-W data generator guidelines requires generating dates starting from the year 1800.12 The OSDL-DBT-1 database follows the scaling rules defined in the TPC-W Specification Clause 4.1, 4.2 and 4.3 and its database population follows the rules defined in the TPC-W Specification Clause 4.1.


No clustering, partitioning, or replication is used in the OSDL-DBT-1 database.

1.3.2.3ACID Properties

The current version of the OSDL-DBT-1 test kit does not contain any ACID tests defined in the TPC-W™ Specification Clause 3.1.

1.3.2.4Integrity Rules

The current version of the OSDL-DBT-1 test does not enforce the integrity rules as defined in the TPC-W Specification Clause 1.7.


1.3.2.5Checkpoints

The current version of the OSDL-DBT-1 test kit does not perform checkpoints as defined in the TPC-W Specification Clause 5.5.3.

1.3.2.6Isolation Level

The current version of the OSDL-DBT-1 test uses the default isolation level set provided by the SAP DB.


In the SAP DB, the default isolation level is 1.13 With this isolation level, dirty reads are not possible, and non-repeatable reads as well as phantom phenomenon can occur.14


The TPC-W Specification requires a Level 3 isolation level between any two TPC-W update transactions, and a Level 2 isolation between any TPC-W read-only transaction and a TPC-W arbitrary transaction.15 This rule does not apply to three transactions: New Products, Best Sellers, and Admin Confirm. The database reads to generate the list of 50 New Products/Best Sellers items may be performed at an isolation level of 0. Admin Confirm requires the database to generate five item IDs; this can be performed at an isolation level of 0.16


For the definitions of dirty reads, non-repeatable reads, phantom phenomenon, TPC-W update transactions, TPC-W™ read-only transactions, and TPC-W™ arbitrary transactions, please refer to the TPC-W™ Specification Clause 3.1.4.1.

1.3.2.7180-Day Space Requirement

The current version of the OSDL-DBT-1 test kit does not follow the 180-day space computation rules defined in the TPC-W™ Specification Clause 4.4.

1.3.2.8Foreign Keys

The current version of the OSDL-DBT-1 test kit does not implement the foreign keys as defined in the TPC-W™ Specification Clause 1.4.

1.3.2.9Stored Procedure

Stored procedures are created to execute the interactions. Stored procedures accomplish the processes defined in the "Processing Definition"17 sub sub-sections for each interaction. In the TPC-W™ Specification, a response page is defined for each web interaction. The OSDL-DBT-1 stored procedures return the same data required in the response page section with some exceptions.


Due to limitations in the SAP DB, stored procedures can not always return the required information defined in the TPC-W™ Specification.

Performance Data

The OSDL-DBT-1 data analysis tool analyzes some performance information (e.g. the percentage of each interaction executed during the measured interval, average interaction response time, and number of interactions executed per second). The OSDL-DBT-1 results are not comparable to the TPC-W™ results. The performance data differences are noted below:

1.4The TPC

The TPC is a non-profit corporation founded to define transaction processing and database benchmarks and to disseminate objective, verifiable TPC performance data to the industry. TPC and TPC Benchmark W are trademarks of the Transaction Processing Performance Council. For more information about the TPC please contact info@tpc.org or visit www.tpc.org.

1 TPC-WSpecification Version 1.7 Clause 6.1.1.

2 TPC-WSpecification Version 1.7 Clause 2.1.3.

3 TPC-WSpecification Version 1.7 Clause 2.1.4.

4 TPC-WSpecification Version 1.7 Clause 2.1.8.

5 TPC-W™ Specification Version 1.7 Clause 6.1.4.

6 TPC-W™ Specification Version 1.7 Clause 2.2.1.

7 TPC-W™ Specification Version 1.7 Clause 6.1.6 and 6.1.7.

8 TPC-W™W™ Specification Version 1.7 Clause 6.1.7.

9 TPC-W™ Specification Version 1.7 Clause 5.2.2.2.

10 TPC-WSpecification Version 1.7 Clause 5.2.2.3.

11 TPC-W Specification Version 1.7 Clause 6.2.2.2.

12 TPC-W Specification Version 1.7 Clause 4.7.1.

13 Reference Manual: SAP DB P 211

14 Reference Manual: SAP DB P 212

15 TPC-W™ Specification Version 1.7 Clause 3.1.4.

16 TPC-W™ Specification Version 1.7 Clause 2.12.3.1, 2.13.3.2 and 2.16.3.3.

17 TPC-W™ Specification Clause 2.3.3, 2.4.3, 2.5.3, 2.6.3, 2.7.3, 2.8.3, 2.9.3, 2.10.3, 2.11.3, 2.12.3, 2.13.3, 2.14.3, 2.15.3, and 2.16.3.

18 TPC-W™ Specification Version 1.7 Clause 5.7.1.

19 TPC-W™ Specification Version 1.7 Clause 5.1.5.

20 TPC-W™ Specification Version 1.7 Clause 5.3.1.2.

21 TPC-W™ Specification Version 1.7 Clause 5.4 and Clause 5.6.2.