TdBench V7.

2 - A Query Driver Designed for

Data Warehouse Benchmarks

Part 1 of 4 – What is TdBench and How is it used?

By Doug Ebel
Teradata Americas Benchmark COE

Copyright © Teradata Corporation - 2016

 Major Features of TdBench

• Easily simulates complex workloads with simple commands

• Tracks tests, captures DBMS/TASM configurations and archives logs
• Provides analysis views and macros that select DBQL and Resusage
data by RunID
• Setup builds analysis views from
– Resusage/DBQL in DBC, or
– DBC unioned to PDCR, or
– current DBC release vs prior release DBC
• Open architecture in Linux allows running other applications including
“brand x” query drivers to leverage tdbench reporting
• Support OS workers to support non-query activities such as data
loading. Can be used to prepare benchmark in parallel.
• tdb.sh for command & control of active tests plus integration of tools like
speedtest, backup_work, wrapup, and setting upTPC-DS and TPC-H
benchmarks

2 Copyright © Teradata Corporation - 2016

 Frequently Asked Questions (1 of 2)

Who Created TdBench?

• Written and supported by Doug Ebel of Teradata Americas Benchmark COE

What is the history of TdBench?

• Originated in 2009 as MS/DOS batch scripts using Teradata Queue tables.
• In October 2010, Presented at Partners and version 5.0 published on Teradata
Developer Exchange
• Version 6.0 released in April, 2012 as a Linux shell script with queues implemented
as Linux pipes to bteq workers. Features were enhanced over versions 6.1 to 6.4.
• Version 7.0 released in October, 2015 with enhanced setup that integrated PDCR
and DBC Resusage and DBQL along with support for multiple Teradata servers
and ability to queue commands for either SQL or Linux workers.
• Version 7.1 released March, 2016 with enhanced performance in “queries per
second”
• Version 7.2 released April, 2016 with support for integrating prior releases of DBC
resusage and DBQL history as well as PDCR history.

3 Copyright © Teradata Corporation - 2016

 Frequently Asked Questions (2 of 2)

What Kind of Benchmarks is TdBench Used for?

• Competitive benchmarks – Determine which vendor can support needs
• Proof of Concept – Test design decisions for new application
• Migration Impact – Moving to a new platform
– One time use benchmark where slice of workload is selected and run with same data on
new platform to predict capacity and ability to catch-up continuous update processes
after migration

• Upgrade Impact – Test before and after maintenance to validate performance

– Reusable benchmark using queries sampled from production workload and adapted to
not modify production data.
– Optionally, data can be sampled from one or more platforms to create seed data that
can be extrapolated when needed for a benchmark, adapted to platform size.
– Queries are modified to ensure key and date references will return rows when executed
now or in the future

• Application Health – Ensure Application is capable of meeting SLAs

– Reusable benchmark with selected queries from application. Tests run weekly
– Looks for performance degradation based on data growth and application or PDM
changes.

4 Copyright © Teradata Corporation - 2016

  What Is TdBench And How Is It Used?
 How To Run TdBench? (Next)
 How Does TdBench Work?
 How To Use TdBench Analysis Tools?

Copyright © Teradata Corporation - 2016

5 Copyright © Teradata Corporation - 2016
 TdBench V7.2 - A Query Driver Designed for
Data Warehouse Benchmarks

Part 2 of 4 – How To Run TdBench

By Doug Ebel
Teradata Americas Benchmark COE

Copyright © Teradata Corporation - 2016

 What Does It Take To Run A Test?
define serial test of all reporting queries
queue q1 scripts/reporting/*.sql
worker q1 me
run

scripts/reporting/…

Rpt01.sql q1.qhold q1_user_rpt01 (BTEQ)

But, the prospect wants us to run 10 concurrent sessions!

7 Copyright © Teradata Corporation - 2016

 Multiple worker can be started to run queries
concurrently
define workload test of all reporting queries
queue q1 scripts/reporting/*.sql
worker q1 me 10
run

scripts/reporting/…

Rpt01.sql q1.qhold
q1_user_rpt01 (BTEQ)

Yeah, but they want the test to run continuously for 30 minutes

8 Copyright © Teradata Corporation - 2016

 Tdbench supports fixed period tests by adding a time
specification to run.
define workload test of all reporting queries
queue q1 scripts/reporting/*.sql
worker q1 me 10
run 30m

scripts/reporting/…

Rpt01.sql q1.qhold
q1_user_rpt01 (BTEQ)

But they have different workloads with different SLAs

9 Copyright © Teradata Corporation - 2016

 Tdbench Allows Multiple Queues To Be Defined With
Different Groups Of Logon IDs for Each
define mixwork test of reporting and data mining
queue q1 scripts/reporting/*.sql
worker q1 user_rpt* 10
queue q2 scripts/mining/*.sql
worker q2 user_hvy* 3
run 30m
scripts/reporting/…

Rpt01.sql q1.qhold
q1_user_rpt01 (BTEQ)
scripts/mining/…

Mng01.sql q2.qhold q2_user_hvy01 (BTEQ)

10 Copyright © Teradata Corporation - 2016

 Running TdBench Interactively
tdbench.sh
..... Selected server: acme. Profile created 2016-04-15 20:17:04. Logs in acme_logs
Starting tdbench 7.2 at 19:30:26 - Copyright (C)- Teradata Corp
..... Note that all file use commands change current directory to: /sdb1/ebel/qdriverdev
Enter Cmd (e.g.: HELP): define demo Show how tdbench works
------------------- Test:demo Title:Show how tdbench works -------------------
Test demo Cmd: queue q1 scripts/sample.lst
..... 3 lines added to temp/q1.qhold which now has 3 lines
Test demo Cmd: worker q1 me

..... There are now 1 workers

Test demo Cmd: run
..... Test:demo RunTitle:"Show how tdbench works"
RunId:0063
..... all sessions started ... test begins
at 2 seconds, 19:31:52, waiting for all work to complete

..... q1 Submitted 3 queries with 0 errors.

..... Run 0063 Complete. ... updating TestTracking table
tdbench 7.2 Execution Summary on 19:31:56 - Copyright (c) 2015 - Teradata Corporation
RunID:0063 Name:demo Title:Show how tdbench works ran from 2016-04-16 19:31:52 to 2016-04-16 19:31:53
q1 Workers:1 ActiveAtEnd:0 Executed:3 Errors:0 AveSecs:.20 MaxSecs:0.38
Total seconds of run execution: 0.69 (RUNSECONDS)
Queries submitted for execution: 3 (RUNCOUNT)
Count of queries with errors: 0 (RUNERRORS)
Note: Logs from the temp directory been moved to acme_logs/RUN0063 (LASTLOGDIR)
Enter Cmd (e.g.: HELP):
11 Copyright © Teradata Corporation - 2016
 Running TdBench .cmd Files in Batch
cat scripts/sample.cmd
cat scripts/sample.cmd
# simple test to validate TdBench is working
define checkout Validate TdBench installation
queue anyname scripts/sample.lst
worker anyname me
run

12 Copyright © Teradata Corporation - 2016

 Running TdBench .cmd Files in Batch
tdbench.sh scripts/sample.cmd
..... Selected server: acme. Profile created 2016-04-15 20:17:04. Logs in acme_logs
Starting tdbench 7.2 at 20:18:40 - Copyright (C)- Teradata Corp
..... Note that all file use commands change current directory to: /sdb1/ebel/qdriverdev
20:18:40 # simple test to validate TdBench is working
20:18:40 define checkout Validate TdBench install
------------------- Test:checkout Title:Validate TdBench install -------------------
20:18:40 queue anyname scripts/sample.lst
..... 3 lines added to temp/anyname.qhold which now has 3 lines
20:18:40 worker anyname .
..... There are now 1 workers
20:18:40 run
..... Test:checkout RunTitle:"Validate TdBench install" RunId:0065
..... all sessions started ... test begins
at 1 seconds, 20:18:41, waiting for all work to complete
..... anyname Submitted 3 queries with 0 errors.
..... Run 0065 Complete. ... updating TestTracking table
tdbench 7.2 Execution Summary on 20:18:45 - Copyright (c) 2015 - Teradata Corporation
RunID:0065 Name:checkout Title:Validate TdBench install ran from 2016-04-16 20:18:41 to 2016-04-16 20:18:42
anyname Workers:1 ActiveAtEnd:0 Executed:3 Errors:0 AveSecs:.22 MaxSecs:0.43
Total seconds of run execution: 0.72 (RUNSECONDS)
Queries submitted for execution: 3 (RUNCOUNT)
Count of queries with errors: 0 (RUNERRORS)
Note: Logs from the temp directory been moved to acme_logs/RUN0065 (LASTLOGDIR)
20:18:45 QUIT
Goodbye
Ending Teradata tdbench 7.2 at 20:18:45 - Active for 5 seconds
de110922@gss184:/sdb1/ebel/qdriverdev>

13 Copyright © Teradata Corporation - 2016

 Commands and Options For Modeling Workload
Queue q1 ….directory path or filename…
– Demonstrated on prior slides
Queue q1 “Select …. ; “
– A SQL statement enclosed in “ “
Queue q2 “tbuild –f … ” type=os
– Makes the queue for OS workers to launch OS type task … e.g. for ETL in mixed workload
Queue q1 “Select … where x=‘:1’ …;” parmfile=….filename….
– Parmfile has values for :1 :2 :3 … :9 appearing in “sql statement” or sql files
Queue q1 “select …. ; “ at=2m
– Specifies when query or OS command should execute. Can be used to replay production.
Specification is relative to start of test with suffixes “s” seconds, “m” for minutes, or “h” for
hour which can be fractional, e.g. .5m = 30s
Pace q1 5s
– Releases queries for execution no faster than the interval specified on pace.
Queue q1 “kill all”
– “kill”, “stop” can be queued to stop one queue from repeating. “kill all” will stop entire run
when one queue is completed.
How many Queues can you have? Answer: How many file names can you make
up instead of q1, q2, etc?

14 Copyright © Teradata Corporation - 2016

 Features To Automate Testing And Simplify Scripts
./tdbench.sh scripts/xxxxx.cmd
– From Linux prompt, allows execution of file with multiple tests. You can also use wild cards
or specify more than one .cmd file on the command line.
exec scripts/xxxxx.cmd
– Causes next input to be read from the cmd file. You can use INCLUDE as an alias to re-use
common statements in definition of different tests. (You can’t nest)
Non-tdbench commands are executed immediately in Linux
– Allows running bteq to switch views to different scale databases or rename files between
tests. (To run within a test, you must queue the statement). Allows you to interactively
build and correct a test using vi, less, etc.
save nnn xxxxx.cmd
– Writes the commands from a prior run (built interactive) into a file to edit and re-use
sleep 23:30:00
– Overlay to Linux sleep, will calculate how long to wait … e.g. until maintenance window
If [ $RUNERRORS –gt 100 ]; then GOTO=aname
– Uses Linux if statement to set a variable. Tdbench will keep reading until it finds “aname”
on DEFINE or LABEL statement. Other variables are RUNSECONDS, RUNCOUNT
read –p “Enter the description of this test series” TestDesc
– An example of a Linux command to set a variable you can use later in the script

15 Copyright © Teradata Corporation - 2016

  What Is TdBench And How Is It Used?
 How To Run TdBench?
 How Does TdBench Work? (Next)
 How To Use TdBench Analysis Tools?

Copyright © Teradata Corporation - 2016

16 Copyright © Teradata Corporation - 2016
 TdBench V7.2 - A Query Driver Designed for
Data Warehouse Benchmarks

Part 3 of 4 – How Does TdBench Work?

By Doug Ebel
Teradata Americas Benchmark COE

Copyright © Teradata Corporation - 2016

 How Does TdBench Work?

Linux Server Teradata

tdbench.sh Server
tdbenchsetup.sh RunId DBC
BenchSystemName
TestName ResUsage
StartTime
RptBenchSummary
ActualStopTime
RptTestDetail
SessionCnt DBQL
RunTitle
RptTestExplain
RunNotes
RptTestObjects
../qdriver ReportingSeconds
RptTestSQL
SystemType PDCR
RptTestSteps
DbmsVersion
../qdriver/xxx_logs ../qdriver/temp
RptTestSummary
Amps
ResUsageSawt
Nodes xxx_Benchmark
NodesUpAtStart
ResUsageScpu
NodesUpAtEnd TestTracking
ResUsageShst
AveNodeCpuAtStart
ResUsageSldv
AveNodeCpuAtEnd RptTestxxxxx
ResUsageSpdsk
RuleSetId RptTestxxxxx
ResUsageSpma
TdwmConfigname RptTestxxxxx
ResUsageSps
ResUsageSvdsk
ResUsageSvpr Customer
Databases
18 Copyright © Teradata Corporation - 2016
 Serial Test

Linux Server Teradata

Server
tdbench.sh SQL Worker s1
tdbenchrun.sh S1S1
tdb.sh DBC

ResUsage
q1
Queue
Master DBQL

../qdriver
PDCR
../qdriver/xxx_logs ../qdriver/temp
q1.qhold ../qdriver/temp/queues
RUN0001 xxx_Benchmark
query01 query04 q1.queue s1.worker
query02 query05 TestTracking
query03
TestRptxxxxx
TestRptxxxxx
TestRptxxxxx

Customer
Databases
s1.log tdbench.trace tdbench.log
19 Copyright © Teradata Corporation - 2016
 Workload Test

Linux Server Teradata

tdbench.sh Server
SQL Worker s1
tdbenchrun.sh DBC
SQL Worker s2 ResUsage
S2
q1 q2
Queue Queue
Master Master SQL Worker h1 DBQL

../qdriver PDCR
../qdriver/xxx_logs ../qdriver/temp
q1.qhold ../qdriver/temp/queues xxx_Benchmark
RUN0001
query01 query04 q1.queue
RUN0002 s1.worker
query02 query05 TestTracking
query03
q2.queue s2.worker TestRptxxxxx
q2.qhold TestRptxxxxx
TestRptxxxxx
query10
query11
h1.worker
query12 Customer
Databases
s1.log s2.log h1.log tdbench.trace tdbench.log
20 Copyright © Teradata Corporation - 2016
 Full Architecture Including OS Workers, tdb.sh, and
Sleep Master For Fixed Period Tests.

Linux Server Teradata

tdbench.sh Server
Sleep OS Worker o1 SQL Worker s1
tdbenchrun.sh
tdb.sh Master DBC
STOP
Status
OS Worker o2 SQL Worker s2 ResUsage
q1 q2
Kill Queue Queue
Master Master OS Worker o3 SQL Worker s3 DBQL

../qdriver
PDCR
../qdriver/xxx_logs ../qdriver/temp
q1.qhold ../qdriver/temp/queues
RUN0001 xxx_Benchmark
query01 query04 q1.queue
RUN0002 o1.worker s1.worker
query02 query05 TestTracking
RUN0003
query03
....
q2.queue o2.worker s2.worker TestRptxxxxx
RUN0135 q2.qhold TestRptxxxxx
TestRptxxxxx
query01 query04
query02 query05
a_master o3.worker s3.worker
query03 Customer
o1.log o2.log o3.log
Databases
s1.log s2.log s3.log tdbench.trace tdbench.log
21 Copyright © Teradata Corporation - 2016
  What Is TdBench And How Is It Used?
 How To Run TdBench?
 How Does TdBench Work?
 How To Use TdBench Analysis Tools? (Next)

Copyright © Teradata Corporation - 2016

22 Copyright © Teradata Corporation - 2016
 TdBench V7.2 - A Query Driver Designed for
Data Warehouse Benchmarks
Part 4 of 4 – How To Use TdBench Analysis Tools

By Doug Ebel
Teradata Americas Benchmark COE

Copyright © Teradata Corporation - 2016

 How To Get History Of All Test Runs In Last 7 Days?

Select * from xxx_benchmark.TestTracking

Where starttime >= current_date – 7
Order by starttime;

24 Copyright © Teradata Corporation - 2016

 What Does It Take To Analyze Run#21?

Select * from xxx_benchmark.RptTestDetail

Where runid = 21
Order by starttime;

25 Copyright © Teradata Corporation - 2016

 How Do I compare Run#21 Before Adding Nodes to
Run#43 After Reconfig?

Select * from xxx_benchmark.RptTestDetail

Where runid in (21, 43)
Order by starttime;

26 Copyright © Teradata Corporation - 2016

 How do I research changes in performance for
query045 across tuning and reconfiguration?

Select * from xxx_benchmark.RptTestDetail

Where queryname = ‘query045’
Order by starttime;

Where did the name “query045” come from?

Select /* tdb=query045 */ cust_no, inv_date, ....

Or
Set query_band=‘tdb=query045;’ for session volatile;

27 Copyright © Teradata Corporation - 2016

 How Do I get CPU load during Run#21?

Select thetime, sum(cpuuexec, sum(cpuuserv),

sum(cpuiowait), sum(cpuidle)
from xxx_benchmark.RptSpma
Where runid = 21
Order by thetime group by thetime;

Or …
Select * from xxx_benchmark.RptSystemCpu
Where runid = 21 Order by 1,2;

28 Copyright © Teradata Corporation - 2016

 Introduced in TdBench 7.1 – An Excel Reporting Template

29 Copyright © Teradata Corporation - 2016

 Using the TdBench Excel Reporting Template

30 Copyright © Teradata Corporation - 2016

 Using the TdBench Excel Reporting Template

31 Copyright © Teradata Corporation - 2016

 Macros Executed Under TdBench Reporting Template

32 Copyright © Teradata Corporation - 2016

 Where do I get the software?

Teradata Developers Exchange:

http://downloads.teradata.com/download/tools/tdbench7-2

What if I have questions?

Douglas H. Ebel
Benchmark & POC Center of Expertise

Teradata
3749 Marble Ridge Ln, Mason, Ohio 45040-3005
513 226-2072
doug.ebel@teradata.com
teradata.com

33 Copyright © Teradata Corporation - 2016