1Q 2002 FAQs

Q1 What is the difference between a Teradata macro and a stored procedure? Which should
: I use when?
A1 Although both a macro and a stored procedure can, in Teradata, be used for some of the same
: things—e.g., they both support parameterized SQL—they are really very different.

A macro has very restricted control logic that is limited to performing or not performing
ABORT statements, which undo changes. A stored procedure, on the other hand, supports
extensive procedural logic and more sophisticated error handling. It can also perform looping
logic or pass along statement values during a procedure. However, macros can return a
multirow answer set, while stored procedures can only send back a single set of values through
the parameter list.

Under some conditions, such as the straightforward insert of multiple rows, a macro will
perform better than a stored procedure because it bundles all statements into one
multistatement reques—a single parsing-and-recovery unit. Stored procedures do not support
multistatement requests, and each insert within a procedure is sent individually to the parser
and dispatched as a separate request, eliminating the potential for parallel single-row inserts.

For simple activities, particularly those involving multiple, repetitious SQL or requiring
multirow answer sets, use macros; where the required logic is beyond what a macro can
accommodate, turn to stored procedures.

Q2 How is data redistributed when I add new hardware to an existing Teradata

: configuration?
A2 A reconfiguration utility is provided with the DBMS to enable you to add new hardware with
: minimal intervention and system unavailability. Hash buckets are assigned evenly across all
parallel units (AMPs) within the Teradata system and are used by the file system to target data
row placement so a table's data is evenly distributed across all AMPs. When new nodes (and
with them new AMPs) are added, a subset of these hash buckets is reassigned in order to
include the new nodes in the even spread of data. Data rows associated with the reassigned
hash buckets are migrated onto the new AMPs on the new nodes.

This utility only moves the proportion of data equivalent to the percentage of increase in
number of AMPs, while leaving the majority of the rows untouched. All tables and indexes are
automatically relocated over the internal high speed interconnect, in parallel on all nodes
simultaneously, by issuing just a single command. When the reconfigure is complete, the
entire system, including all tables and indexes, is immediately and fully available, without the
need to rewrite load scripts or make any changes to jobs, queries or database parameters.

Q3
Can Teradata support star schemas?
:
A3 Teradata has handled "star" and "snowflake" joins for a long time. Often, the optimizer will
: build a plan that initiates a product join among several dimension tables after qualifying them
and then redistributing the result so it can easily join the large fact table. Teradata customers,
especially those in retail, have used this technique very effectively since 1988.
 Snowflake joins occur when star tables are joined to other tables in the query. Some of those
joins can reduce the cardinality of the star tables. The optimizer recognizes joins that reduce
cardinality and executes them prior to performing the product join, resulting in fewer rows
and, thus, a shorter merge join phase. The algorithm is sophisticated enough to recognize
multiple star joins in a single query and to recognize when one of the star joins is a snowflake
of another.

Most Teradata optimizations for star joins rely on a large fact table with a composite primary
index of columns that are the equi-join fields for the dimension tables. All of the dimension
tables with a join field included in the primary index must be included in the star join. This
allows the optimizer to join the dimension tables to produce the composite primary index and
thus execute a high-performing merge join known as a "row hash match scan."

Teradata is very good at dealing with queries that do not fit a pattern, no matter what the
underlying physical model. You won't limit the type of queries you can ask by using
dimensional modeling in your Teradata system. The Teradata optimizer can build an optimal
plan for any query involving the tables, regardless of whether it fits the star join conventions.

For this reason, and because the Teradata optimizer is good at delivering on complex requests,
and additionally because many users have found it easier to add subject areas using a third
normal form model, a normalized schema is often recommended. In addition, many users find
it easier to add subject areas with a normalized model. This recommendation is often
misinterpreted to mean that third normal form is required with Teradata, when in fact the
Teradata technology is completely neutral when it comes to the data model.

Q4 What happens when one query runs faster than another other during a synchronized
: scan?
A4 Teradata can perform a sync scan (synchronized full table scan) if more than one query is
: scanning the same table. It does this by starting the second query scan at the same data block
in the table where the first query is positioned. Then the queries share the reading of data
blocks going forward. When the first one finishes, the second query continues the scan until it
reaches its starting point.

While synchronization is encouraged, it's not forced. The optimizer selects sync scan only on
tables too big to fit into the data cache. Small table blocks are likely to be cached, and
therefore their I/Os are already potentially shared.

Once a sync scan begins, the DBMS makes a note of it and periodically checks the progress of
all participants. It keeps an eye on how many tables are involved in the scan and how much
memory is available to support the activity, and it monitors each query's progress. When
synchronization is no longer optimal, the DBMS will spin off one or more of the participants,
preventing fast queries from being hampered by slow queries.

Q5
When does Teradata reuse spool files within the same request?
:
A5 The Teradata optimizer can generate a query plan that will produce a spool file or temporary
: work file as output in one step for use as input in multiple subsequent steps. This can save
significant processing time, and the optimizer will choose to do this whenever it senses that
the same intermediate file will be built in different places in the plan. For instance, it will reuse
 files when a subquery accesses the same tables with the same selection criteria as the outer
query, even when the subquery goes on to aggregate the resulting data and the outer query
does not.

Correlated subqueries often benefit from reusable spools, as do multistatement requests

containing multiple insert/select statements that all select from the same source. Teradata will
select from the source table once into a re-usable spool file, which will then feed inserts to the
several tables.

Under some conditions, table triggers will result in a plan where reusable spool files maintain
both the base table and a trigger-specified table. When tables containing multiple join indexes
are updated in a batch mode, e.g., by means of an insert/select operation, the prejoining
happens once and the results are held in a reusable spool file, which is then applied to each of
the participating join indexes. This eliminates the need to do the same prejoin activity for each
join index structure requiring maintenance.

2Q 2002 FAQs
Q1 I am the Medical Director for a large (2.4 million member) healthplan insurer, and we
: are taking a serious look at an integrated enterprise decision support system that uses
Teradata as the back-end warehouse and platform. I should note that although I am a
physician, I also oversee the medical informatics area, and I am computer (hardware,
software, database and SQL) savvy enough to be dangerous.

We are currently a mainframe shop with primarily DB2 data marts, which are called
warehouses, and a number of distributed and specialized data marts in Oracle residing
on UNIX platforms. This is an old company, and there is a lot of vested interest amongst
some areas of the company in the existing systems, which some people had direct input in
developing (many years ago).

My purpose in writing and my question is this. One of the "objections" I hear being
raised from these areas about potentially moving to the new structure is that they cannot
write SQL directly against the warehouse in a Teradata world. And, since that is true (in
their view of the world) then obviously this has no value for power users and will not
meet the needs of the organization. My view of the world is that we have been brought up
on traditional relational databases (one can argue about DB2 filling that role) and the
standard use of SQL query tools.

We (including I) do not yet know enough about star schema architecture and how to
understand and access the data model. But, I refuse to believe that it is not possible to
write queries directly against the warehouse. Healthcare is certainly not unique in its
need to access data in many different and powerful ways.

So, can one write SQL directly against a Teradata warehouse? If not, how does one go
directly against the data, bypassing the GUI front ends that power users hate in order to
do complex queries in this environment?
A1 If this is the only issue then we better put your system on the truck and ship it to you!
:
Seriously though, this is exactly what we do. Our whole philosophy is to integrate all the data
in the organization into a single enterprise view—a logical data model that represents the
enterprise. We enable access for all users who have a question, regardless of scope or size. All
of those questions are asked directly against the enterprise copy of the data, not against
extracted versions or data marts. Over time, we would help you consolidate all the disparate,
disjointed decisioning systems you currently have into a single enterprise decision solution—
one copy of the data, one version of the truth.

Among our current customers, most use one or more query tools (GUI tools, as you call them)
from the leading BI tool vendors (Cognos, Business Objects, MicroStrategy ...). However, we
expect each of those tools to write SQL to send to Teradata, receive the answer set back and
format the reports as specified by the end user. Some of these tools can ask very complex
questions and submit very sophisticated SQL to Teradata. For a small set of power users, these
tools may well not be flexible enough to formulate the required SQL. When that is the case we
have both a batch SQL query tool and GUI-style SQL query tool, which allow users to code
SQL queries/scripts to their hearts' content.

Even better, though, is that Teradata loves complex queries. The more complex, the better.
The more joins, expressions and nesting, the better. I keep a QFH file where users send me
queries they or some tool have written. The current leader is a 25-page SQL statement (in 10-
point font) that accesses 56 different tables in the data model. Derived tables, subqueries, case
expressions, statistical functions (stdev, skew, kurtosis and linear regression) are all included
in our ANSI-compliant SQL implementation. And we have implemented a new SQL construct
called Ordered Analytics that allows what used to be complex jobs or large offloads to become
simple SQL statements. Rank, Quantile, Moving average, Moving Sum—all can be done
inside the DBMS. Where other DBMSs force multi-step scripts and intermediate tables to
handle many way joins, we want you to write the single SQL statement in all of its glory and
let Teradata's optimizer figure out how to execute it.

Speaking of the optimizer, we have a No Hint mechanism; instead of requiring you to tell us
which joins to execute in which order and what indexes to use when, our optimizer performs
all of those jobs automatically and consistently so you do not have to have an IT expert
between your users and their data. I personally guarantee that your power users and your
DBAs will be the most religious converts to Teradata once you bring it in. The freedom the
power users will gain and their ability to answer questions that they could never before even
consider asking will make them ask how they got along without it for this long.

If your power users are asking data mining questions, we have a data mining user interface
that allows them to do very high-powered SQL underneath. Statistical analysis, decision trees,
logistic regression and many other functions are supported, all taking advantage of the parallel
power of the Teradata platform by writing very complex SQL.

Regarding star schema, we recommend to our customers that they implement a physical
database design that closely mirrors the logical data model for the organization. Thus, on
Teradata systems the physical model tends to be a normalized model, with only occasional
denormalizations or summaries for especially performance-sensitive, high-volume queries.
Star schema is okay if you know the questions ahead of time, but a normalized model can
answer the next question.

Stars are notoriously difficult to extend to a new business area, but a normalized model is easy
to add to and extend as the enterprise's questions grow and extend. An enterprise model allows
the SQL query writers to have maximum flexibility across all of the areas of the business, e.g.
matching patients with procedures, costs and outcomes all in one query to the database. The
industry will tell you that star schema is required to avoid database performance problems and
overly complex queries. Teradata's optimizer and execution engine make those issues moot so
that you can afford to get the value of an enterprise model.
 Highmark and Anthem both are enthusiastic users of Teradata in your industry. One of my
favorite customer stories comes from Anthem where, among many uses, Teradata helps
analyze patient outcomes from several different perspectives. The implementation there helped
identify doctors, hospitals and procedures that had higher percentages of negative outcomes.
By identifying those situations so Anthem could correct them, Teradata contributed directly to
better patient care. We have a lot of good stories about helping companies grow, save money
or improve profitability from the revenue they get, but it is even more fun when we can help
save lives.

Q2 A person designs a temporary table and loads that table with many records. These
: records have many columns with "like" information, except for the Primary Index.
From a Load and Extract perspective, is it more efficient to define the primary index as
the first column in that table or can it be defined anywhere within the record with the
same efficiency and speed using SQL/Queryman?
A2 The position of the primary index column(s) as defined in the CREATE TABLE statement has
: no performance impact on loading, accessing or extracting table rows with Teradata. This is
true whether the structure is a permanent or a temporary table. In fact, the file system often
rearranges columns within a row before it physically stores the row. For example, any
variable-length columns are moved to the end of the row.

The row hash of an individual row is used to directly access the row, and also to reflect where
the row should be physically stored at the time it is inserted. Row hash information is carried
in the header of each row, no matter where in the row the columns it is based on reside.

Information held within the table header (of which there is a copy on each AMP) shows the
offset of each column within the row. Teradata will use this offset value to find the primary
index column directly in the actual row—when it is needed—without having to search the
entire row.

If the column selected for a primary index is, in fact, a variable-length column, then a small
amount of additional overhead will exist, in that we have to go not to one, but to two different
locations in the header information to determine the offset for that column. This is very minor,
but in general, fixed length data types are preferable for primary index columns.

Q3 I've been perusing the archives of Teradata Magazine and recently found a Spring 2000
: piece Todd Walter authored in which he nicely explains the wonderful uses of SQL
Views. You close this section with the statement that Teradata views entail no
performance hit because "Teradata handles views differently from other DBMS
products in this regard." Todd then refers to a query rewrite process that results in
references to the base tables on which the view is defined, and he comments that the
rewritten query is what gets passed to the optimizer.

I'm wondering to which DBMS systems you refer as those that handle views differently
than what you describe briefly. If I understand what you say in this piece, this is what
IBM's research prototype, System R, did and is what DB2 has always done. I would be
surprised if NonStop SQL and some others I might name didn't do the same thing in this
regard.

The rewrite involves taking a query that references a view (in a FROM clause) and
logically "merging" it with the original query that defined the view. This merged query,
 with references to the base tables only, is what the optimizer works on ("roughly
speaking"). Isn't this (again "roughly") the sort of thing you say Teradata does?
A3 There are easy views and hard views. An easy view would be a simple projection of a single
: table. A hard view might include some or all of joins, aggregation, subqueries, derived tables,
etc. Most DBMSs today handle the easy views. As the views become more difficult, they
switch to materializing the contents of the view and then performing the query over the result.
Informix is the worst, followed quickly by Oracle. You can find in their classes and
documentation warnings to avoid the use of any but the simplest views. The Data Mart
databases are really bad or don't even support views, such as RedBrick or Sybase IQ.

3Q 2002 FAQs
Q1 How do you adjust the time on a Teradata system. How would I adjust the time if the 'get
: time' Ferret command shows the time as 03:06:11, whereas the exact time is 03:36:11?
A1 Adjusting time across all the nodes in an MPP system is handled by a UNIX command. First,
: to view the time on each node, issue this UNIX command:

rallsh -sv date

To change all the nodes to a given time, issue this command from any node, and it will be
proliferated out to all nodes that are BYNET connected:
rallsh -sv date mmddHHMM2002
mm=2-digit month
dd=2-digit day
HH=2 digit hour
MM=2d digit minute

Only months, days, hours and minutes are changeable, not seconds.

Q2 When we set Priority Scheduler Facility (PSF) settings on a Teradata server for
: rationing CPU time, does it affect all the CPUs in all the nodes of Teradata MPP system?
A2 Priority Scheduler setup commands are issued by means of the UNIX schmon utility.
: Commands may be entered on any node in the system and will be proliferated out to all other
nodes. All nodes will have the same, exact settings. Each node, however, will have its own
instance of priority scheduler, and will manage its node's priorities independently of other
nodes in the system. For more information, please refer to the Teradata RDBMS Utilities
Guide.

Q3 Please validate that the following script will backup a Teradata database.
:
ARCHIVE Script:
logon demo1099/dbc,dbc;
ARCHIVE DATA TABLES (CRMDEMO),
RELEASE LOCK,
INDEXES,
FILE=textcrm;
LOGOFF;
RESTORE:
 logon demo1099/dbc,dbc;
RESTORE DATA TABLES (CRMDEMO),
RELEASE LOCK,
FILE=textcrm;
LOGOFF;

Would the archive/restore option automatically update the Data Dictionary entries?
A3 Your script is valid. You have an INDEX keyword in the script that will archive all the USI &
: NUSI for every table on the database. That typically, will increase the time taken for the
archive and need more tapes. Also, when restoring, you will need to BUILD your indexes.
May we suggest a better way, which would be to recreate the SIs after restoring.

Q4 I have to load NULL into a date field if the input has '?'. The code is:
:
FIELD SALES_TRANS_DT = CHAR (08) NULLIF SALES_TRANS_DT = '?';

Note that the input is a text field of length 8 bytes. If the input field has ? in the first byte,
the TPump works. If the ? is in any other position other than the first byte, TPump
writes the error log with the message "Invalid date supplied for
TEST11.SALES_TRANS_DT."

The same is the case for other type of fields like SMALLINT & DECIMAL. How do you
solve this riddle?
A4 Since the field is character, comparing it to '?' is equivalent to comparing it to '? '.
:
You can't say "nullif position('?' in sales_trans_dt) > 0", so you will have to clean up the input
before running TPump.

Q5 How can an individual user view what queries he/she currently has running on the
: Teradata system? We recently experienced a spool space problem while running a
passthrough SAS job to Teradata and thought the query was not getting terminated on
the Teradata side after canceling the job in SAS. The spool was not releasing after the
jobs completed for some reason.
A5 One suggestion is to use PMON if you are running V2R4.1.1 of Teradata. It shows what users
: are active. If there are processes blocking a user, for example in version (5.0.0), you can pull
up the SQL currently being executed (if the process is still active) by a given user. There is a
lot of other information that it provides...this is just a part.

Another suggestion is to use Teradata Manager to help you research the issue.

Or, if you just run a select from dbc.sessioninfox that will get all sessions that your userid has
running. Don't know if in SAS passthrough you logoff and on again for each query as that
would have a bearing on what you would see, but the view sessioninfo (without the x) shows
all sessions running on the system.
1Q 2003 FAQs
Q1 What is the best way to sum up three monthly fact tables (annualize) and load the
: results? In the example below the tables are relatively big, with about 65 million rows
 and 30 amt columns (decimal(14,2)) per row:

Case (acct exist in month1, month2, month3)

(amt_month1 + amt_month2 + amt_month3) * 4 -> amt
Case (acct exist in month1, month2)
(amt_month1 + amt_month2) * 6 -> amt
Case (acct exist in month1)
(amt_month1) * 12 -> amt
A1 Think of this as an OLAP problem, and the solution will be much simpler and more generic:\
:
Step 1: Combine the three monthly fact tables
DELETE AnnualTable ALL;
INSERT INTO AnnualTable ( acct, monthnumber, amt ) SELECT acct, 1,
amt_month1 FROM month1
;INSERT INTO AnnualTable ( acct, monthnumber, amt ) SELECT acct, 2,
amt_month2 FROM month2
;INSERT INTO AnnualTable ( acct, monthnumber, amt ) SELECT acct, 3,
amt_month3 FROM month3

Step 2: Sum (annualize) and load table

SELECT acct
,monthnumber
,SUM( amt ) OVER (
PARTITION BY acct
ORDER BY acct, monthnumber
ROWS UNBOUNDED PRECEDING ) AmtToDate
,AmtToDate * 12 / monthnumber AS ProjectedAnnualAmt
FROM AnnualTable
/* Optional - this piece gives you the row with the highest monthnumber only.*/
QUALIFY Rank(monthnumber) = 1

Of course, there are many variations to the OLAP approach. In particular, you can see where
an AnnualTable containing rows for ALL possible accounts and ALL possible months with
NULL amounts for non-existing account-months would allow you to also analyze missing
data problems, perhaps projecting annual amounts using algorithms for partial or incomplete
data.

Q2 What are the pros and cons of assigning surrogate keys in the data warehouse, as
: opposed to inheriting your source system natural keys? I realize that some of the pros for
surrogates are that you might have some space savings, and you can hide some
complexity of the source system keys, etc. But is it a good idea to assign different levels of
surrogate keys in each and every instance of parent and child tables and take it multiple
levels down? Should you always assign a surrogate when you have multiple sources (with
different number of composite natural keys) for the same entity?
A2 There are many tradeoffs to be considered. Let's assume the term NK stands for all of the
: columns of the natural key while SK stands for a single-column surrogate key. While there are
other design considerations, let's assume that the NK is NOT included in any tables other than
a single cross-reference table (which indicates the relationship of the SK to NK).

The pros:
1. Often improves performance of a large set of queries due to less I/O (assuming the NK is
 excluded from all tables). More rows fit in each block.

2. Improves performance on a large set of queries due to reduced join complexity. The impact
of both this issue and #1 above can be significant. The savings is proportional to the number of
bytes associated with the NK relative to the size of the rows returned in the intermediate and
final result sets. In my testing there has been I/O and CPU savings from 10% to 50%.

3. Improves ease of joins between tables since the join now involves only one column.

4. Reduces the probability that the user might construct a "killer query" (product join) because
less predicates may be necessary to perform joins. This issue alone can overshadow all of the
other issues.

5. Eases transition to historical continuity—that is to say that one can more easily associate
different values of the NKs to SKs. The cross-reference table can correctly record the fact that
an account number has changed simply by creating another row associating the new account
number with same surrogate key as the old account number. This may be important when any
part of the NK may change over time.

6. Reduces storage in many tables where only the SK is used. Another table of the cross-
reference between NKs and SKs are maintained and therefore increase storage requirements.
However, the overall storage requirements can often be reduced significantly.

7. Isolates the impact of change to the columns of the NK. Should any column of the NK
change it can be easily performed against a single table rather than the multitudes of tables
where the NK may have been used as a primary key or foreign key reference.

The cons:
1. All SKs must be assigned by some process. While there are many ways to assign SKs, one
can leverage the parallelism of Teradata and assign new SKs directly through SQL by using
the RANK or similar function.

2. May add a layer of obscurity by now having to use the SK where the business might be
accustomed to the NK.

3. May impede performance on a small set of queries where the columns of the NK may
provide selectivity for a secondary index on the columns of the SK.

There are cases where it makes sense to carry the NK along with the SK in all/some of the
tables.

I would not suggest that one use SKs as a substitute for each and every primary key in the
model. SKs bring a lot of maintenance and therefore should generally be applied only when
the benefit far outweighs the cost. I may not use a SK to replace a single or small two column
NK. It really depends on what can be gained in consideration of all of the other requirements.

Contributed by Doug Drake, Winter Corporation

Q3 I'm trying to figure out how to implement row-level security without using a ton of views.
: Our BI folks say it should be through Teradata. I say it should be through BI.
A3 Row-level security within Teradata is implemented using views. Having said that, it needn't
: involve a "ton" of views:

For an example you could implement row-level security using one view per fact table. Say you
are a retailer with multiple chains and you keep all of your data in the same tables, where
chain is part of the key on all tables. Some users work specifically for one chain and are not
allowed to see other chains' data. Corporate users may see data for all chains. Some users may
see two of the chains' data, etc.

Create a small table mapping chain groups to chain ids. Create another table with user ids and
their chain group. A view on top of both gives you the chains a user may see. The users have
access via views only and all the fact table views include a join to your security view with a
WHERE clause of WHERE user id=USER.

The beauty of having it implemented within Teradata means that all access—whether through
a BI tool or through a "power user" tool—is subject to the same rules.

Q4 Is there a method by which you can calculate the number of tactical (single amp) queries
: a Teradata system can support? I have been approached by one of our business partners
to support the development of an application that would be accessed by 5,000 users twice
a month. Assuming the worse case—that all 5,000 users tried to access their statement at
the same time—how long would it take for all queries to be satisfied?
A4 The NCR/Teradata performance people should be able to help you size it, but in general, it
: will depend on the query, its duration, and the volume of data to be returned. And, of course, it
depends on what else is running at the time. Since you mentioned that it's single amp, I'm
assuming that it will be primary index select. You should be easily able to handle 30 or more
queries per amp per second.

Do a single-AMP RETRIEVE step from

mwdw_prd_qmtbls.sxs_repeat_callers by way of the unique primary
index "mwdw_prd_qmtbls.sxs_repeat_callers.MTN = '2194033732'" with
no residual conditions. The estimated time for this step is 0.03
seconds.
-> The row is sent directly back to the user as the result of
statement 1. The total estimated time is 0.03 seconds.

And, if you use persistent connections, there's no logon time. So with a boatload of sessions
and on your configuration, you should be able to handle all 5,000 queries within a second or
two—worst case.

2Q 2003 FAQs
Q1 V2R5 Query Log Functionality
: If you have a lot of ad hoc users writing their own SQL, then query logging is a necessity.
How can Query Log help?
A1 The V2R5 Query Log should not be thought of as a single, binary, on/off kind of thing. It has
: a number of options for controlling the logging performed. Some of these options are going to
be reasonable to have on all the time. Others are going to be more costly in resources and
space and will probably only be used for investigating a part of the workload or a new
application in detail.
 For ad hoc, strategic queries, expect query-level logging to be on all the time. However,
logging every step executed for every one of those queries will probably be more expensive
than is desirable. But when there is a performance issue or when a new application is being
deployed, the ability to investigate at the step level will be extremely valuable.

Likewise for objects—when object logging is on, each object touched will result in a record in
the log. This will be a LOT of logging for a high volume of queries. And of course if you don't
wish to pay the price of that logging for every query, it will be difficult to use it to identify
unused objects.

Note that if the primary reason for logging objects is to understand what is used, then a daily
job to aggregate that log to capture one record per object rather than one per object per query
will radically reduce the long-term storage for the information. In some cases, it will probably
not be appropriate to turn on logging at all for certain user IDs. Most obvious perhaps is
TPump where query-level logging will result in a record in the log for every pack sent by
TPump—probably much more logging than desired relative to the value of the information.

Q2 Unnecessary Spool Step

: In the following explain plan, spool 2 is created in step 1, which is then all-rows-scan
read (last use) and put in spool 1 in step 3.1. So, what is the purpose of step 3.1? Spool 1
and Spool 2 are the same, and step 2 could have directly created Spool 1.

1) First, we lock xxx_DB.yyy_TBL for access.

2) Next, we do a SUM step to aggregate from

xxx_DB.yyy_TBL by way of an all-rows scan with a
condition of (
"(xxx_DB.yyy_TBL.date_col >= DATE
'2002-12-02') AND
(xxx_DB.yyy_TBL.date_col <= DATE
'2002-12-04')"), and the grouping identifier in field 1030.
Aggregate Intermediate Results are computed globally, then placed
in Spool 2. The input table will not be cached in memory, but it
is eligible for synchronized scanning. The size of Spool 2 is
estimated with low confidence to be 101 rows.

3) We execute the following steps in parallel.

1) We do an all-AMPs RETRIEVE step from Spool 2 (Last Use) by

way of an all-rows scan into Spool 1, which is built locally
on the AMPs. The size of Spool 1 is estimated with low
confidence to be 101 rows. The estimated time for this step
is 0.17 seconds.
2) We do a SUM step to aggregate from
xxx_DB.yyy_TBL by way of an all-rows scan
with a condition of (
"(xxx_DB.yyy_TBL.date_col >=
DATE '2002-12-02') AND
(xxx_DB.yyy_TBL.date_col <=
DATE '2002-12-04')"), and the grouping identifier in field 1.
Aggregate Intermediate Results are computed globally, then
placed in Spool 4. The input table will not be cached in
 memory, but it is eligible for synchronized scanning. The
size of Spool 4 is estimated with high confidence to be
793,963 rows.

4) We do an all-AMPs RETRIEVE step from Spool 4 (Last Use) by way of

an all-rows scan with a condition of ("Field_4 > 0") into Spool 1,
which is built locally on the AMPs. Then we do a SORT to order
Spool 1 by the sort key in spool field1. The size of Spool 1 is
estimated with high confidence to be 264,755 to 793,963 rows. The
estimated time for this step is 0.66 to 1.56 seconds.

5) Finally, we send out an END TRANSACTION step to all AMPs involved

in processing the request.
-> The contents of Spool 1 are sent back to the user as the result of statement 1.
A2 In most complex queries, you will see a last step, which appears to simply re-spool the result
: spool file. It is not really re-spooling it; rather, it is reformatting the data to prepare it for
return to the requesting application. This is required because all of the internal processing
steps (joins, scans, aggregates, etc.) work on data in the Teradata internal formats, while your
application wants the data in the client data-type formats (appropriate character sets,
formatting applied, right floating point representation, big/little endian, and so on).

For aggregate results, the grouping key also needs to be stripped out. In this particular plan it
is a little more confusing since the result comes from a UNION operation. Step 3.1 formats the
result of the first SELECT (Spool 2) into client format.

Step 4 does the same for the result of the second SELECT (spool 4) after applying the
HAVING clause and the ORDER BY. Note that step 3.1 and step 4 both write their results
into Spool 1, which is the final result spool all formatted and ready to be returned to the client
application.

Q3 DISTINCT
: Please explain how the DISTINCT command works in Teradata SQL queries.
A3 Including the DISTINCT clause in a SQL query results in elimination of duplicate rows.
: DISTINCT applies to the entire row and not to any single column.

Ex. SELECT DISTINCT a, b, c FROM x;

Here DISTINCT does not mean just "a" being unique, but the combination of a, b & c

You can produce distinct result sets in two ways:

SELECT DISTINCT x FROM TABLE;

-or-
SELECT x FROM TABLE GROUP BY x;

Teradata uses two different methods to produce the result sets, however, and the performance
varies dramatically depending on the data.

SELECT DISTINCT is designed for data sets that are nearly unique to begin with and works
by sorting the entire intermediate spool file and discarding duplicate values. SELECT/GROUP
BY is designed for data sets that have relatively few unique values and works by performing
 an AMP local grouping operation and then merging the partial result sets for final processing.

Q4 Teradata Priority Scheduler

: We are looking to use the Teradata Priority Scheduler to help us out with resource
management.
A4 The Teradata Best Practices Document recommends moving all non-system session and work
: to a separate resource partition. This is phase 1 of our plan. I have a couple of related
questions:

1) If I just move all non-system sessions to RP1 and go with all defaults (RP1 = none, default,
M1, AG 41) are there recommended weightings? Our plan was 100 (RP0), 75 (RP1).

2) With such a high weighting on each (RP0 and RP1), can you migrate clients and batch
sessions in phases based on the fact that only "active" sessions are considered and that each of
the RPs has a high weighting? Or should it be a "big bang"-type of move?

3) Because we are planning to create the remaining 4 RPs and only use the first two (initially),
I created my "user partition" (RP1) (development) performance groups, half expecting that the
system would create its own. I created 8(L1), 10(M1), 12(H1) and 14(R1), and when I
displayed it after the fact, I was expecting to see 11(low1), 13(med1), 15(High1) and
17(Rush1). I did not see this.- Is it safe to assume that I have to create the system performance
groups myself or are there system performance groups required beyond the default RP0
partition?A4: Regarding the RP weightings, keep in mind that the entire Teradata Priority
Scheduler algorithm is relatively based. That means the RP weights are "high" only relative to
each other. The relative difference between the weightings is what's important. Therefore, in
the instance where you have two partitions, the effect would be the same whether you had 100
and 75 or 1000 and 750. There would no difference in the way the system behaves.

As far as moving things out of the default partition (RP0), that's often a good thing to do but it
is not always necessary. Typically, you should keep RP0 for system work and, optionally, for
very predictable, high-priority work such as an application that does OLTP processing or
canned reporting. With RP0 containing only system work, you could comfortably make the
RP0: RP1 weighting ratio 2:1 (i.e. 100 vs. 50) or even higher. Just remember that the more
extreme the weighting differential, the more important it is to ensure that ONLY system work
runs in default. Otherwise, a very poorly written, highly weighted query that inadvertently gets
in RP0 could have a dramatic impact on system performance. Also, if you haven't switched the
DBS Control setting to have rollback occur at the same priority as the executing transaction,
this could also impact performance in a way you might not have intended.

Migrating in phases is certainly doable. Just appreciate the fact that, all things being equal, as
you migrate work from RP0 to RP1 the performance characteristics of the work that gets to
RP1 first will probably degrade. (That's somewhat of a guess, since I don't know specifically
what your workloads look like.)

An alternative approach might be to set the weight of both partitions to be the same (i.e. RP0
100, RP1 100), migrate out the workloads to RP1, and then change the RP weights
accordingly. As the work migrates, the system should behave as if everything was in the same
partition because, essentially, everything will be prioritized the same. And then you could
more heavily weight the system work if that's your desired end state.

In response to your last question, you can only run the Teradata Priority Scheduler, schmon,
 from a node that is running Teradata. Schmon requires services from the PDE layer, which is
not available on the application node. Although you can run schmon from any Teradata node,
you will require root privileges to issue commands.

You may have already seen this information, but there's a small repository on SupportLink that
contains technical papers (usability tips and techniques) around things like priority scheduler.
If not and you're interested, here's how to get to it.

If you are on page 1 of SupportLink, scroll down and there should be a link under the Updates
column that takes you to the active data warehouse repository. The icon you are looking for
says "Teradata ADW Documents." Click on that icon, and you will see a list of papers that you
can download. There are several Teradata Priority Scheduler how-to papers with things like
sample code, templates, discussion of trade-offs, dos and don'ts, etc. They might make your
next steps easier.

3Q 2003 FAQs
Q1 What is NCR's 64-bit MPP Server Update?
:
A1 As you know, Teradata's roadmap is dependent on Intel's roadmap for server-class nodes. We
: planned to announce 64-bit MPP servers as early as the end of this year. However, the current
64-bit server nodes from Intel haven't achieved performance parity with the 32-bit nodes.
Based on market predictions from industry analysts, including Gartner and IDC, we expect the
performance difference to continue through 2006. The Teradata Database software is ready
and is currently available on non-NCR Intel 64-bit SMP systems. Teradata Database is
positioned to deliver on 64-bit MPP servers when they reach comparable performance levels
to the 32-bit Intel servers. When that happens, Teradata will provide 32-bit customers with
investment protection options for their current systems.

Q2 What is difference between perm space and spool space?

:
A2 The difference between perm space and spool space is that perm space is allocated to the
: database and will not share with other databases, while spool space is a reserved space that is
shared by different users and, within the users' queries, is used to format the result sets.

Q3 I am using the Teradata JDBC type 4 driver. What port number does the driver use to
: talk to the database?
A3 With the type 4 driver it is not necessary to specify a port in the connection string since a
: gateway is not used. The type 4 driver communicates directly with Teradata, which uses port
1025.

Example: "server" is the name of the system where Teradata is running

String url4 = "jdbc:teradata://server";
con=DriverManager.getConnection (url4,"uid","password");

Q4 We have a DDL for our database layout. Currently, we have a column definition as
: follows:

activity_dt DATE NOT NULL FORMAT 'mm/dd/yyyy' DEFAULT -18989899,

The programmer ensured us that the default date would be 01/01/0001. But it is not; the
UNIX timestamp above gives a date in 1969.

What is the syntax to define 01/01/0001 as the default date?

A4 Your programmer is right. The date field in Teradata is not a UNIX timestamp, but a different
: scheme altogether. The default shown above is correct, but for the best solution, use an ANSI
date literal, as shown below.

activity_dt DATE NOT NULL FORMAT 'mm/dd/yyyy' DEFAULT DATE '0001-01-01'

Q5 We are just starting to work on Teradata and we have a question about building views.
: Do you know of a tool that could help us build dimensional views over third normal form
models? We want these to be linked to the physical but still be independent, to inherit
definitions but be modifiable, and to be documented (i.e. catalogued).
A5 If I read this correctly, you're saying that you have a normalized model and want to generate
: dimensional views to sit on top of it. You can do that by following the guidelines on how to
develop views in the documentation.

Q6 Currently, there is no way to cancel a rollback. The rollback will survive a TPARESET.
: However, the GSC can stop it with a utility (I believe a chargeable event). They will
perform a TPARESET to cap the new entries to the TJ, then stop the journaling by using
the utility. You will lose all data in the transient journal, not just your particular
rollback. All tables involved in the transient journaling will be in an invalid state and will
either need to be dropped/recreated or restored. Is there a fix for this?
A6 There will be a new command in V2R5.1 Recovery Manager that will take care of this. It is:
: CANCEL ROLLBACK ON TABLE.

Q7 Suppose that incoming data targeted for an enterprise data warehouse is in various
: "character sets," i.e. kanji, Latin, etc. We would like to store the data in a consistent
character set, but when the info is presented back to the client, we want to place it in the
appropriate character set for that client.

Is there a "super type" character set that would allow for conversion to/from any
character set? UNICODE seems close but still has exceptions. Is there a method to
dynamically '.set session charset' if we were to present data from a method other than
BTEQ?
A7 UNICODE is the "super type" server character set that Teradata offers. If it doesn't meet your
: specific need, then you would need to submit a request for change. We do offer some
capability to customize conversions from client character sets to internal character sets, and
this may address the exceptions that you mention. However, without specifics on the
exceptions, it is unclear if there are solutions to this issue using current capabilities.

KANJISJIS might be another possibility to use as the server character set, but I suspect that
would be less suitable than UNICODE. You can define the character column to be UNICODE
or LATIN (you can also choose GRAPHIC, KANJI1 and KANJISJIS, but we encourage users
 to choose either UNICODE or LATIN).

e.g., create table test (col1 char(5) character set latin,

col2 char(5) character set unicode);

Incoming data will be translated internally to the form specified in the column definition.
Outgoing data will be translated to the character-encoding scheme specified by the user
(session character set). Users can change their character set without logging off, i.e. the
session character set can be changed during a session.

I think most Teradata interfaces, including BTEQ and CLIv2, support a way to set the
session's client character set. However, this must be done before logon (of course, one could
log off, change setting and log back on when a change is necessary). I am not an ODBC/JDBC
expert, but I assume they would have this capability also. The client character set cannot be
changed during a session. The default server character set can be changed using the MODIFY
USER statement; it can also be overridden for a column by explicitly specifying the
CHARACTER SET after the character type.
1Q 2004 FAQs
Q1 How do you calculate the average of timestamp values in Teradata? For example,
:
Create table avgts(i int, ts timestamp(0));
ins avgts(1, '2003-09-01 12:00:00');
ins avgts(1, '2003-09-03 12:00:00');
select i, avg(ts) from avgts group by 1;
gives me the error:
5407: Invalid operation on ANSI Datetime or Interval value
A1 You can do conversions like this:
:
sel runname,qryname,sum(timeinsecs)/count(=) as avgtime, avgtime/3600 as timehours
,(avgtime-(3600=timehours))/60 as timeminutes
,avgtime-((3600=timehours)+60=timeminutes) as timeseconds
,trim(timehours)||':'||trim(timeminutes)||':'||trim(timeseconds) as averagetime
from results
group by 1,2
order by 1,2

Or for a much simpler expression for what you are doing, check into ANSIINTERVAL!
select avg((cast(timestampfield as time) - time '00:00:00.000000') hour to second)
from table

Q2 Will the optimizer materialize a multi-table view before any additional table joins or
: constraints are made? Here is an example:

I have an aggregated view that is a join of tables A, B, C and D. When users query this
view, they sometimes want to join the view to table D to further constrain on column Z.

Will the optimizer join tables A, B, C and D first without the extra table D constraint on
column Z? Or will the optimizer recognize the additional column Z constraint before
materializing the view?
 Would the plan be different on Teradata Database V2R5 compared to Teradata
Database V2R4?
A2 The case is a bit abstract here, but it makes me wonder if the intent of the query will actually
: be realized.

You state that the view specifies an aggregation over the result of a join between tables A, B,
C and D, but you do not state whether Z is in the group list and made visible via the view. I am
guessing that it isn't; if it were, then it would be easy to apply the condition to Z without an
additional join to D.

If Z is not visible in the view, then it is highly likely that the additional join to D constrained
on Z will not create the desired result. The aggregation will be performed first as specified,
then the additional join to the constrained version of D will be performed—it has to be
executed this way to get the correct SQL result for the query. This is likely to be a very
different result than if the constraint was applied to Z on D prior to the aggregation because
rows will be eliminated prior to consideration for grouping.

Q3 With the arrival of Teradata Database V2R5 comes the long-awaited multi-value
: compression facility. Can you discuss its features and its pros and cons?
A3 The use of multi-value compression within Teradata Database V2R5 can provide some big
: savings, both in disk space and in the general performance of queries.

As an example, one customer had a table in Teradata Database V2R4.1, which used single-
value compression, and they saved 30% in disk space. Since an upgrade to Teradata Database
V2R5, the customer has managed to save an additional 25% in disk space and has noticed a
general performance improvement of around 40%, which the customer thinks had a lot to do
with the fragmentation of the table as well. The use of compression and the additional CPU
overhead was negligible, estimated around 0.5%.

To date, customers who have implemented multi-value compression under Teradata Database
V2R5 are seeing favorable results all around. However, be aware that there is a tradeoff: Some
middle-sized tables actually grew in size with multi-value compression due to overheads of
table headers and where users had applied to columns that had previously been declared as
VARCHARs.

All in all, however, users have noticed a "general performance improvement."

Q4 We would like to be able to store the results of any and all queries (the actual answer
: sets) for subsequent auditing. Our fear is that just having the query is not enough since
rerunning the query might not produce the same answer set the user originally got.

It seems to me it would have to be part of the database query log, with an option
provided to store, say, up to 10K of the final spool before it's sent to the user. But that
option is currently not available.

Any thoughts as to how we could store the results of any and all queries (or some portion
thereof) for some period of time?
A4 Run every query as a CREATE TABLE AS WITH DATA with ROWNUMBER as the first
: column in the select. Then select from that table to return the result to the user. This process
 requires a driver program to grab the SQL and rewrite it in order to do the create with an
appropriate table name that can be tracked later (date, user, etc.). The driver should probably
log on with a driver ID so the user can't delete/drop the table.

It would not be practical to put in a database query log since every result has a different table
layout. WARNING: The log will be vastly larger than the data, so be sure to have enough
space available to support this query.

Q5 In using the Teradata Priority Scheduler Facility, the milestone type parameter is set at
: the performance group, not at the allocation group. Therefore, assuming no manual
intervention through the PM/API, the query will be assigned to the allocation group
associated with the performance group at logon and then be handled according to the
milestone type rules defined.

One possible exception to this case might be a situation where you implement a process
that switches your PSF schemes, either based on some event or at regular time intervals.
If, in this scenario, you were to "redefine" your performance group such that you change
the milestone type from "time" to "query," I'm not sure what would happen to any "in-
flight" queries. I'm not recommending that, but I'm guessing that it's possible.

Does PSF continually reevaluate the definition of performance groups and/or allocation
groups when computing the relative weighting?
A5 You can change a performance group definition at any time, with active work running under
: its control. This will cause a change to the Teradata Priority Scheduler definition, which will
cause the system.GDO record to be rewritten. Any time the system.GDO record is changed, all
CPU accumulations (whether at the session or the query level) across all performance groups
are zeroed out. The system.GDO record exists on each node and is the operating system's copy
of the current priority settings.

This means that if you issue any schmon command, it will change a Teradata Priority
Scheduler component. Say an allocation group weight in another resource partition, or changes
a system level limit, that change will cause all queries under the control of any milestone limit
in any performance group to start over in their accumulations of CPU.

Most definitely, relative weights will be recalculated when the system.GDO record changes, as
is continuously happening under normal circumstances.

2Q 2004 FAQs
Q1 I am loading data into Teradata via TPump. There seems to be an issue
: inserting a row into one table from two sources at the same time. When I
look at the performance monitor, I can see that the data is being blocked.
Logically, I find it difficult to believe that multiple sources could be loading
the same table simultaneously without locking issues. I guess it could work,
depending on blocks, cylinders, etc., but it sounds like at some point
(statistically) you'd run into a blocking scenario. Are there any best
practices around using TPump and multiple sources loading the same
table?
A1 There are two parameters to control locking in TPump: PACK and SESSIONS.
: With a large PACK value, TPump will run faster, but blocking will happen more
frequently. SESSIONS just indicates the number of users concurrently inserting
data via TPump.

Start by lowering PACK and SESSIONS on the TPump jobs you are running.
TPump has a SERIALIZE function that avoids cross-session collisions by
hashing identified key values to particular sessions; this also preserves order.
Ultimately, you can run with PACK=1, SESSIONS=1, SERIALIZE ON. This
way you may end up with no blocking, but TPump may run slow.

Q2 I need help regarding Teradata's file writing utilities. In my application, I

: have a procedure that retrieves data from more than one table and stores
data in variables after manipulation. I need to write the data to a file, which
in turn can be loaded into a target table using MultiLoad. (This is because
direct writing to the table takes a lot of time for large data volumes.) Which
Teradata file writing utility can be used in this case?
A2 In Teradata we always try to get things done using set logic. You do not state
: what kind of data manipulation you need to do. If there is any way to do it in
SQL statements, then the best solution for this type of application is to use
INSERT SELECT without the data ever having to leave the database.

If it is absolutely necessary to retrieve and process a large amount of data outside

Teradata, then it sounds like a job for the FastExport utility. FastExport accepts
any SQL statement and retrieves the result using an interface optimized for high-
volume export. It also has an interface for OUTMOD or Access Module that
allows you to process the data on the way out of FastExport and lets you to write
it anywhere you desire.

Q3 Test (1) Environment:

: database: dev1
Tables: tab1, tab2

Test (2) Environment:

Database: dev2
Tables: tab1, tab2

Both databases run on the same Teradata system and have identical table
structures, names, access rights, etc.

I have a query that looks like this:

select col1, col2, col3...

from dev1.tab1, tab2

where inner-join conditions ...

When I try to run this query, one of several things happens:

1. When the default database is 'dev2,' I get a Table/View/Macro name

'tab2' ambiguous error.
2. When the default database is 'dev1,' the query runs fine.
3. When the default database is either 'dev2' or 'dev1' without the
 'dev1' qualifier on 'tab1,' the query runs fine.
4. When the default database is either 'dev1' or 'dev2' with both the
tables properly qualified as dev1.tab1 and dev2.tab2...5.

Is this a bug or am I missing a link?

A3 Think of the default database as one entry in a search list of databases, not as a
: text substitution for a missing database specifier. All other databases named
explicitly in the request are also added to the search list. Then each object is
looked up in the databases in the search list. An object that appears in multiple
databases in the search list is ambiguous.

Q4 I've looked as some test results showing that single-AMP (PK access)
: queries are not impacted by VLC to any notable degree. If compression is
defined but not used (because the particular rows' columns were not
compressible), then there was less than 1% overhead detected (-0.14%
throughput change) when performing five sessions doing PK selects. This is
equivalent to nothing, and it is a heavy load.

Another test with 100% compression (all the columns had to be

uncompressed) showed single-AMP selects under the same conditions also
having less than 1% overhead (-0.53% throughput change). That is also too
small a factor to care about.

With short all-AMP tactical queries you may be able to benefit from VLC
to some degree, or at least not experience degradation. In addition, because
VLC has the potential of reducing overall I/O demand on the platform, it
can provide a secondhand benefit to short tactical queries because it can
reduce contention for I/O. Tactical queries always do better in
environments that are not maxed out on a particular resource. VLC can be
helpful in restoring a healthier balance between CPU and I/O demand, if
the platform tends to be I/O-bound.

What do you think?

A4 Actually, Teradata VLC compression has shown in testing as well as customer
: sites to offer a benefit in the load utilities. There are two reasons for this:

1.) When rows are compressed prior to loading, more of the rows can be placed
into a single data block. Therefore, the load utility is doing less physical I/O to
store more rows.
2.) With PPI, if you have partitioned on day, for example, and are loading rows
that all go into the same partition, often those data blocks are so few and so
concentrated that they stay in the FSG cache. While this benefit will depend on
many factors (your mileage will vary), and you might not see this at all, let me
give you an illustration. During one test we loaded a PPI table via TPump where
the rows targeted one partition, and we experienced almost zero I/O load. That
significantly reduces time to load, as you can imagine. ARCMAIN will benefit
as well, as there will be fewer blocks to back up.

In terms of the test results, I've seen 20-25% improvement using FastLoad. But
what you will experience will depend very much on how many columns and
rows can be compressed, and how large they were to start with. However, you
should always see some improvement.
3Q 2004 FAQs
Q1 I'm trying to subtract one year or 12 months from the leap year 2004-02-29 and I get the
: following error message: <2665 Invalid date>.

SELECT CAST('2004-02-29' AS DATE) - INTERVAL '12' MONTH

SELECT CAST('2004-02-29' AS DATE) - INTERVAL '1' YEAR
A1 ANSI standard says MONTH/YEAR interval arithmetic should not adjust day of month.
:
Teradata implemented the ADD_MONTHS() extension specifically to address this problem:

SELECT ADD_MONTHS(date '2004-02-29', -12)

Results: 2003-02-28
Q2 I am planning on using the schmon -i or the schmon -b command in Priority Scheduler.
: Do you know if I need to adjust the CPU percentages on a regular basis? If so, how often
should this be done?
A2 Here are some explanations regarding just what these Priority Scheduler commands do:
:
The schmon -i (lowercase i) enables the prioritized queuing of FSG I/O requests. It should be
turned on by default. It is not recommended that you turn it off, but it may not make much
difference to do so if you are not experiencing I/O waits of any magnitude on your platform.
What it does is enforce the prioritization of requests that are waiting for an I/O, which is
usually a good thing. I am not aware of any negative trade-offs with having I/O Prioritization
turned on.

The schmon -b command creates or modifies resource partitions. It is not uncommon to issue
several schmon -b commands to alter the relative priorities of one or more already-defined
resource partitions. Often a small batch of schmon -b commands is packaged into a cron job
and scheduled regularly as a way to change priorities at different times of day. The most
common change pattern I have seen is twice a day, once to change settings just prior to the
batch cycle at night and then just before the day cycle in the morning. The resulting impact on
CPU usage will depend on how much the assigned weights of the affected resource partitions
are changed and the nature of the work running within them and the other active resource
partitions.

If you are using PSA (Priority Scheduler Administrator), PSA can automate the changes in
settings at specific times of day, so the DBA no longer needs to submit cron jobs or schmon
commands to establish that level of control.
Q3 When does an AMP give up the allocated worker tasks (AWTs)? Will it wait until the
: complete step has been finished on all AMPs before it releases all AWTs, or will it release
the AWTs AMP by AMP? What if the step is highly skewed?
A3 Teradata will release a given AWT at the time that particular AMP completes that query step.
: Each AMP will make a notification that it has completed that query step and then free up its
AWTs for other work.

Only when the last AMP is done with that step does the system notify the dispatcher that all
AMPs are done so the next step can be sent. If the step is highly skewed, some AMPs will be
holding AWTs on behalf of that step, while other AMPs will have completed and released
their AWTs to do other work.
Q4 I am executing a query with a value in the where clause for a NUPI (Non-Unique
: Primary Index) on a table with a PPI (Partitioned Primary Index). The query is not
using the NUPI.

My question is this: When you have a PPI and a NUPI (and the PPI is not part of the
NUPI) and you supply values for the NUPI but not for the PPI, should the optimizer use
the NUPI or go around it? Also, is it necessary or just recommended to incorporate the
PPI column into the NUPI?
A4 The answer depends on the size of the table and number of partitions.
:
When only the NUPI value is specified on a PPI table, every partition has to be accessed in
order to find the NUPI value. If there are lots of partitions and it would be faster to scan the
table, then that is what the optimizer will choose. This is the trade-off of partitioning a table
that also needs to be accessed directly by a single PI value. If the PPI is valuable enough, then
it is probably appropriate to put a NUSI on the direct selection columns to avoid all the probes
to the partitions.

Should you put the partitioning column in the PI (Primary Index)? Usually not. In this case, if
it is possible to specify the partitioning column value and the NUPI value in the where clause,
then do that so that you have good PI-type access. If the partition value is not available, then
having the partitioning column in the PI makes the PI useless even more of the time.
Q5 We need to use big comments columns because we include management rules in the
: comments. However, the comment columns in Teradata are only 250 characters long. Is
there anyway to capture the complete comment?
A5 Teradata offers Metadata Services (MDS), which provides the facility for both capturing the
: business metadata and for disseminating the business and technical metadata to users. It has
the flexibility to contain the large amount of text that is desirable for the business metadata
descriptions. It also helps with relationships between objects.
1Q 2005 FAQs
Q1 During spool redistributions to different AMPs (merge joins where primary indices are
: different), some records come to the same AMP after rehashing on the basis of join
columns. Do they really go to the BYNET or message-passing layer for rehashing, or are
these records directly spooled locally?
A1 When rows are redistributed during join processing, there is a same-node optimization that is
: done by the message system. The redistribution message, when it is sent, uses VPROC and
node addressing rather than hash addressing (it could be a different node, different AMP, same
node, different AMP, or same node, same AMP). The optimization consists of recognizing
when the destination is on the same node as the sender and routing that message to the correct
AMP, without leaving the node. Only rows being redistributed off the node from which they
were sent will be handed off to the BYNET. A row being sent out and returned to the same
AMP leaves the AMP within a sender task and returns to the same AMP within a receiver
task.
Q2 Does Teradata support API/CLI for pre-defined, vendor supplied, third-party supplied
: and user defined complex data types on the database? To clarify, I am looking to
understand if things such as composite field types, create sub-tables, arrays of like types
or free form arrays are supported.
A2 The upcoming Teradata Warehouse 8.1 (Teradata Database V2R6.1) will support user defined
: types (UDTs), including complex data/structured types. It does not provide direct array
support and array operations yet. Teradata Database V2R5.1 included BLOBs and CLOBs for
storing large unstructured data and user defined functions (UDFs) for processing that data.
 While the typing is not explicit, any array or structured type can be stored using these types
and can be processed in the UDFs. The data structure must be represented in C programming
language within the UDFs rather than the database knowing the type structure. This ability
continues and is extended by UDT implementation in Teradata Database V2R6.1.
Q3 When insert selecting, I assume the system is doing something to the data on the way,
: such as creating a spool file. Further, because maybe only one field is being changed, I
assume that the spool file is uncompressed then recompressed on the other end. I am
insert selecting a 600GB table to a new data structure with new columns, a couple of
transformations and both partitioned primary index (PPI) and multi-value compression.
Am I right to assume that I will need 2 x 600GB plus the size of the uncompressed data in
spool (40%-50% of 600GB)? If so, I will need 1.5TB of space and the spool file will need
complete cylinders. The system currently is 67% full CurrentPerm/MaxPerm.
A3 You are correct that the spool is not compressed. How much space it requires depends on the
: plan. Does the plan need to spool it only once or does it have to do joins, for example, to get
the additional columns and transforms? All of this should be taken into account. You will need
to target the necessary source space as well as the space required by the uncompressed copy in
spool.
Q4 When performing INSERT SELECT within target tables and concurrently generating
: reports with access locking, we have found that large amounts of data disappear from
the target tables. We know that other factors may influence this observation, such as a
faulty description. We are running Teradata Database V2R4.1 and moving to Teradata
Database V2R5.1.1 and want to know if this has been resolved in the new release.
A4 This should not be an issue with INSERT SELECT. Based on your description, there is only
: one case where this is an issue. That is when UPDATE WHERE is being used with a join.
This can result in an access lock reader seeing a lot of missing rows. You can always review
the explain plan to see if it shows a delete step followed by a merge insert. If the select sees
the table during the time that the delete has operated, then those rows will be missing until the
merge step completes. In Teradata Database V2R5, merge update, in most cases, has been
changed to perform a direct merge into the table; again this can be seen in the explain plan.
Prior to Teradata Database V2R5, any Update that uses the delete/merge plan should be
executed with a LOCKING EXCLUSIVE clause if access lock selects might possibly be
performed concurrently.

Here are some additional questions answered by Carrie Ballinger, a Teradata Certified Master
and a senior technical consultant in Teradata's Active Data Warehouse Center of Expertise. If
you would like to get some additional tips and insight from Carrie, be sure to check out her
column titled "Learning from each other."
Q5
Can partitioned primary indexes (PPIs) replace value-ordered indexes (VOIs)?
:
A5 In many cases, yes. However, it depends on the application. A VOI is a special type of index
: whose sub-table rows are stored in order by index value. With VOIs you have flexibility over
the range of access. The same VOI structure will provide efficient access for ranges of two
days, two weeks or two months. With PPIs, your partition boundaries are fixed. If you
partition by month, then partition elimination cannot be less than one month, even if you only
require data from one day. If your access range size varies, consider finer partitions in your
PPI table—a day or a week instead. Often, this is preferable to supporting a VOI on the same
column as the partitioning column.
Q6 Are AMP worker tasks (AWTs) held by queries when they are returning answer sets?
: During a quiet time on my system, I noticed 10 sessions, all in RESPONSE mode;
however, when I issued a puma -c command, there were no AWTs in use. What is
happening here?
A6 AWTs working on behalf of a query are released at the end of the final query step before the
: BYNET merge process begins. The last AMP to complete its work in building a query's final
spool collects row counts from all AMPs, responds to the dispatcher that the step is complete
and gives the final row count. The dispatcher then tells the BYNET to start a merge process.

The BYNET driver has special data buffers where rows that will feed into the merge are to be
placed. The BYNET driver also has a special place in memory to track details about a query's
response processing, such as its merge-ID; the spool-ID where the rows to be returned are
located; and row-ID, which tells how far that spool has been read. AWTs are acquired as
needed and only for the brief period of time it takes to move data from the final spool file into
the BYNET buffers.

After all rows in the first BYNET buffer have been returned to the client, and if there is more
data in the spool, a new AWT will get involved briefly to move data from the spool into the
BYNET buffer. Once the merge is complete, the BYNET driver tells the dispatcher it is
finished, completing the response cycle.

So, except for very brief periods of time when AMP activity is required to fill the BYBNET
buffers, there are no AWTs held during response processing.