459A

Calc Scripts for Mere Mortals

Presented by: Edward Roske, interRel Consulting

Agenda

Review the Basics Calc Script Commands Using Functions Controlling Execution Creating Blocks Organizing Calc Scripts Real World Calculations Optimization

Types Of Calculations
Outline consolidations Member formulas

Unary operators + - * / % Relies on hierarchy Allows complex calculations Can use functions and conditional processing Override outline operations Offer most control

Calculation scripts

Calculation Order

Two basic categories

Outline order
Dense Sparse

Dimension order

Calculation Order All Dimensions

First, Accounts Second, Time Third, all remaining dense dimensions Forth, all remaining sparse dimensions Last, Two-pass calculations

Calc Scripts

Reasons to use calc scripts:

Override outline operators Specific calculation commands available Iterative calculations Variables Optimization

Calc Script Editor

Choice of the Essbase XTD Analytic Services

Calc Script Editor or another text editor If you use another editor:

Save file with .csc extension Easier to perform some functions

Find/Replace See more lines of the script Printing options

Can mix and match

Essbase XTD Analytic Services Calc Script Editor

Has a number of functions built in Command buttons simplify entries Validation function to detect errors Limited work area

Calc Script Editor Functions

Most simply enter the character that are on

the face of the button The is a notable exception

Provides a list of available functions Grouped by function type Can include parameters

Member Selection

Associate the outline Can select members from the member list Frequently easier/faster to just type the
member names

Automatically adds quotes if needed

Online Help

Use menu option

Functions and commands are included Can also use Essbase XTD Analytic Services
documentation

F1 does not work

Technical Reference

Calculation Commands

Limited number of true calculation

commands

Not functions they are in another chapter Not Set statements they are covered under
optimization

Agg

Syntax

Summary

AGG(Dimname, Dimname,); Aggregates dimension(s) according to the outline

operators Member formulas are ignored Very fast

Fewer than 6 levels

Array

Syntax

Summary

Array ArrayName[DimName] = {initial values }; Declares an array variable with as many

Optional
members as the dimension name that follows it Can be initialized with values if desired

Calc All

Syntax

Summary

Calc All; Calculates all dimensions in the database Uses default calculation order This is typically the default calc Very efficient Can add Except to remove
dimensions/members

Calc First

Syntax

Summary

Calc First; Calculates members tagged as TB First Does not update Time dimension, only Accounts

Calc Last

Syntax

Summary

Calc Last; Calculates members tagged as TB Last Does not update Time dimension, only Accounts

Calc Average

Syntax

Summary

Calc Average; Calculates members tagged as TB Average Does not update Time dimension, only Accounts

Calc Dim
Syntax

Summary

Calc Dim(DimName, DimName,) Calculates the dimensions that are listed All dense dimensions are calculated first in the order
they appear Sparse dimensions are calculated last - also in the order they appear If you need a different order, use multiple Calc Dim commands Includes member formulas

Calc TwoPass

Syntax

Summary

Calc TwoPass; Calculates members tagged as Two-Pass in the

Accounts dimension

CCONV

Syntax

Summary

CCONV type->Currency name Special command reserved for currency

conversion Converts values to the currency name specified Only usable if you have the currency conversion module

ClearBlock

Syntax

ClearBlock ALL | UPPER | NONINPUT |

DYNAMIC

Summary

Designed to clear blocks by replacing values with

#Missing Within a FIX block, may only clear a portion of a block Options for which types of block to clear

ClearData

Syntax

Summary

ClearData MemberName; Designed to set a member value to #Missing Works within a fix

ClearBlock vs. ClearData

ClearBlock is designed to clear an entire

block

No Member name, just block type ClearData is designed to clear a member In many cases either one will work

At least a large portion of the block Usually before it is recalculated

DataCopy

Syntax

Summary

DataCopy MemberName To MemberName; Copies data from one member to another All intersections are included Great for copying versions Will create blocks if they do not exist

If/Else/ElseIf/EndIf

Syntax

Summary

IF (condition) ElseIf/Else EndIf; Condition must be if parenthesis In a calc script, associate with a member name Each If must end with EndIf Most effective on dense members
Does not really matter which member name

Fix/EndFix

Synatx

Summary

Fix(MemberNames, Functions Etc.) ENDFIX Used to focus or limit a calculation Most effective on sparse members Used extensively

Loop/EndLoop

Syntax

Summary

Loop (integer, break) EndLoop; Causes a block of code to be repeatedly

executed The integer determines the number of times the loop runs The break value allows early exit Although it is a loop, its not a true programming loop

Using Commands

Need to combine commands with functions Execute in the order they are written None by itself is that useful Used together, they provide a powerful tool set to apply business rules to the database

Not Commands But

A member name by itself

Example
East;

Calculates the member formula if present Calculates outline rollup if there is no formula Multiple members can be listed Member set functions can also be used
Example
@IDESCENDANTS(2003)

Assignment Statements

Assign a value to a member Example There are other math operators Every command ends with a semicolon (;)

Budget = 0; Budget = Budget * 0.95; More later

Okay, there are exceptions but most do

More on Member Names

All the members are calculated at once if

possible To force multiple passes of the database, group the member names in parenthesis

One pass for each group in parenthesis Not usually what you want, so use with caution

Function Categories

Conditional Boolean Relationship Calculation Operators Mathematical Member Set Range and Financial

Allocation Forecasting Statistical Date and Time Miscellaneous Custom Defined

Calculation Operators

Nothing too exciting +-*/% Use parenthesis to control order of

operations

Conditional

Typically used within an If command >, >=, <, <=, <>, != Can combine with And, Or and Not

Boolean

More traditional functions

@ISACCTYPE @ISANCEST @ISCHILD @ISDESC @ISGEN @ISIANCEST @ISICHILD @ISIDESC @ISIPARENT
@ISISIBLING @ISLEV @ISMBR @ISPARENT @ISSAMEGEN @ISSAMELEV @ISSIBLING @ISUDA

Typically used in If Statement Returns True/False

Relationship Functions

Returns Values Frequently used to get other values in

database @ANCESTVAL @LEV @ATTRIBUTEBVAL @MDANCESTVAL @ATTRIBUTESVAL @MDPARENTVAL @PARENTVAL @ATTRIBUTEVAL @SANCESTVAL @CURGEN @SPARENTVAL @CURLEV @XREF @GEN

Mathematical Functions

Most functions available @ABS @MIN

@AVG @EXP @FACTORIAL @INT @LN @LOG @LOG10 @MAX @MAXS

@MINS @MOD @POWER @REMAINDER @ROUND @SUM @TRUNCATE @VAR @VARPER

Member Set Functions

Returns a List of Member Names

@ALLANCESTORS @ANCEST @ANCESTORS @ATTRIBUTE @CHILDREN @CURRMBR @DESCENDANTS @GENMBRS @IALLANCESTORS @IANCESTORS @ICHILDREN @IDESCENDANTS @MERGE @ILSIBLINGS @PARENT @IRSIBLINGS @RANGE @IRDESCENDANTS@RDESCENDANTS @ISIBLINGS @RELATIVE @LEVMBRS @REMOVE @LIST @RSIBLINGS @LSIBLINGS @SIBLINGS @MATCH @UDA @MEMBER @WITHATTR

Range and Financial Functions

Accepts a range of members instead of

individual members
@ACCUM @AVGRANGE @COMPOUND @COMPOUNDGROWTH @CURRMBRRANGE @DECLINE @DISCOUNT @GROWTH @INTEREST @IRR @MAXRANGE @MAXSRANGE @MDSHIFT @MINRANGE @MINSRANGE @NEXT @NEXTS @NPV @PTD @PRIOR @PRIORS @SHIFT @SLN @SUMRANGE @SYD

Allocations Functions

Only two functions in this category Used in simple situations to allocate an

amount down to lower levels

Allocate MDAllocate

Forecasting Functions

Similar to what you may have used in Excel @MOVAVG

@MOVMAX @MOVMED @MOVMIN @SPLINE @TREND

Statistical Functions

Most common are available @CORRELATION

@COUNT @MEDIAN @MODE @RANK

@STDEV

@STDEVP @STDEVRANGE @VARIANCE @VARIANCEP

Date and Time

Just one to convert strings to a date

ToDate

Miscellaneous

The catch-all category

@CALCMODE @CONCATENATE
@SUBSTRING @NAME

Using Functions

Frequently, functions are nested or used as

parameters for another function Functions are prefixed with @ Parameters are separated by commas Used in many places

Right of = FIX statements IFs etc.

Focusing Calculations

Two most common methods Other methods:

Fix If

CrossDim operator MemberSet functions Relationship functions Variables

FIX

FIX limits the members that are processed Most effective when used on members of
sparse dimensions Only blocks that meet the Fix criteria are processed

Typical uses

Remaining blocks are simply skipped Only calculate one scenario Limit calc to current year

More on FIX

FIX works properly on dense members May result in all blocks being processed All statements between the FIX and ENDFIX
are executed as a block FIX statements an be nested Multiple members from the same dimension are treated as or

In other words, all members will be processed

What You Cannot Do With FIX

You cannot assign values to a member that

is included in the FIX statement

Actually, values cannot be assigned to any

Fix(Sales)
Sales = 100;

member from a dimension in the FIX statement Example

ENDFIX

Would result in an error

FIX On

Members from one or more dimensions Can list specific member names Can include member set functions Can include AND and OR
functions Caution on empty sets

Use AND to get the intersection of two or more

IF Statements

Conditional processing is often needed to

process business rules Can include complex conditions

And, Or, Not, etc.

What You Can Check For

Most common are

Members Values

Specific members MemberSet comparisons Use boolean functions >, < = etc. Use #Missing for test

True is treated as 1 and false is 0

Syntax for IF

Place condition in parenthesis Associate with a member

Doesnt matter which member MemberName (

If (condition) EndIf )
Statements

Not required for member formulas

More on IF

Can have ElseIf (condition) Can Have Else Can have both Still terminate with single EndIf

All ElseIF/Else conditions have to remain

within parenthesis for the member association

Be sure to include ;

To If or FIX

Generally, If works best on dense dimension

members Particularly when ElseIF or Else are included
conditions are processed Fix would bring block in multiple times

Block is only brought into memory once and all related

Conversely, FIX works best on sparse

dimension members Old saying Fix on Sparse, If on dense If youre dense, fix on sparse? There are always exceptions

Other Method to Focus Calculations

Cross Dim operator

Relationship functions
database

Allows reference to specific cells Frequently in another block Technique for complex allocations Can be used to get a value from another part of the
Example is @ParentVal()

MemberSet functions

Return member names Can be used to calculate those members or in a Fix/If

Loops

Can use the Loop/EndLoop to repeatedly

execute calc commands Not designed to increment variables and reference different members Used to solve goal seeking problems

Why Create Blocks

Often need blocks that do not exist
writing calc scripts

Actually, one of the most challenging aspects of

Typically when performing allocations

Except outline calculations

If a block does not exist, not processed by a calc script Frequent cause of calculation problems
IF (Budget > 100)
Actual = 50;

Example

ENDIF Assumes Scenario is sparse

Ways to Create Blocks

There are basically four methods to create

blocks:

Load data Calculate the database Use the DataCopy command Place a sparse member on the left side of an
assignment statement

A little more complex than this

Load Data

During a data load, a block is created if it

doesnt already exist

Can use this to create blocks Create data records based on source information Load the file One trick is to use the Replace function of the
load rule

Calculate the Database

An outline rollup will create upper level

blocks Both Calc All/Dim and AGG will cause blocks to be created Only blocks containing data are created

Empty blocks are not written

DataCopy Command

Easiest way to create blocks If an intersection for the To member does

not exist, it is created Frequently, the only practical method to create the needed blocks Danger in that it can create excessive numbers of blocks

Assignment Statements

If

The member on the left of the assignment

statement is sparse
And

The database setting Create bocks on

Blocks are created Not always possible to have a sparse
a sparse member equations is selected

member on the left side of the assignment statement

Try fixing on a dense member and assigning to

E.g. Fix on the account member and assign to another

May need to change dense/sparse settings

Changing Database Settings

A quick diversion When you change database settings, you

should stop and then start the database (or application) for the settings to take effect

Organizing Calc Scripts

Most calc scripts have common parts or

sections

Housekeeping Initial Calculation Mainline Rollup Back Calc

Housekeeping

The calculation environment is established Variables are declared Comments describing the purpose of the calc
script are entered

Intelligent calculation is turned on/off Other Set commands are issue

Use /* and */ for comments

Key Set Commands

Set AggMissg ON | OFF

#Missing

Controls if #Missing values are aggregated If all children and #Missing, parent is et to Caution needed if values loaded at upper levels

Upper level step-on

SET MSG ERROR | WARN | INFO | SUMMARY |

DETAIL | NONE; Detail lists each block being processed Good for debugging Generates a large log and is slow

Initial Calculation

This is where business rules are applied Frequently include allocations

FIX statements calc script

There may be many calculation blocks with Typically the most complex portion of the

This usually requires a partial database rollup Or If statements

Mainline Calculation

This is the rollup of the database May be focused Regardless, this is the major rollup of the
database

e.g. FIX(Actual)

Back Calc

This is where two-pass members and any

other members requiring special treatment are handled Often requires recalculation of all or most of the blocks Corrects percentages and ratios that may have been summed

Testing Calc Scripts

The following methodology works well:

Use a test database Create test data in Excel

Keep it on its own sheet

Clear the database Use Lock & Send to load test data Run the calc from Excel Retrieve onto a test sheet

More on Testing

Best to know expected outcome before

retrieving Can create a calc script to clear the database

Minimizes switching between Excel and App

Manager

The key is small amounts of data which

allows short test cycles

Many people test with large amounts of data

resulting in long cycles and inability to get many tests in per day

When initial testing is complete, add to the

test data set to make sure everything is working properly

Typical Issues
Intelligent Calc

FIX on too few or too many members Dynamic Calc runs after Batch Calc Referring to the wrong block
the calculation is focused

Prevents blocks from being calculated Nested FIXs are ORs, not ANDs

Dynamic calc members do not make good counters Remember, every intersection is processed unless

Real World Calcs

These exercises are derived from

calculations created for customers but modified for the Sample application

Exercise

Simple Allocation Complex Allocation Comparable Sales Compute LQ (last quarter) in the scenario dimension Compute 12 month rolling average Compute Market Share and Product Mix Get data from another database

Exercise

(Continued)

Putting it together

This is for the Budget dimension only Clear upper level blocks Add the member Insurance to the Measures

dimension and load a value at the top level Allocate the insurance expense to all products based on Budgeted Sales Calculate Market Share in the calc script (may need to change storage settings); insure it is correct at all levels Include values for Last Quarter and a 6 month rolling average Perform a rollup of the database

Goal

Make the calculation run faster Variety of settings/changes available to make

things run faster Many limiting factors

Existing reports/processes Source systems Hardware

Parallel Calculation

Apply multiple processors to a calc Only straight forward calcs can use this

Maximum of 4 processors Recommend 1 less than number on server Available in Essbase XTD Analytic
Services 6.5

If order dependent portions, calculated in serial

Cache Settings

More is not always better

Sometime Hyperion Essbase seems to spend

more time maintaining caches than calculating

Usually find a point where increasing cache

does not improve performance Typically a trial and error process Set uncommitted / 0 on Transaction tab

Dense/Sparse Settings

This will make the largest difference Other factors influence decisions If only a portion of the database is calculated
ideally you want to Fix on it as sparse members

Attribute dimensions

May need to try a number of combinations

Reduces the number of blocks processed Smaller blocks almost always calc faster

Review Dimension Order

Largest dense dimension to smallest Smallest sparse dimension to largest Hourglass shape Dimension calc order is influenced by order of dimensions in the outline Goal is to process fewest blocks possible

Eliminate Database Fragmentation

Previously calculated database takes longer

to calculate access

Non-adjacent blocks can take longer to

be rewritten to same place

Faster to create blocks than to update them As data is added to blocks, they grow and cannot Old DeFrag Command no longer available

Export, ClearData (All), Import OR

Restructure time

Want to calc everything in the block at same

Review Blocks that are Processed

Can use SET MSG DETAIL Each block is listed as it is calculated Can determine number of times each block is
processed Can also see if expected blocks are being processed

See application log

Often, extra blocks are being read Correct by updating FIX

Control Calculation Environment

Use SET Commands

Set CACHE HIGH; Set

New cache Tracks the block used in the calc Good for flat dimensions
CALCHASHTBL ON;

Set FromBottomUp ON;

Faster than TopDown Make sure you get correct results

Other Methods

Remove dimensions from the cube

Can make a huge difference Some companies keep multiple databases

One with all dimensions and other with limited

Partition

Move data that does not need to be calculated to

another database

Intelligent Calc

More SET Commands

SET LOCKBLOCK HIGH; SET CALCTASKDIMS n;

Allows more blocks to be locked Not really optimization but Determines how many dimensions are used to
generate opportunities for the parallel calculation

Simulate the Calculation

Method to predict the actual calc time Based on the number of blocks that would be
created SET MSG ONLY; SET NOTICE HIGH; CALC ALL; Does not actually calc the database but provides timings Run the calc, and use ratio of actual time to simulated

Not perfect but

Dynamic Calc

Can improve calc times significantly Make entire dimension dynamic calc Especially effective for Time dimension All upper level members of dense dimension Upper 1-2 levels of sparse dimension(s)
calc highest level

Never needs to be calculated

Normally calculates very slowly Keeps block smaller

Be aware of how many blocks are needed to

Custom Functions

Can write functions in Java Compile, save on server, and register Call from calc scripts Run slower than same calc command May run faster than complex series of database passes required to perform the same thing

Custom Macros

Can predefine combinations of commands

SumRange is actually a Macro

Not necessarily faster but might avoid errors Need to define parameters May be possible to save a pass of blocks

Combines Sum and Range functions

Database Statistics

Not a optimization technique but a tool Can monitor calculation impacts Can refresh while calc is running Copy and paste into Excel

Number of blocks created

Block creation rate Not necessarily best for the calc

Comments / Questions ?

Edward Roske

Office: Email:

972-735-8716 eroske@interrel.com

Please Complete Your Session Evaluation.

459A
Calc Scripts for Mere Mortals
Presented by: Edward Roske, interRel Consulting