Doing Data Analysis with Jaspersoft

Mary Flynn, Director of Worldwide Sales Engineering, Jaspersoft Sherman Wood, Vice President Product Management, Concentrix Odie Henderson, Senior Professional Services Consultant, Jaspersoft

Todays Discussion
Common analysis tasks A brief history of analysis products at Jaspersoft JasperReports Server ad hoc
Domain capabilities and limitations Analyzing data from any source: topics Performance tracking

Jaspersoft OLAP
OLAP Data and Process Schema design with Jaspersoft OLAP Designer MDX query profiling Cache, no cache or shared cache in a load balanced environment

Great resources
2011 Jaspersoft Corporation. Proprietary and Confidential 2

Common analysis tasks

Drilling, slicing and dicing, pivoting, filtering and charting

Demonstrations
Drilling, slicing and dicing, filtering, pivoting and charting Discovery, segmentation, refining and display

2011 Jaspersoft Corporation. Proprietary and Confidential

A brief history of analysis products at Jaspersoft

In the Beginning

Reporting server
Repository, security Report generation and scheduling Shared services

Optional service: JasperAnalysis

Renamed: Jaspersoft OLAP

R-OLAP engine and UI integrated into the server

Mondrian engine + JPivot user interface (branch)

2011 Jaspersoft Corporation. Proprietary and Confidential

Advent of Ad Hoc
Requirement: easier reporting for business users Added web-based report designer
Banded detail reports Automatically extract queries from JasperReports files -> TOPICS
<queryString language="plsql"> <![CDATA[{call mypackage.paramsproc($P{cursor}, $P{region})} ]]> </queryString>

2011 Jaspersoft Corporation. Proprietary and Confidential

Accessible Analysis
Requirement: easier analysis for business users Extended ad hoc report designer
Drag-and-drop fields to design a crosstab Domains for dynamic queries

Enhanced Crosstabs: in-memory analysis

Much more dynamic

2011 Jaspersoft Corporation. Proprietary and Confidential

Jaspersoft OLAP
Minor enhancements and engine updates over the years Not-so-secret work underway to query the powerful OLAP engine (Mondrian) from the Jaspersoft ad hoc user interface What makes the engine powerful? Answer: MDX

2011 Jaspersoft Corporation. Proprietary and Confidential

Parallel Period: One Calculation

[Measures].[Parallel Period] Formula: ( ParallelPeriod( [Time].[Year] , 1 , [Time].CurrentMember) , [Measures].[Unit Sales] )

[Measures].[Diff Parallel] Formula: [Measures].[Unit Sales] [Measures].[Parallel Period]

2011 Jaspersoft Corporation. Proprietary and Confidential

Parallel Period: Multi-level Results

Calculates at all levels: Year, Quarter, Month, etc.

2011 Jaspersoft Corporation. Proprietary and Confidential

Prior Period: One Calculation

Calculated Member: [Measures].[Prior Unit Sales] Formula: ( [Time].PrevMember, [Measures].[Unit Sales] )

Calculated Member: [Measures].[Diff Prior] [Measures].[Unit Sales] [Measures].[Prior Unit Sales]

2011 Jaspersoft Corporation. Proprietary and Confidential

Prior Period: Multi-level Results

Calculates at all levels: Year, Quarter, Month, etc.

2011 Jaspersoft Corporation. Proprietary and Confidential

Direction: Merge the Best of Both

The JasperReports Server web-based report designer is a superior user interface The Mondrian engine is very powerful Direction: front-end the Mondrian engine with the Jaspersoft web-based report designer

2011 Jaspersoft Corporation. Proprietary and Confidential

14

Until Then
Learn and use the best of both! First up: JasperReports Server Then: Jaspersoft OLAP Finally: Great Resources

2011 Jaspersoft Corporation. Proprietary and Confidential

15

JasperReports Server: ad hoc

Tips and tricks: Domain capabilities and limitations Analyzing data from any source: topics Performance tracking, see the query

Domain Capabilities and Limitations

Domain:
Database schema represented in XML

XML converts to SQL

defaultSQLGenerator oracleSQLGenerator postgreSQLGenerator sqlserverGenerator
MS SQL Server Ingres

db2SQLGenerator mmxSQLGenerator custom

2011 Jaspersoft Corporation. Proprietary and Confidential

17

Domain Capabilities and Limitations

XML snippet from a domain:
<joinedDataSetRef> <joinString>join store store on (sales_fact_1997.store_id == store.store_id)</joinString> </joinedDataSetRef>

defaultSQLGenerator converts at runtime to:

inner join `store` on (`sales_fact_1997`.`store_id` = `store`.`store_id`)

Notice `back-ticks` were generated for MySQL in this example

2011 Jaspersoft Corporation. Proprietary and Confidential

18

Domain Capabilities and Limitations

Sample generated by SQLGenerator
Query select store.store_type as store_store_type, sum(sales_fact_1997.store_sales) as newSet1_store_sales_Sum from sales_fact_1997 inner join store on (sales_fact_1997.store_id = store.store_id) where store.store_country = 'USA' group by store.store_type order by store_store_type

2011 Jaspersoft Corporation. Proprietary and Confidential

19

Domain Capabilities and Limitations

The database and database schema matter a lot Complex join paths => complex queries => long queries Operational database engines (e.g., MySQLs Innodb) are optimized for rapid, small INSERTS, not SELECTS

2011 Jaspersoft Corporation. Proprietary and Confidential

20

Domain Tips
You can get a domain from a star schema in minutes Columnar databases are well worth evaluating If you dont have a star or snowflake schema, consider:
Staging data Using a Topic (next up) that runs an optimized stored procedure

Not shown: Derived Tables run as sub-queries; this may impact performance After Topics: performance tracking tips
2011 Jaspersoft Corporation. Proprietary and Confidential 21

Analyzing Data From Any Source: Topics

Topics were Jaspersofts first data source for ad hoc report design

2011 Jaspersoft Corporation. Proprietary and Confidential

22

Analyzing Data From Any Source: Topics

Topics are JasperReport units stored in the Topics folder

2011 Jaspersoft Corporation. Proprietary and Confidential

23

Analyzing Data From Any Source: Topics

JasperReports Server lists all reports in this folder in the Topics panel

2011 Jaspersoft Corporation. Proprietary and Confidential

24

Analyzing Data From Any Source: Topics

Why is this great? You can use ANY query, ANY data source Jaspersoft has a rich custom data source capability Custom queries (hibernate, Hadoop, web services, etc.) Stored procedures, etc.

2011 Jaspersoft Corporation. Proprietary and Confidential

25

Analyzing Data From Any Source: Topics

Topics
Queries are extracted from JasperReports (JRXML) files The query language only needs to be registered with JasperReports Server Topics can use any data source, including custom data sources The query is pre-defined / canned

Domains
Queries are built on-demand by the SQLGenerator The SQLGenerator converts XML (the Domain definition) to SQL at run-time Domains require a JDBC or JNDI data source Domains can make many more fields available to users

Tip: use Domains for relational databases; best with star schemas Tip: use Topics for non-SQL data sources or complex queries
2011 Jaspersoft Corporation. Proprietary and Confidential 26

Analyzing Data From Any Source: Topics

Topics
Queries are extracted from JasperReports (JRXML) files The query language only needs to be registered with JasperReports Server Topics can use any data source, including custom data sources The query is pre-defined / canned

Domains
Queries are built on-demand by the SQLGenerator The SQLGenerator converts XML (the Domain definition) to SQL at run-time Domains require a JDBC or JNDI data source Domains can make many more fields available to users

Tip: use Domains for relational databases; best with star schemas Tip: use Topics for non-SQL data sources or complex queries
2011 Jaspersoft Corporation. Proprietary and Confidential 27

Analyzing Data From Any Source: Topics

Topics
Queries are extracted from JasperReports (JRXML) files The query language only needs to be registered with JasperReports Server Topics can use any data source, including custom data sources The query is pre-defined / canned

Domains
Queries are built on-demand by the SQLGenerator The SQLGenerator converts XML (the Domain definition) to SQL at run-time Domains require a JDBC or JNDI data source Domains can make many more fields available to users

Tip: use Domains for relational databases; best with star schemas Tip: use Topics for non-SQL data sources or complex queries
2011 Jaspersoft Corporation. Proprietary and Confidential 28

Analyzing Data From Any Source: Topics

Topics
Queries are extracted from JasperReports (JRXML) files The query language only needs to be registered with JasperReports Server Topics can use any data source, including custom data sources The query is pre-defined / canned

Domains
Queries are built on-demand by the SQLGenerator The SQLGenerator converts XML (the Domain definition) to SQL at run-time Domains require a JDBC or JNDI data source Domains can make many more fields available to users

Tip: use Domains for relational databases; best with star schemas Tip: use Topics for non-SQL data sources or complex queries
2011 Jaspersoft Corporation. Proprietary and Confidential 29

Performance Tracking, See the Query

Demonstration: Ad Hoc cache

2011 Jaspersoft Corporation. Proprietary and Confidential

30

Jaspersoft OLAP
Tips and Tricks: OLAP Data and Process Schema design with Jaspersoft OLAP Designer MDX query profiling Cache, no cache or shared cache in a load balanced environment

Jaspersoft OLAP Process

Get your data: clean it up, physical star schema design
RDBMS decisions, data integration (if needed)

Define the OLAP schema

Describe the data, identify facts and dimensions

Get some starting points for analysis

OLAP views in the JasperReports Server

Deploy and tune

Server configurations for effective use Excel Pivot Tables with Jaspersoft OLAP services

Keep going
Regular data updates Expect expanding number of views over time

Data for Jaspersoft OLAP

Need a star schema
Model against an existing database Create a new database optimized for OLAP operations

OLAP environment characteristics

Select/read queries with grouping across large data sets Bulk loads Multiple data sources may be needed to get the data needed for analysis: need to consolidate

Technology options
Can your transactional environment take it? Can your RDBMS take it? Or is it not efficient? Or cost effective? Look at different RDBMS options within your current environment, or analytic databases

2011 Jaspersoft Corporation. Proprietary and Confidential

33

Schema Design: Jaspersoft OLAP Designer

DBA tool to create an OLAP schema
XML file

Also creates:
Access Grant Schema MDX queries (for OLAP Views)

Client-side installation: Java Swing application Tip: download from

http://support.jaspersoft.com/downloads-JAWBPro4_0_0.html
2011 Jaspersoft Corporation. Proprietary and Confidential 34

Schema Design: Jaspersoft OLAP Designer

Identifies errors

Tip: look here for troubleshooting help

2011 Jaspersoft Corporation. Proprietary and Confidential 35

Schema Design: Jaspersoft OLAP Designer

Automatically generates MDX queries Great feature for testing

2011 Jaspersoft Corporation. Proprietary and Confidential

36

Schema Design: Jaspersoft OLAP Designer

Interrogates the cube(s) to create MDX queries Check results for issues MDX can be saved or copied Helpful when creating OLAP Views in JasperReports Server

2011 Jaspersoft Corporation. Proprietary and Confidential

37

Deploying OLAP, Step 1

In Jaspersoft OLAP Designer:

1. Design and test the schema 2. Run and save an MDX query 3. Save the schema to disk (it will be an XML file)

2011 Jaspersoft Corporation. Proprietary and Confidential

38

Deploying OLAP, Step 2 In JasperReports Server

3.3

3 4

2011 Jaspersoft Corporation. Proprietary and Confidential

39

Deploying OLAP, Step 2 In JasperReports Server

1. Create a JDBC or JNDI data source (if not already 2. 3.

available) Upload the OLAP Schema Create an OLAP Client Connection
1. 2. 3. 4. 1. 2. 3. Type: Mondrian Select the OLAP Schema (from step 2) Select the JDBC or JNDI data source (from step 1) Do not link an Access Grant (until youre expert) Type: Mondrian Select the Mondrian Client Connection (from step 3) Paste your MDX query string remove the semicolon at the end

4. Create an OLAP View

2011 Jaspersoft Corporation. Proprietary and Confidential

40

MDX Query Profiling

Analysis Options > Performance Profiling Good for testing, developing better MDX queries Saves the query and the response time See: JasperAnalysis Ultimate Guide 3.7, Section 5.3.2 Profiling Performance Note: Auditing is different. See the chapter on Auditing in the JasperReports Server Administrator Guide.
2011 Jaspersoft Corporation. Proprietary and Confidential 41

Jaspersoft OLAP Cache: default is ON

c

Shared Repository

load balancer Other services

Application servers running JasperReports Server

2011 Jaspersoft Corporation. Proprietary and Confidential 42

Traditional Data Sources

Jaspersoft OLAP XML/A Provider

Shared Repository

load balancer Other services

c
Application servers running JasperReports Server
2011 Jaspersoft Corporation. Proprietary and Confidential 43

Jaspersoft OLAP XML/A provider

Traditional Data Sources

Alternative: Use Databases Cache

Shared Repository

load balancer Other services

Application servers running JasperReports Server

2011 Jaspersoft Corporation. Proprietary and Confidential 44

Analytic or Columnar Database

Great Resources
JasperReports Server and Jaspersoft OLAP

JasperReports Server, Ad Hoc

Documentation
JasperReports Server Evaluation Guide release 4.0 Chapter 3 Getting Started with Ad Hoc Reports Section 3.3 Exploring and Analyzing Data in the Ad Hoc Editor JasperReports Server User Guide release 4.0 Chapter 4 Working with the Ad Hoc Editor Chapters 6 and 7: Creating and Advanced Domain Features JasperServer Ultimate Guide 3.7 Chapter 3 Ad Hoc Reports and Data Exploration
2011 Jaspersoft Corporation. Proprietary and Confidential 46

Training
Using Jaspersoft BI For business users 8 hours delivered in (2) 4-hour classes Live online Jaspersoft Technical Training For software developers and system architects Four days Live online

Jaspersoft OLAP
Documentation
JasperReports Server Evaluation Guide release 4.0 Chapter 4 Getting Started with OLAP Views Jaspersoft OLAP User Guide release 4.0 Mondrian 3.0 Technical Guide JasperAnalysis Ultimate Guide 3.7

Training
OReilly: DBA 4: Analyzing Data* Jaspersoft Technical Training

* When registering for OReilly courses, please enter university@jaspersoft.com in the Referred Email field.

2011 Jaspersoft Corporation. Proprietary and Confidential

47

Thank You
Mary Flynn Sherman Wood Odie Henderson