How Simple Finance Removes

Redundancy
https://blogs.saphana.com/2014/09/30/how-simple-finance-removes-redundancy-2/

At SAP, we set out to change how finance is done. Traditional systems relied on
inflexible models, precomputed data, and slow computational systems. With SAP
HANA and SAP Simple Finance we changed that, to give you a more agile approach
to financial management and planning. By eliminating things such as needleless
duplication and precomputation of data which clogs other systems, we have
significantly lowered your cost of managing Financials. And because our experience
at building robust accounting systems, we can do this without disruption to your
business. By freeing your system of the constraints of precomputed data models, we
also open up your system to a more agile way of computing, allowing financial
analysts to quickly experiment with what-if models while speeding quarterly closes.
This blog post is the first of a series that dives deeper into various aspects of SAP
Simple Finance and their technical underpinnings.
Recently, Hasso Plattner wrote about the impact of aggregates and explored the
negative impact of materialized aggregates and why relying on them for performance
severely restricts flexibility. Generally speaking, the generic term materialized
view and the special case materialized aggregate refer to the physical storage of
derived and redundant data in a database. In this deep dive, we look closer into the
technical foundations and highlight the positive impacts that can be realized when
removing materialized views. We do this by looking at SAP Simple Finance SAPs
next-generation Financials solution that for the first time does accounting without
materialized views powered by SAP HANA. We are going to explore why
materialized views and materialized aggregates are no longer necessary and how
removing redundant data storage in a non-disruptive manner improves transactional
throughput and lowers storage costs without compromising analysis performance.

In this blog post, we focus on the non-disruptive changes to the data model that
removed redundancy from SAP Simple Finance and why switching to Simple
Finance is possible in an entirely non-disruptive manner. In the next two blogs we
will investigate the concepts of materialized views and materialized aggregates,
respectively, and demonstrate that it is indeed feasible with in-memory database
systems to get rid of these redundant constructs. Future parts of the deep dive series
will also highlight additional improvements and paradigms of Simple Finance that are
possible thanks to SAP HANA and focus on, for example, the business value
associated with Simple Finance, non-disruptive innovation, and how Simple Finance
enables decision makers to overcome aggregate information loss.
In this first part of the series, we begin with exploring the concept of redundancy in
general. Afterwards, we look deeper into thechanges of the data model brought with
Simple Finance and highlight how this non-disruptive innovations has been possible,
allowing an almost seamless switch-over to SAP Simple Finance. We demonstrate
the positive impact of the new data model ondatabase footprint and transactional
throughput.

While we focus on the Financial Accounting component, other components such as

Management Accounting (Controlling) have similarly been changed, so our
comments apply to Simple Finance as a whole. The following explanations focus on
the classical data model in order to foster easier understanding for all readers, even
those not familiar with New General Ledger Accounting (New G/L). Please note that
SAP Simple Finance is using New G/L.

SAP HANA Makes Redundant Data Obsolete

Redundant data refers to data that is physically stored in a database but could be
derived by calculating it from other data in the database (Codd: A Relational Model
of Data for Large Shared Data Banks; Communications of the ACM 13(6); 1970).
Redundancy is a frequent source of inconsistency and anomalies. As redundant data
needs to be kept in sync on updates, redundancy in a data model leads to slower
updates. To reduce redundancy, database normalization techniques such as normal
forms have been introduced (Codd: Further Normalization of the Data Base
Relational Model; IBM Research Report RJ909; 1971). Historically, redundant data
has nevertheless often been stored explicitly only to improve read performance, as
the calculations to derive it in the absence of materialization required too much
additional effort.

For years, enterprise applications have employed redundant data storage in order to
provide sufficient performance to users in transactional and analytical applications
alike. The limited performance of traditional, disk-based database systems required
redundantly kept data that could be accessed quickly, but needed to be updated and
kept in sync with transactional changes. SAP ERP Financials has historically not
been an exception. In view of millions or billions of accounting documents (headers,
stored in table BKPF) and their line items (table BSEG), materialized views and
aggregates as mechanisms to maintain redundant data provided fast access to items
with specific properties. For example, SAP ERP Financials contained a materialized
view for all open Accounts Receivable line items (table BSID) and materialized
aggregates of the debit, credit, and balance amounts for each customer per fiscal
period (table KNC1).

Building on SAP HANAs in-memory technology, it has now been possible to non-
disruptively transform the Financials system into a purely line-item-based Simple
Finance that gets rid of all redundant financials data. Thus, SAP Simple Finance
overcomes the associated costs such as reduced transactional throughput and
increased database footprint. At the same time, SAP Simple Finance is a non-
disruptive innovation of the classical ERP Financials because it replaces the
materialized views with non-materialized compatibility views. All applications that
have read from the materialized views, be it SAP standard reports or customer
modifications, continue to access the views as before without requiring any changes.

Although these are already big breakthroughs, the most important advantage of SAP
Simple Finance is the dramatically improved flexibility that, for example, encourages
exploring the data according to various analysis needs and contributes to the
success of Simple Finance as an information system. Researchers have since long
stressed the value of information flexibility see, for example, Searching and
Scanning: How Executives Obtain Information from Executive Information Systems
(Vandenbosch and Huff, 1997; MIS Quarterly 21(1)), Exploring the perceived
business value of the flexibility enabled by information technology infrastructure
(Fink and Neumann, 2009; Information & Management 46), or An empirical
investigation of the factors affecting data warehousing success (Wixom and
Watson, 2001; MIS Quarterly 25(1)). A follow-up blog post will elaborate on the
business value that can be realized thanks to the gain in flexibility and the
simplification.

A New, Non-disruptive Data Model with Zero Redundancy

The fundamental and essential data tuples of every financial accounting system are
the accounting documents and line items, in this case stored in the
tables BKPF and BSEG, respectively. The traditional data model of SAP ERP
Financials which is running on SAP HANA as part of SAP Business Suite powered
by SAP HANA (in the following: Suite on HANA) additionally contained
materialized views that allowed fast access on disk-based database systems to open
and cleared line items, separated by accounts receivable, accounts payable, and
general ledger accounts (tables BSID, BSAD, BSIK, ). As calculations and
observations in future installments of this deep dive series will show, SAP HANAs
in-memory database makes materialization obsolete thanks to the improved
performance. Hence, the decision for SAP Simple Finance was clear: the previously
existing materialized views have been removed and replaced by compatibility views.
These compatibility views transparently provide access to the same information
calculated on the fly to ensure that the changes are non-disruptive.

Materialized aggregates in the traditional Financials data model included totals per
account receivable (KNC1), account payable (LFC1), and G/L account
(GLT0 / FAGLFLEXT). In addition to removing the materialized views, SAP Simple
Finance also evolves the traditional Financials data model by removing the
materialized aggregates from the system, again replacing them by compatibility
views.

In the end, the transactional data model of core Financial Accounting besides
master data consists only of the essential accounting document header (BKPF)
and accounting document line item tables (BSEG). All the other tables mentioned on
the left side of the following Figure 1 are then obsolete. Instead, transparent access
to exactly the same information is provided on-the-fly by equally named compatibility
views, as outlined on the right side of Figure 1.

Figure 1: Key elements of Financials data model before and after simplification

Non-disruptive Innovation
SAP ERP Financials is being used all over the world by many thousand customers. It
offers rich modification options to customer to adapt the system to their needs,
including custom modifications. Changes to the core data model need to take this
into account. They have to be non-disruptive with regard to the large amount of code
(by SAP and its customers) that builds on top of the data model. To ensure adoption
of the benefits of SAP Simple Finance, all innovations have been implemented in a
non-disruptive manner. Switching to SAP Simple Finance does not require updating
any custom coding for reading from the database.

So-called compatibility views have been the means to achieve this non-
disruptiveness: a compatibility view is a non-materialized view essentially, a named
query that seamlessly replaces a materialized view (or materialized aggregate). As
it has the same name and is transparently accessed via the data dictionary, existing
applications that so far relied on the materialized view for accessing items or
aggregates do not experience any difference. They continue to work as before
without any changes to the code. For example, a query accessing BSID has
previously read the value from the materialized view. Now, the same access
to BSID is resolved into a query using the view definition and calculated on-the-fly.
The result is the same in both cases. As the calculations have demonstrated, this is
feasible without compromising performance.

As a consequence of the non-disruptive innovation, all of the benefits associated

with Simple Finance such as increased throughput, flexibility, and reduced
database size are available at no adaptation costs to customers a migration of
the database (mostly removing the redundant tables) is sufficient.

A future deep dive will look into non-disruptive innovation and the technology of
compatibility views in more detail.
As a benefit of the non-disruptive nature of the above-mentioned changes, code
changes in the Financials component (done by SAP) were only necessary to remove
the now unnecessary writing operations to former materialized views and aggregates
when posting accounting documents. In addition, code has been adapted to further
leverage the possibilities of SAP HANA. This includes code pushdown of data-
intensive logic to the database and backend decoupling for new Web-based user
interfaces (SAP Fiori).

Reduction of Database Footprint

The removal of both materialized views and materialized aggregates significantly
reduces the database footprint of Financials that has been occupied by redundant
data. In an exemplary system from an SAP customer, the total size occupied by the
set of core tables for financial accounting is reduced by a factor of 6.5 with SAP
Simple Finance. For details, please refer to the following table that includes a non-
exhaustive list of tables that have been replaced by compatibility views.

Table sizes of customer

system
ERP Simple
Financials in Finance on
Suite on HANA
Table Description HANA
BKPF Accounting documents 1.2 GB 1.2 GB

BSEG Accounting document line 4.5 GB ** 5.0 GB

items
BSID Open items Accounts 0.1 GB
Receivable
BSIK Open items Accounts 0.0 GB
Payable
BSIS Open items General Ledger * 23.2 GB
BSAD Cleared items Accounts 3.3 GB
Receivable
BSAK Cleared items Accounts 2.1 GB
Payable
 BSAS Cleared items General * 5.5 GB
Ledger
KNC1 Totals Accounts Receivable 0.1 GB

LFC1 Totals Accounts Payable 0.0 GB

GLT0 Totals General Ledger 0.0 GB

Total (of above tables) 40.0 GB 6.2 GB

* Includes partially archived items (not considered here)

** Includes additional fields for New G/L
The numbers shown above are in both cases already for a system running on SAP
HANA as the database. Factoring in the additional savings in database size enabled
by HANAs storage architecture and compression, the potential savings are even
more impressive. Looking at SAPs main internal ERP system, the following Figure 2
demonstrates the end-to-end possibilities in database size reduction. The
combination of SAP HANAs inherent compression with the removal of redundancy
allows running a company such as SAP with revenue of close to 17 billion Euro in
2013, over 250,000 customers, and over 65,000 employees completely in-memory
using commodity enterprise hardware.

Figure 2: Database sizes of SAPs main internal ERP

Increase of Transactional Throughput

The removal of materialized views and materialized aggregates immediately has a
positive impact on the transactional throughput of the system. Posting an accounting
document no longer requires inserting redundant duplicates into materialized views
nor updates of redundant total figures. Beforehand, especially the concurrent
aggregate updates could lead to contention as materialized aggregates had to be
locked for updating. In case of, for example, heavily used G/L accounts, the
database system had to handle the otherwise parallel postings sequentially. In
Simple Finance, posting now only requires inserting the actual document and its line
items. Experimental measurements based on customer data (see Figure 3) have
shown that the end-to-end transaction time required for posting 500 documents with
6 line items each has been reduced by a factor of more than 2, doubling the
throughput. The number of tuples affected by the database operations for posting the
documents has been reduced by a factor of almost 2.5 thanks to the no longer
required synchronization of redundant data.

Figure 3: Increased transactional throughput

Benefits of SAP Simple Finance in Summary

The data model of SAP Simple Finance is now exclusively consisting of line items. It
is simply recording all business transactions as they happen. Everything else is
being calculated on-the-fly by algorithms on the data. This implies that the previously
needed modifying operations to maintain the materialized views are no longer
necessary, thereby simplifying the program code and increasing the transactional
throughput.
The fact that it is feasible to remove the redundancy from the Financials data model
demonstrates that in-memory database systems enable new levels of flexibility.
Instead of pre-defining materialized views and materialized aggregates, it is possible
to aggregate flexibly directly on the line items. Instead of being restricted to specific
analysis questions supported by the underlying model of materialization, all
questions are feasible that can be answered based on the actual items themselves
via any aggregation on top of them. An improved and user-friendly user experience
adds to that and allows unprecedented user productivity and novel insights.

This first blog post of the Simple Finance deep dive series looked at the non-
disruptive changes to the data model. In particular, we outlined the practical benefits
associated with removing redundancy of materialized views and materialized
aggregates; increased throughput and reduced database footprint. In the upcoming
blog posts of the series, we will demonstrate how SAP HANA as an in-memory
database makes these changes feasible using performance calculations and
technical considerations.