SAP BW Real Time Data Acquisition

Introduction Currently in Appraisal Cycle for Personal Development Area we are not able to view the most recently updated appraisal status in BW queries. Requirement is to reflect the most recently updated appraisal status in BW reports. Reasons for BW reports not showing the most recent data in queries might be: 1. Its hard to automate the daily loads in less than an hour interval with the normal data-source loading mechanism. If we do so by using multiple process chains, we will get multiple entries per hour in RSMON and its hard to debug and see logs for readily increasing number of requests in RSMON. 2. Data will not available for reporting unless the load request is activated for them (Activating request is not the spontaneous process). 3. Time loss in OLTP and OLAP processing for reading a delta queue which might have big bunch of data. One of the possible solutions for the above mentioned problem of not getting most recent data for BW reporting can be the Real Time Data Acquisition (RDA) concept of ETL. Business Scenarios where we can use RDA concept: RDA functionality is normally used when some tactical decisions need to be taken from the most recent data.

RDA Introduction We might be having complex reports in our BI System, which helps in making decisions on the basis of data of our transactional system. Sometimes (quarter closure, month end, year ending...) single change in the transactional data can influence our decision, hence it is very important to get the most recent data into BW reports from the transactional system. Using new functionality of Real-time Data Acquisition (RDA) with the NetWeaver (BI) 2004s system we can now load transactional data into SAP BI system every single minute The source system for RDA could be SAP System or it could be any non-SAP system. SAP is providing most of the Standard Data-Sources as real-time enabled. Prototype Scenario: There are 4 main extractors (Figure: 1) which load some critical applications like Job Status. In order to track the loading details for these extractors at real time we use RDA functionality of BI 7. Requirement is like whenever the extractors are executed their date of run and username should reflect in BW query with no loss of time. Extractor last run details table

Figure: 1

Prototype Solution To achieve the real time reflection of data, the Step by Step approach is as below:

Create generic data source from Transaction Code RSO2 as below (Figure: 2): ZBW_JOB_STATUS is a table for last run details of the extractors

Figure: 2

For RDA the data source must be delta enabled with the real time data access check (Figure: 3). Click on GENERIC DELTA of the generic data-source just created (Figure: 3).

Figure: 3

Once created the REAL TIME ENABLED data source, we can test the extractor with RSA3 T-Code (Figure: 3). Replicate the data source into the BW system. After the data source is replicated, we need to activate the Data source with the following setting: We can clearly see the Real time data enabled Data source. Select the adaptor as Real time Extraction (Figure: 4)

Figure: 4

Data source setting of the fields:

Figure: 5

Create a standard DSO with the below mention settings; Key and Data fields.

Figure: 6 We have created DSO and data source both. Now we need to create the transformation between Data-source and the DSO

Figure: 7

For data load to DSO from Data-source create DTP as below (Figure: 9):

Figure: 8

For real time data load we must have init load done up to the Info-provider (Figure: 9):

Figure: 9

Once DTP is over and the init data load till DSO is done we can create Real time enabled Info-package.

Make the already created DTP (Loading the Data to DSO) a Real Time DTP by clicking on the icon given below (this can be seen when we double click on DTP):

Once we clicked on the button we get the DTP changed to below screen Figure: 10

Figure: 10 The button in the DTP maintenance (screen shot below) we will take us to Real time Data load Monitor:

We can get the option of making RDA enabled info-package as in below screen shot (Figure: 11): Choose below encircled check box (Figure: 11) for making info-package RDA enabled.

Figure: 11

For one RDA data-source only one RDA info-package possible. For the RDA info-package select adaptor as: Real time extraction from SAP System :

Figure: 12

For further upload of data we need to click on Assign icon the below screen (Figure: 13).

in

Figure: 13 We can use T-Code RSRDA to get into RDA monitor too.

Figure: 14

Below is the screen shot (Figure: 15) of the RDA Monitor:

Figure: 15 Click on the below given icon to create new Daemon for upload the data near to real time.

Daemon definition: Background process that processes the InfoPackages and data transfer processes assigned to it at regular intervals. Daemon use: The daemon controls and monitors the transfer process for real-time data acquisition. The daemon performs the following tasks depending on the scenario used: If you are transferring data from an SAP source system, the daemon starts the transfer of data into the PSA using an Info Package. It controls the status of the data transfer. It starts the further processing of data into the DataStore object using a data transfer process. It closes and opens requests when threshold values are reached. It triggers the subsequent process chains.

The daemon receives information from the InfoPackage and data transfer process about the data to be transferred or further updated, PSA tables or DataStore objects to be filled, and when a request should be closed and a new one opened. On clicking

Icon we got to screen as below where we can set the number of minutes after which we need to look into data source delta queue for data. Here we are taking 1 min. Figure: 16

Figure: 16 Now we have to assign our RDA data source to Daemon. This can be done with context menu of the data source which by default get assigned to Unassigned Nodes

Figure: 17 Selecting Assign Daemon we will come to the screen as below where we assign data source to Daemon 5:

Figure: 18

After assigning click on to confirm our assignment. Now we can see our RDA data source assign to Daemon.

Figure: 19 From the context menu of Data source as per screen shot below ,assign DTP for loading data from PSA to DSO

Figure: 20 For starting RDA process upload: o Go to context menu of Daemon

Figure: 21 One you clicked Start Daemon with all InfoPackages the upload begins and we can see the process continuation as in below screen

Figure: 22 From the above screen shot (Figure: 22) the section given below gives us the information about the new records read by daemon from delta queue (Figure: 23) ):

Figure: 23 From the context menu of Traffic light signals for status Figure: 24(Can be seen in below screen shot):

Figure: 24 We can go to the detailed monitor for the upload request.

Figure: 25

From the Manage screen of the DSO we can see that the running request is also available for reporting and at the same time it will get data from the upload process see Figure: 26.

Figure: 26 We can do manual stopping of the upload request from the context menu of the assigned DTP Figure: 27 (See below screen)

Figure: 27

We can also do manual stopping of the upload request from the context menu of the Daemon assigned Data source Figure: 28 (See below screen)

Figure: 28

One interesting point in this is that the numbers of requests are not going to be increased as anything. As the daemon looking into delta queue is a minute process but a separate request will be created as per the RDA InfoPackage setting which can be from 1 hour to a day or so.

Figure: 29 Every minute process can be monitored via a PSA monitor as per below screen shot(Figure: 30):

Figure: 30

Some result outputs: 1. At very first instance the query out put is (please see Figure: 31 the time encircled in red):

Figure: 31 2. We manually run the extractor and try to get our names in the report with the recent date.

3. The query output is as below Figure: 32 (Encircled is the real time updated record):

Figure: 32 References: 1. https://www.sdn.sap.com/irj/sdn/weblogs?blog=/pub/wlg/4918 2. http://help.sap.com/saphelp_nw04s/helpdata/en/52/777e403566c6 5de10000000a155106/frameset.htm