In this article, we will be discussing about the different data provisioning techniques available for SAP HANA.
When we built Data Warehousing system with SAP HANA centric, it is very important to have a clear idea on different data provisioning techniques available.
For HANA, we can divide the data provisioning options available into two categories, they are
- SAP HANA in-built tools
- External Tools
The below image gives us the pictorial representation of data provisioning techniques currently available to us for HANA.
SAP HANA in-built options
These are the data provisioning techniques that come in-built along with SAP HANA or as an additional option which doesn’t require a separate infrastructure to maintain.
We have 5 different techniques present in this category as of SAP HANA SPS 10.
1.EIM (Enterprise Information Management):
- The Enterprise Information Management-based data provisioning option uses smart data integration and smart data quality to load data, in batch or real-time, into HANA (on premise or in the cloud) from a variety of sources using pre-built and custom adapters.
- We implement this method by installing a Data Provisioning Agent to house adapters and connect the source system with the Data Provisioning server, housed in the HANA system.
- The next step is to create replication tasks, using Web IDE, to replicate data, or flow graphs, using Application Function Modeler nodes, to transform and cleanse the data and load to HANA.
- This feature is available from SAP HANA SPS9 version and requires additional licensing cost.
2.Flat File Upload:
- Using the SAP HANA in-built functionality, we can load the data from a flat file (excel, .csv) into SAP HANA using HANA Studio.
- Please go through the article ‘Data Loading to HANA from Flat File’ to know the step by step procedure of the same. This option is available from HANA initial version.
3.Remote Data Sync:
- The Remote Data Sync service on HCP (HANA Cloud Platform) is used to synchronize huge numbers of remote databases into a consolidated SAP HANA database in the cloud. This service is based on SAP SQL Anywhere and its MobiLink technology.
- The Remote Data Sync service can be used for scenarios with occasional Internet connectivity and it provides a sophisticated strategy for resolving data change conflicts. By this, it ensures transactional integrity also over unstable networks.
- The synchronization of data can be used bi-directional, that is, to synchronize data both from a remote database into the cloud database, and vice versa. Typical scenarios in which Remote Data Sync service can be used are from the Internet-of-Things (IoT) area.
Note: This service is a beta functionality that is available on the SAP HANA Cloud Platform trial landscape for developer accounts. This option is available only from SAP HANA SPS10 revision.
4.Smart Data Streaming:
- The SAP HANA smart data streaming option processes high-velocity, high-volume event streams in real time, allowing us to filter, aggregate, and enrich raw data before committing it to your database.
- With SAP HANA smart data streaming, you can accept data input from a variety of sources including data feeds, business applications, sensors, IT monitoring infrastructure and so on, apply business logic and analysis to the streaming data and store your results directly in SAP HANA.
- This option is available from SAP HANA SPS 9 revision.
5.Smart Data Access (SDA):
- This option is used to remotely access the data from any source without physically loading to SAP HANA and can be used to build modeling objects on top of the data.
- This is achieved by creating remote connection and then virtual tables on top of source tables. The restriction with virtual tables are, it can be only used to build calculation views in SAP HANA.
- This option is available from SAP HANA SPS6 revision.
Note: We need to pay additional license cost to make use of additional data provisioning techniques in SAP HANA like EIM, Smart Data Streaming and Remote Data Sync..
Apart from in-built data provisioning tools, we also have SAP certified tools which can be used to load data from different sources to HANA either in real-time or batch. Licensing is different for this tools and needs separate infrastructure to run these tools.
We have four tools in the category, they are
1.SAP Replication Server (SRS):
- SAP Replication Server (SRS) moves and synchronizes transactional data including DML and DDL across the enterprise, providing low impact, guaranteed data delivery, real-time business intelligence, and zero operational downtime.
- SRS supports log-based replication from and to heterogeneous databases, except for the homogeneous SAP HANA to SAP HANA replication, which is trigger-based. We can use SAP Replication Server to do an initial load as well as replication in real time to SAP HANA, at both table and database level, from these primary databases:
- SAP Adaptive Server Enterprise
- Microsoft SQL Server
- DB2 UDB LUW
- SAP Business Suite running on SAP ASE, Oracle, MSSQL Server or DB2 UDB LUW
- SAP HANA database
- Also, SRS includes Replication Agent for SAP HANA (RAH), a light-weight server that replicates data from a primary SAP HANA to a replicate SAP HANA database using trigger-based replication, primarily used for real time data distribution and real time reporting.
2.Direct Extractor Connection (DXC):
- The SAP HANA Direct Extractor Connection (DXC) is used to redirect data from embedded SAP BW system (For SAP ECC extractors) to HANA table using http connection.
- We face significant complexity while building modeling objects for SAP ECC extractors in SAP HANA.
- In many cases, data from different areas in SAP Business Suite systems requires application logic to appropriately represent the state of business documents. SAP Business Content Data Source Extractors have been available for many years as a basis for data modeling and data acquisition for SAP Business Warehouse; now with DXC, these SAP Business Content Data Source Extractors are available to deliver data directly to SAP HANA.
- DXC is a batch-driven data acquisition technique; it should be considered as a form of extraction, transformation and load although its transformation capabilities are limited to user exit for extraction.
3.SAP Data Services:
- SAP Data Services is an enterprise level ETL (Extraction, Transformation and Loading) tool which can used be used to load data from any source to any target in either real-time or batch.
- SAP Data Services is a certified ETL tool from SAP to perform batch loading into SAP HANA. Please go through the below articles to know more about SAP Data Services.
4.SAP Landscape Transformation (SLT):
- The SAP Landscape Transformation tool uses trigger-based technology to transfer the data from any source to SAP HANA in real-time. Most of the time this tool is used if the source is an SAP application like SAP ECC, CRM.
- Please go through the below article to get details about how to replicate data from source to SAP HANA in real-time using SLT.
With this we have seen different in-built and external data provisioning techniques available to load/replicate data into SAP HANA system.
Thank you for reading and hope this information is helpful. Please do share with your friends if you feel the information is useful.