Highlights of the HANA smart data streaming (SDS) SPS11 release:
Introduction of Machine Learning Algorithms for stream processing
SPS11 introduces 2 new machine learning algorithms in SDS. These are native CCL functions that can be utilized directly within streaming projects. These have been implemented by the PAL team and the chosen algorithms are specifically designed to operate against live data streams in real-time.
SPS11 introduces SDS support for HANA System Replication
SDS nodes will be created in the replica system
Deployed SDS projects will be copied to the replica
Streaming projects will require a manual restart to allow coordination with outside data sources
State information for running projects including log files for log backed windows are not replicated in SPS11. Support for this functionality is on the roadmap for a future release
Simplified Configuration of the SAP HANA Output Adapter
Older options that explicitly set the commit interval, number of events to be committed in a single batch and the number of worker threads to be used have been replaced by new parameters that set upper limits for each of these values that the adapter will then work within
The total number of parameters that can be set for the HANA Output Adapter has been reduced. The adapter now automatically adjusts the insert array size, the max queue size and the maximum length of time to buffer data before outputting it to HANA.
Custom Header Protection Support for the HTTP Client Output Adapter
This is an important new feature for streaming projects that integrate to an XS application. XS application enforce the use of a Cross-Site Request Forgery (CSRF) token.
With this feature, the HTTP Client Output Adapter can now automatically request and use a CSRF token
New Custom Adapter File Directory
SDS supports the development, integration and use of custom adapters to provide integration options to data sources not already supported by our 'out-of-the-box' adapters
Custom adapters built using the Adapter Toolkit can be integrated directly into the palette in the SAP HANA Streaming Development perspective in HANA Studio
To provide better organization of custom adapters, the custom adapter files are now stored in the STREAMING_CUSTOM_ADAPTERS_HOME directory
Streaming Web Service (SWS) Enhancements
In SPS 10, the SWS is configured, started and stopped as a separate process. While it was often run on the same host as the SDS server node, it still had to be manually started and stopped.
With SPS 11, the SWS is now managed as part of the SDS cluster and can be configured to automatically start when the SDS cluster starts
If you are upgrading from SPS10, then there are some one time manual migration steps required to migrate your current SWS settings from the old configuration file into HANA Cockpit
Data Service Enhancements
This is a security enhancement that provides greater control over who has access to specific data services. Prior to SPS11, any data service was visible across all workspaces on the SDS server. With SPS11 you can now choose to limit the data service availability to a single workspace, or continue to make it available across all workspaces on the server.
Incremental Enhancements Based on User Feedback
Added multi-threading support for custom row based formatters used in building custom adapters
Support for additional dynamic attributes for file/hadoop adapters
New methods have been added to the C, .NET and Java SDKs to process formatting errors with fixed decimal values