Smart Data Streaming - Push or Pull technology?

[former_member258450]

I was trying to understand the similarities and differences between Apache Kafka and SAP HANA Smart Data Streaming. Kafka's topic subscribers, consumers make use of the "pull" methodology to request for messages whenever needed from Kafka's queue - this is done to avoid consumer bottlenecks at the subscriber's end.

What methodology does SAP Smart Data Streaming follow - Push or Pull? I read on a blog that it uses "push"? Is this right? How are consumer bottlenecks prevented in this scenario?

[bintu__]

Hi jeff.wootton ,

Even I am facing Slow consumer issue while consuming messages via JSON Kafka adapter. Slow consumption resulted in lag in Source side.

I am using below mentioned parameters in my kafka Input adapter.

ATTACH INPUT ADAPTER i_kafka_1 TYPE toolkit_kafka_json_input

TO i_biz_stream

PROPERTIES

kafkaTopicPartition = 'EVENT1' ,

kafkaOffset = 0 ,

kafkaFromBeginning = FALSE ,

kafkaBootstrapServers = 'bk.0001.111' ,

kafkaGroupId = 'hanasds_consumer' ,

kafkaConsumerProperties = 'consum.properties' ,

kafkaPollInterval = 1 ,

jsonColumnMappingFile = '' ,

jsonColumnMappingXml = '' ,

jsonColsMappingList = 'topic,offset,value.content.eventId',

jsonRootpath = '' ,

additionalStreams = '' ,

gdMaxRetryTimes = 1 ,

jsonSecondDateFormat = '' ,

jsonMsDateFormat = '' ,

jsonTimeFormat = '' ,

jsonBigdatetimeFormat = '' ,

jsonCharsetName = '' ,

enableGdMode = TRUE ,

maxPubPoolSize = 100000 ,

useTransactions = FALSE ;

In the above comment , in 4th point you mentioned regarding controlling data rate. Could you please guide me how configure data rate. Is there any parameter change I need to do from my end. Please suggest. Thanks in advance.

[RobertWaywell]

You need to post this as a new question.