In this blog we will try to fetch the sensor data generated periodically at regular intervals and written to a .csv file using a standard file adapter implemented in my
previous blog which in turn uses the 'FILE' protocol.
When To Use A File Adapter
Let's assume that a company has to analyze huge amount (Can be further integrated with SAP Cloud Analytics) of data coming from numerous sensors at the same time.These data are written to .json or a .csv file.
Implementation Steps
As mentioned in my last blog,the Internet of Things Gateway Edge adapter implementation requires a data model to publish data.This data model can be created either in the IoT service cockpit by creating 'Sensor Types' which in turn has multiple 'Capabilities' and 'properties'. Each sensor type has a 1:n mapping with all the capabilities which represents the data received from the sensors like volume,pressure,temperature in terms of properties etc.
The other way of creating the the Sensor Types -> Capabilities is by uploading a data model which would resemble the type of data that would come from the sensors.This would require downloading the IoT sdk starter kit and creating the model template and the capabilities manually.
It depends on the business scenario and amount of capabilities and how to visualize them to decide which way to follow out of the above two.
Lets follow the first step which is straight forward and is suitable when we have to deal with few sensor data types and which requires less effort.
Suppose the data collected from the sensors is written in a .csv file and has the following details:
- Log on to the IoT Service Cockpit with your user credentials.
- Create a capability by clicking on the 'Capabilities' link on the left hand navigation, call it as say 'TestCapability'.Add the same properties as mentioned in the .csv file.Use the '+' symbol to add new properties.You can give any 'Alternate ID' as you like while creating the capability.Also you can select any data types as per your business requirement:
- Create a new sensor type by clicking on the 'Sensor Types' link on the left hand navigation.And add the above created capability to it as a measure.
- Go to the local folder where you had configured the file adapter as mentioned in the previous blog.
- Navigate to the folder 'IOTCHCP20_0-70002561\gateway-4.20.0\config\schemas' and create a schema file for the data model to map the capabilities and corresponding properties.Name it say 'test_data_FILE.json':
- Open the 'test_data_FILE.json' in a text editor and copy the following content:
{
"type": "object",
"parserData": {
"format": "CSV_POSITION",
"resource": "FILE_content",
"messageType": "test_data"
},
"actions": [{
"type": "ADD",
"mapTo": "sensor_type_alternate_id",
"value": 396
}],
"properties": {
"0": {
"type": "number",
"mapTo": "value",
"actions": [{
"type": "ADD",
"mapTo": "capability_alternate_id",
"value": "236"
},{
"type": "ADD",
"mapTo": "property_name",
"value": "ID"
}]
},
"1": {
"type": "string",
"mapTo": "sensor_alternate_id",
"format": "integer"
},
"2": {
"type": "number",
"mapTo": "value",
"actions": [{
"type": "ADD",
"mapTo": "property_name",
"value": "loadDate"
}]
},
"3": {
"type": "number",
"mapTo": "value",
"actions": [{
"type": "ADD",
"mapTo": "property_name",
"value": "loadIndex"
}]
},
"4": {
"type": "number",
"mapTo": "value",
"actions": [{
"type": "ADD",
"mapTo": "property_name",
"value": "loadTemp"
}]
},
"5": {
"type": "number",
"mapTo": "value",
"actions": [{
"type": "ADD",
"mapTo": "property_name",
"value": "loadSignalStrength"
}]
},
"6": {
"type": "number",
"mapTo": "value",
"actions": [{
"type": "ADD",
"mapTo": "property_name",
"value": "loadBatterystrength"
}]
},
"7": {
"type": "number",
"mapTo": "value",
"actions": [{
"type": "ADD",
"mapTo": "property_name",
"value": "loadCapacity"
}]
}
}
}
- Now navigate to the folder 'IOTCHCP20_0-70002561\gateway-4.20.0\config\readFileConfig' and create or edit the 'readFile.json' with the following content:
{
"files" :
[{
"path" : "config\\gateway-file-examples\\test.csv",
"mode" : "periodic",
"interval": 20000,
"schema": "test_data",
"deviceAlternateId": "863835021503557",
"backupFolder": "config\\fileBackupFolder\\test_data"
}]
}
Note: You can give any device alternate Id as you wish,this will bind the newly created device with sensor type and corresponding capabilities and measures.
- Now click on the 'gateway.bat' for windows or 'gateway.sh' for macOS to start your adapter registration process.You can check the progress in the CMD to see that the nodes that you have mentioned in the schema file are created along with the new device and corresponding alternate id for the device as we had mentioned in the 'readFile.json':
- Now log on to your IoT Cockpit and click on 'Device Management->Devices'.You can now see your newly registered device along with the test sensor that we had created earlier.
- Click on the 'Data Visualization' tab and select the associated sensor and capability and select the properties you want to analyse:
- Enter some more values for the corresponding properties in the 'test.csv' file which would be backed up in the 'test_data' folder and put it in the 'gateway-file-examples' folder which is periodically scanned by the file adapter to check for new contents.
Note: We can further send this data to some analytic platform like SAP Analytic Cloud for more in depth analysis and prediction with the inbuilt predictive features based on the underlying machine learning algorithms which forms a basis for SAP Analytic Cloud solution offered by SAP.Details on how we can do this in my future blogs so stay tuned.