CAP on Node.js OData v2 $batch collecting req.errors issue - only one of requests returns errors

[#1]

Hi colleagues,

I have an OData v2 service based on Node.js CAP.

The service receives queries in a batch format.

--batch_df3d-e5a9-254b
Content-Type: multipart/mixed; boundary=changeset_0359-ad5d-07cc

--changeset_0359-ad5d-07cc
Content-Type: application/http
Content-Transfer-Encoding: binary

MERGE POItems_QuantityChange(purchaseOrder='4500000002',purchaseOrderLine='00010') HTTP/1.1
X-Requested-With: XMLHttpRequest
Prefer: handling=strict
sap-contextid-accept: header
Accept: application/json
Accept-Language: en
DataServiceVersion: 2.0
MaxDataServiceVersion: 2.0
Content-Type: application/json
Content-ID: id-1691421850671-932
Content-Length: 258

{"newQuantity":"1111","__metadata":{"type":"CreateRequests.POItems_QuantityChange","uri":"https://port4004-workspaces-ws-ghq2b.eu10.applicationstudio.cloud.sap/v2/create-requests/POItems_QuantityChange(purchaseOrder='4500000002',purchaseOrderLine='00010')"}}
--changeset_0359-ad5d-07cc
Content-Type: application/http
Content-Transfer-Encoding: binary

MERGE POItems_QuantityChange(purchaseOrder='4500000002',purchaseOrderLine='00011') HTTP/1.1
X-Requested-With: XMLHttpRequest
Prefer: handling=strict
sap-contextid-accept: header
Accept: application/json
Accept-Language: en
DataServiceVersion: 2.0
MaxDataServiceVersion: 2.0
Content-Type: application/json
Content-ID: id-1691421850671-934
Content-Length: 258

{"newQuantity":"1111","__metadata":{"type":"CreateRequests.POItems_QuantityChange","uri":"https://port4004-workspaces-ws-ghq2b.eu10.applicationstudio.cloud.sap/v2/create-requests/POItems_QuantityChange(purchaseOrder='4500000002',purchaseOrderLine='00011')"}}
--changeset_0359-ad5d-07cc
Content-Type: application/http
Content-Transfer-Encoding: binary

MERGE POItems_QuantityChange(purchaseOrder='4500000002',purchaseOrderLine='00012') HTTP/1.1
X-Requested-With: XMLHttpRequest
Prefer: handling=strict
sap-contextid-accept: header
Accept: application/json
Accept-Language: en
DataServiceVersion: 2.0
MaxDataServiceVersion: 2.0
Content-Type: application/json
Content-ID: id-1691421850671-936
Content-Length: 258

{"newQuantity":"1111","__metadata":{"type":"CreateRequests.POItems_QuantityChange","uri":"https://port4004-workspaces-ws-ghq2b.eu10.applicationstudio.cloud.sap/v2/create-requests/POItems_QuantityChange(purchaseOrder='4500000002',purchaseOrderLine='00012')"}}
--changeset_0359-ad5d-07cc--

--batch_df3d-e5a9-254b--<br>

What I need is to be able to receive multiple errors from several requests of the batch. In the update handler I'm currently raising it in a next way:

    const creationRequestErrors = await this.applyCreationRequestValidations(itemForRequest, newQuantity, requestReasonId, comment, req);

    if (creationRequestErrors.length) {
      
      creationRequestErrors.forEach((e) => req.error({
        code: httpCodes.badRequest,
        message: e.longMessage + itemForRequest.purchaseOrderLine,
        target: 'newQuantity',
        status: httpCodes.badRequest,
      }));

      return;
    }<br>

When there's more then one error in a single request - everything is fine. I receive "Multiple errors occurred" with details of each one.

Although when errors appear in multiple requests of the $batch - the response contains only errors from one of the requests of $batch (not necessarily first one).

--batch_df3d-e5a9-254b
content-type: application/http
content-transfer-encoding: binary
content-id: id-1691421850671-936

HTTP/1.1 400 Bad Request
content-type: application/json
dataserviceversion: 2.0
content-length: 359

{"error":{"code":"400","message":{"lang":"en","value":"Request Reason is required.00012"},"target":"newQuantity","severity":"error","ContentID":"id-1691421850671-936","innererror":{"errordetails":[{"code":"400","message":{"lang":"en","value":"Request Reason is required.00012"},"target":"newQuantity","severity":"error","ContentID":"id-1691421850671-936"}]}}}
--batch_df3d-e5a9-254b--

My OData v2 setup

cds.on('bootstrap', (app) => {
    app.use(cov2ap({
        propagateMessageToDetails: true,
        continueOnError: true
    }));
});

My Node.js env packages are

cds -v
@sap/cds: 6.8.4
@sap/cds-compiler: 3.9.4
@sap/cds-dk: 7.0.2
@sap/cds-dk (global): 6.8.1
@sap/cds-foss: 4.0.2
@sap/cds-mtx: -- missing --
@sap/cds-odata-v2-adapter-proxy: 1.9.21
@sap/eslint-plugin-cds: 2.6.3
@sap/textbundle: 4.2.0
@sap/xssec: 3.2.18
Node.js: v16.19.0

Can it be fixed? Or is there any workarounds?

Thank you in advance.

[OliverKlemenz · Accepted solution]

OData V4 runtime stops processing of batch, after a single request did fail. It has to be indicated that you want to continue on error. Can you try OData V2 Adapter option continueOnError?

See: https://github.com/cap-js-community/odata-v2-adapter#options

- continueOnError: Indicates to OData V4 backend to continue on error. Default is false.

[epamtiosteel · Accepted solution]

Hi Oliver,

thanks for a quick response!

I already added that option, also attempted to add propagateMessageToDetails

cds.on('bootstrap', (app) => {
    app.use(cov2ap({
        propagateMessageToDetails: true,
        continueOnError: true
    }));
});

Unfortunately I was not able to recognise any changes.

Though through the debugger I entered handlers of every request despite the value of the flag. So even though one request raised an error - the else are attempted to be executed which is bad because I have a few external calls there.

[OliverKlemenz · Accepted solution]

Ok, that's strange. Normally the behavior should be, that each request is processed, I checked the OData server code and here it's mentioned as described:

@sap/cds/libx/_runtime/cds-services/adapter/odata-v4/okra/odata-server/serializer/BatchMultipartSerializer.js (line 49)

// If continue-on-error is not set, exit after the first failed request.
if (!this._batchContext.isContinueOnError() && this._batchContext.isRequestFailed(id)) break

[OliverKlemenz · Accepted solution]

Maybe you can check at that line, if setting the option continueOnError really takes effect, as assumed. propagateMessageToDetails is not relevant in this matter.

[epamtiosteel · Accepted solution]

I just set

cds.on('bootstrap', (app) => {
    app.use(cov2ap({
        propagateMessageToDetails: true,
        continueOnError: false
    }));
});

And tried the same request. This is the process from cap logger perspective. Debugger also hits 4 times.

[cds] - POST /create-requests/$batch 
[cds] - > UPDATE POItems_QuantityChange(purchaseOrder%3D'4500000002'%2CpurchaseOrderLine%3D'00010') 
[cds] - > UPDATE POItems_QuantityChange(purchaseOrder%3D'4500000002'%2CpurchaseOrderLine%3D'00011') 
[cds] - > UPDATE POItems_QuantityChange(purchaseOrder%3D'4500000002'%2CpurchaseOrderLine%3D'00012') 
[cds] - > UPDATE POItems_QuantityChange(purchaseOrder%3D'4500000002'%2CpurchaseOrderLine%3D'00013') 
[cds] - Error: Request Reason is required.00010
    at /home/user/projects/ppe-poa-common/srv/create-requests.js:237:48
    at Array.forEach (<anonymous>)
    at createRequests.onUpdatePurchaseOrderItems (/home/user/projects/ppe-poa-common/srv/create-requests.js:237:29)
    at async next (/home/user/projects/ppe-poa-common/node_modules/@sap/cds/lib/srv/srv-dispatch.js:76:17)
    at async createRequests.handle (/home/user/projects/ppe-poa-common/node_modules/@sap/cds/lib/srv/srv-dispatch.js:74:10)
    at async _updateThenCreate (/home/user/projects/ppe-poa-common/node_modules/@sap/cds/libx/_runtime/cds-services/adapter/odata-v4/handlers/update.js:94:14)
    at async /home/user/projects/ppe-poa-common/node_modules/@sap/cds/libx/_runtime/cds-services/adapter/odata-v4/handlers/update.js:150:24 {
  code: 400,
  target: 'newQuantity',
  status: 400,
  severity: 'ERROR',
  numericSeverity: 4,
  id: '1090337',
  level: 'ERROR',
  timestamp: 1691425709519
}
[cds] - Error: Request Reason is required.00013
    at /home/user/projects/ppe-poa-common/srv/create-requests.js:237:48
    at Array.forEach (<anonymous>)
    at createRequests.onUpdatePurchaseOrderItems (/home/user/projects/ppe-poa-common/srv/create-requests.js:237:29)
    at async next (/home/user/projects/ppe-poa-common/node_modules/@sap/cds/lib/srv/srv-dispatch.js:76:17)
    at async createRequests.handle (/home/user/projects/ppe-poa-common/node_modules/@sap/cds/lib/srv/srv-dispatch.js:74:10)
    at async _updateThenCreate (/home/user/projects/ppe-poa-common/node_modules/@sap/cds/libx/_runtime/cds-services/adapter/odata-v4/handlers/update.js:94:14)
    at async /home/user/projects/ppe-poa-common/node_modules/@sap/cds/libx/_runtime/cds-services/adapter/odata-v4/handlers/update.js:150:24 {
  code: 400,
  target: 'newQuantity',
  status: 400,
  severity: 'ERROR',
  numericSeverity: 4,
  id: '1090337',
  level: 'ERROR',
  timestamp: 1691425709524
}
[cds] - Error: Request Reason is required.00011
    at /home/user/projects/ppe-poa-common/srv/create-requests.js:237:48
    at Array.forEach (<anonymous>)
    at createRequests.onUpdatePurchaseOrderItems (/home/user/projects/ppe-poa-common/srv/create-requests.js:237:29)
    at async next (/home/user/projects/ppe-poa-common/node_modules/@sap/cds/lib/srv/srv-dispatch.js:76:17)
    at async createRequests.handle (/home/user/projects/ppe-poa-common/node_modules/@sap/cds/lib/srv/srv-dispatch.js:74:10)
    at async _updateThenCreate (/home/user/projects/ppe-poa-common/node_modules/@sap/cds/libx/_runtime/cds-services/adapter/odata-v4/handlers/update.js:94:14)
    at async /home/user/projects/ppe-poa-common/node_modules/@sap/cds/libx/_runtime/cds-services/adapter/odata-v4/handlers/update.js:150:24 {
  code: 400,
  target: 'newQuantity',
  status: 400,
  severity: 'ERROR',
  numericSeverity: 4,
  id: '1090337',
  level: 'ERROR',
  timestamp: 1691425709527
}
[cds] - Error: Request Reason is required.00012
    at /home/user/projects/ppe-poa-common/srv/create-requests.js:237:48
    at Array.forEach (<anonymous>)
    at createRequests.onUpdatePurchaseOrderItems (/home/user/projects/ppe-poa-common/srv/create-requests.js:237:29)
    at async next (/home/user/projects/ppe-poa-common/node_modules/@sap/cds/lib/srv/srv-dispatch.js:76:17)
    at async createRequests.handle (/home/user/projects/ppe-poa-common/node_modules/@sap/cds/lib/srv/srv-dispatch.js:74:10)
    at async _updateThenCreate (/home/user/projects/ppe-poa-common/node_modules/@sap/cds/libx/_runtime/cds-services/adapter/odata-v4/handlers/update.js:94:14)
    at async /home/user/projects/ppe-poa-common/node_modules/@sap/cds/libx/_runtime/cds-services/adapter/odata-v4/handlers/update.js:150:24 {
  code: 400,
  target: 'newQuantity',
  status: 400,
  severity: 'ERROR',
  numericSeverity: 4,
  id: '1090337',
  level: 'ERROR',
  timestamp: 1691425709530
}
[cov2ap] - Batch: Changeset order deviation {
  req: [ 'id-1691425696886-211', 'id-1691425696887-213', 'id-1691425696887-215', 'id-1691425696887-217' ],
  res: []
}<br>

Response contains error from line 00010.

--batch_cfd0-dc15-f47c
content-type: application/http
content-transfer-encoding: binary
content-id: id-1691425696886-211

HTTP/1.1 400 Bad Request
content-type: application/json
dataserviceversion: 2.0
content-length: 359

{"error":{"code":"400","message":{"lang":"en","value":"Request Reason is required.00010"},"target":"newQuantity","severity":"error","ContentID":"id-1691425696886-211","innererror":{"errordetails":[{"code":"400","message":{"lang":"en","value":"Request Reason is required.00010"},"target":"newQuantity","severity":"error","ContentID":"id-1691425696886-211"}]}}}
--batch_cfd0-dc15-f47c--<br>

[epamtiosteel · Accepted solution]

I checked

node_modules/@sap/cds/libx/_runtime/cds-services/adapter/odata-v4/okra/odata-server/serializer/BatchMultipartSerializer.js

The specified line clearly hits after requests were executed already and _responses property looks complete

[OliverKlemenz · Accepted solution]

OK, I see. I guess there is something not working as documented in OData V4 processing. It can be seen in log, that OData V2 adapter only gets one changeset id: [cov2ap] - Batch: Changeset order deviation ...
FYI vansyckel, david.kunz2: What is the expected behavior for continueOnError ("prefer", "odata.continue-on-error") ...? Shouldn't it process all request AND return for each request the error response?

In addition the response does not contain change sets anymore, reflection the request batch changeset structure with content-ids, and the last changeset with its content-id is included in batch response. This does not look right to me...

[david_kunz2 · Accepted solution]

> What is the expected behavior for continueOnError ("prefer", "odata.continue-on-error")

The expected behavior is that if one change set contains multiple requests, the change-set processing is immediately stopped at the first failure (there's a commit, so there's no need to process the other requests). But multiple change sets are processed.

Example response:

[
        '--batch_1',
        'content-type: application/http',
        'content-transfer-encoding: binary',
        '',
        'HTTP/1.1 405 Method Not Allowed',
        'odata-version: 4.0',
        'content-type: application/json;odata.metadata=minimal',
        '',
        '{"error":{"code":"405","message":"Method POST not allowed for ENTITY"}}',
        '--batch_1',
        'content-type: application/http',
        'content-transfer-encoding: binary',
        '',
        'HTTP/1.1 200 OK',
        'odata-version: 4.0',
        'content-type: application/json;odata.metadata=minimal',
        '',
        expect.stringContaining(',"ID":4,"NAME":"Mike","BIRTHYEAR":1950}'),
        '--batch_1--',
        ''
      ]

[OliverKlemenz · Accepted solution]

OK, so the idea I think would be to split the request up into separate change-sets (currently all requests are in one changeset), so that they are processed independently. And with continueOnError the next changeset is processed.

Open questions:

a) What I still find strange is, that the response does not contain the changeset information anymore and only has batch level structure.
Having multiple change set where some are successfully and some fail, I think it would be necessary, to also have the response on change set level? Why is this?

b) And also it seems, that all requests are processed (all handlers are called), see attached screenshots, but this contradicts, that requests in changeset are processed sequentially, and stop after the first one fails.

[epamtiosteel · Accepted solution]

>> But multiple change sets are processed.

>> OK, so the idea I think would be to split the request up into separate change-sets

I'd highlight that the payload is generated by default Fiori Elements Save button on Object page with editable table. My final goal is to show all errors of all lines raised by CAP. Here though I can only receive errors for 1 line. Any idea how to make them being submitted in different changesets?

>> b) And also it seems, that all requests are processed (all handlers are called), see attached screenshots, but this contradicts, that requests in changeset are processed sequentially, and stop after the first one fails.

Yes, exactly, all requests were executed. if it's important - it's srv.on('UPDATE', ...) handler.