After upgrading from 2.13 to 2.15 Discover in Dashboards is barely usable due to the above error. Other functions are possibly affected, too. The error message refers to core.js so it seems like a OpenSearch core issue not a Dashbaords issue.
Configuration:
OpenSearch is run in a Kubernetes environment. JSON processors are used in ingest pipelines but it is to be determined whether the JSON keyword in the error message refers to the processors or not.
Relevant Logs or Screenshots:
The full error message is:
JSON.parse: bad escaped character at line 1 column 207567 of the JSON data
HttpFetchError@https://opensearch.our.domain/7749/bundles/core/core.entry.js:15:235732
fetchResponse@https://opensearch.our.domain/7749/bundles/core/core.entry.js:15:243032
This could be an general OpenSearch bug, a specific bug in the JSON processor, or bad logs/documents being sent to OpenSearch.
The most pressing question is how to debug this further. There are no obvious errors in the OpenSearch logs and no indication how and where to search for the possibly corrupted documents.
The error is only present in one index pattern. And it does not appear every time but in about 2/3 of all cases, e.g. when refreshing. So, I think if it is caused by a document it’s a needle-in-a-haystack situation.
@pablo Of course, if there is a specific document causing this, it would definitely help. Part of my initial question how to find this document. I have in my installation about a thousand documents within the last 15 minutes. They come from different sources, so they are different in structure and content.
I will try to narrow it down by using filters but I am not sure if that helps.
Add.: I just realized, that does not work. Once the error is present, there are no documents in discover. After refreshing the documents might be there again and the error vanished, but I guess this is due to the problematic document being no longer within my 15-minutes window.
@noeplana Discover UI gives you option to narrow down to a specific time frame other than last 15 minutes.
Then you could keep narrowing down to the specific document within that time frame.
If this is caused by a document, then there is a chance that this issue will repeat.
We have an open issue about this on Github. @kksaha@noeplana@OpenAndreas can you add details there about the issue you are facing? I will need some help here to reproduce the issue since this doesnt seem to occur with the sample dataset, so getting the offending document is important so that we can root cause this.