JSON.parse: bad escaped character

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

2.15

Describe the issue:

After upgrading from 2.13 to 2.15 Discover in Dashboards is barely usable due to the above error. Other functions are possibly affected, too. The error message refers to core.js so it seems like a OpenSearch core issue not a Dashbaords issue.

Configuration:

OpenSearch is run in a Kubernetes environment. JSON processors are used in ingest pipelines but it is to be determined whether the JSON keyword in the error message refers to the processors or not.

Relevant Logs or Screenshots:

The full error message is:

JSON.parse: bad escaped character at line 1 column 207567 of the JSON data

HttpFetchError@https://opensearch.our.domain/7749/bundles/core/core.entry.js:15:235732
fetchResponse@https://opensearch.our.domain/7749/bundles/core/core.entry.js:15:243032

This could be an general OpenSearch bug, a specific bug in the JSON processor, or bad logs/documents being sent to OpenSearch.

The most pressing question is how to debug this further. There are no obvious errors in the OpenSearch logs and no indication how and where to search for the possibly corrupted documents.

Any troubleshooting hints are welcome.

Thank you!

@OpenAndreas Do you get this error with any index pattern in OpenSearch Dashboard Discover?

Same here. Clean install 2.15. Push some logs. Something breaks opensearch

@noeplana Have you tried creating index manually and configuring index pattern for it?
Could you check if you also get the error then?

Could you share an example document from your reported index?

Hi @pablo ,

oops, I found your posting just now.

The error is only present in one index pattern. And it does not appear every time but in about 2/3 of all cases, e.g. when refreshing. So, I think if it is caused by a document it’s a needle-in-a-haystack situation.

@OpenAndreas I think narrowing down to that specific document would help to investigate this issue further.

@pablo Of course, if there is a specific document causing this, it would definitely help. Part of my initial question how to find this document. I have in my installation about a thousand documents within the last 15 minutes. They come from different sources, so they are different in structure and content.

I will try to narrow it down by using filters but I am not sure if that helps.

Add.: I just realized, that does not work. Once the error is present, there are no documents in discover. After refreshing the documents might be there again and the error vanished, but I guess this is due to the problematic document being no longer within my 15-minutes window.

Sorry cant share, millions of documents in minutes

Downgrading to 2.14

@pablo Would it be sufficient to remove the current index and immediately recreating it manually? If not, which procedure is advised?

@noeplana Could you report back here on the effect of downgrading?