No type found; Types are required when writing

OpenDistro
1

Hello,

We use Spark to insert data to Opensearch.

We have migrate to Open Distro to Opensearch.

And now, we have this error message : No type found; Types are required when writing

It’s normal ? (disappeared since version 7). Type is re-introduce ?

org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: No type found; Types are required when writing in ES versions 6 and below. Expected [index]/[type]

Best Regards

Gnarly

2

Seems like the spark library recognizer Opensearch as an older version of ES. Guessing it’s working well on ES>7?

3

Hello,

Yes this spark library works in ES v5, v6. And Open Distro 1.13. So i dont understand why not work with Openseacrh 1.0

We must use which spark library compatible ?

Thanks for help.

4

@GnarlyCapricorne So, what’s happening here is that the library is specifically checking the version number and nothing else. The version number says 1.0 and thinks it’s Elasticsearch 1.0.

What you can do to fix this is run in compatibility mode which returns version information like Open Distro 1.13.

You can turn it on by running this:

PUT _cluster/settings
{
  "persistent": {
    "compatibility": {
      "override_main_response_version": true
    }
  }
}

(more info)

Also, FYI, Elasticsearch-hadoop (including the spark connector) is on the list of clients that are in the process of forking. So in the coming months a new, OpenSearch specific version will come out and this setting won’t be needed.

1 Like
5

hello,

I will test your tip and back to you to confirm :slight_smile:

6

Nice thanks you all ! Bye