We use Spark to insert data to Opensearch.
We have migrate to Open Distro to Opensearch.
And now, we have this error message : No type found; Types are required when writing
It’s normal ? (disappeared since version 7). Type is re-introduce ?
org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: No type found; Types are required when writing in ES versions 6 and below. Expected [index]/[type]
Seems like the spark library recognizer Opensearch as an older version of ES. Guessing it’s working well on ES>7?
Yes this spark library works in ES v5, v6. And Open Distro 1.13. So i dont understand why not work with Openseacrh 1.0
We must use which spark library compatible ?
Thanks for help.
@GnarlyCapricorne So, what’s happening here is that the library is specifically checking the version number and nothing else. The version number says 1.0 and thinks it’s Elasticsearch 1.0.
What you can do to fix this is run in compatibility mode which returns version information like Open Distro 1.13.
You can turn it on by running this:
Also, FYI, Elasticsearch-hadoop (including the spark connector) is on the list of clients that are in the process of forking. So in the coming months a new, OpenSearch specific version will come out and this setting won’t be needed.
I will test your tip and back to you to confirm
Nice thanks you all ! Bye