Unable to upload ndjson files to /api/saved_objects/_import

Open Source Elasticsearch and Kibana
1

We are attempting to move from vanilla Elasticsearch/Kibana to OpenDistro Kibana. We are about 95% there, but currently we’re stuck getting our script that imports dashboard and index pattern objects to work.

We’ve secured Kibana with SAML and have replicated the SAML auth flow via python requests. We can get to various endpoints that we’ve pulled out of the security-kibana-plugin source code, e.g. /api/v1/configuration/indices. However, we’re trying the following in python:

headers = {
  'Content-Type': 'application/json',
  'kbn-xsrf': 'true'
}

files = {
  'file': open('test.ndjson', 'rb')
}

resp = session.request("POST", "https://<kibana>/api/saved_objects/_import?overwrite=true", headers=headers, files=files)

And are getting:

{"statusCode":415,"error":"Unsupported Media Type","message":"Unsupported Media Type"}

Any help is appreciated. I’ve tried application/x-ndjson and a content-type, sending the file as part of data instead of files, and opening it with and without rb. Unfortunately SAML authentication makes testing this via curl exceedingly difficult, and I haven’t been able to replicate the request on the command line.

Thanks,

Sam

2

Managed to get this to work on my own. For anyone who runs into the same issue, what worked for me was omitting the Content-Type header altogether and letting the requests module submit the file as multipart form data instead of as json. Otherwise the above request should work as written.

1 Like
3

Do you have information on where to look for the API endpoints for visualizations and dashboards import and export?

4

I use a curl command like the following to load (import) content (dashboards, visualization, etc.) into Kibana:

curl -XPOST "http://myserver:5600/api/saved_objects/_import?overwrite=true" -H "kbn-xsrf: true" -H "securitytenant: global" --form file=@mycontent.ndjson

Obviously, you would need to change the host, port and name of the ndjson file…and, potentially, the securitytenant if you want to import to a different tenant.

This API is documented in the Kibana API doc rather than in the Elasticsearch API doc.

1 Like
5

Thanks for the reply, that is very useful, I will apply it. The username and password in kibana.yml are used for health checks and maintenance, and probably they can’t be used for accessing the server with the commands. The port in kibana.yml is 9200 at my server, but using that port with curl returned different error messages like connection refused. For finding and export, I used the following two examples. The first is in .ndjson format, it has a backslash at the end of every line, the second is normal .json.:

curl \
-u USER:PASSWORD \
-H ‘Content-Type: application/json’ \
-H “kbn-xsrf: true” \
-XGET “http://localhost:5601/api/saved_objects/_find?type=dashboard” \

curl -u USER:PASSWORD -X POST “http://localhost:5601/api/saved_objects/_export” -H ‘kbn-xsrf: true’ -H ‘Content-Type: application/json’ -d’
{
“type”: “dashboard”
}
’

6

You must change the the content-type to ‘Content-Type: application/ndjson’

7

@GSmith Greg, Is this API supported in the new open search dashboards? Because I have a script to automatically import saved objects that stopped working after I upgraded to OpenSearch 1.0.0 from Opendistro 1.12.x

8

@gopal_m I have not moved to OpenSearch yet so I can not say for sure whether there were any changes to this API. But I did a quick search on the OpenSearch Dashboard GitHub repo and see results (code and tests) referring to the API, so I would guess it should still be working.

What errors or response do you get when you call the the API now?

9

That’s right. The API support is still there. It’s just expecting a different header.

10

Great…glad to hear you got it working. Just so I know when I have to do this, what changed with the headers?

11

The header definition is changed from { 'kbn-xsrf' : 'true' } to { 'osd-xsrf' : 'true' }