Hello. My team is using Elasticsearch on AWS to provide search functionality over our customer database. The index is continuously updated by a Logstash instance using the logstash-output-amazon_es plugin. To ensure that changes in existing customers cause document updates rather than the creation of new documents, we set a custom document ID like so (id
is one of the columns being read by the JDBC input plugin):
filter {
mutate {
add_field => { "[@metadata][_id]" => "%{id}" }
}
}
output {
amazon_es {
...
document_id => "%{[@metadata][_id]}"
}
}
Now we would like to update to OpenSearch and use the logstash-output-opensearch plugin, but apparently that plugin does not support setting a custom document_id
. Is there another way we can ensure a 1:1 relationship between customers in the database and in the index?
Thanks in advance!