Comparing documents from two different indices

OpenSearch
1

Hi.
We need to compare documents from two different indices.

We have new mechanism that inserts a new document to a new index.
Is there a way to make sure that the new document is the same as the document in the old index?
thank you

2

Hi @zarof ,

How do you insert a new document to a new index?

3

One of the possible solutions is to write a script that gets a document from the old index via API and then finds and compares it in the new index.

4

Can you tell me more about new mechanism?

As @Eugene7 said, you can easily compare two documents indexed in two different indices just using _search API.

5

Before flushing operation, it could also be possible to scrutinize two translogs in each shard. The documents are stored in the path like /data/nodes/0/indices/{uuid}/index/0/

6

I have an old Java service that inserts data to Opensearch.
and a new Java service, that uses different mechanism.
I want to make sure that I don’t have any gaps.

I was wondering if there is a way to query Opensearch about multiple documents, and get the results only (I have thousands of documents to compare).

7

Even you’ve adjusted the different way to index documents, each _id of documents is same right? (Because you have the very same dataset)

If so, just by calling the below APIs,

# (1)
GET {index}/_doc/{_id} # Compare two documents only with '_doc'

# (2)
POST {index}/_search
{
  ... # Use_includes, term for matching '_id' so this query could return only one document
}
8

The _id will not be the same.
I do know a unique value that I can use for the above queries.

9

As you know it’s simple to define same _id for different index if the only purpose is just comparing two documents are same by different indexing methods.