Hello everyone, has anyone idea to backfill the existing data from DynamoDB to Opensearch ?
I already set up the infrastructure that every time when DynamoDB updates, DDB-stream will trigger lambda. Then the lambda function will map data and save data in Opensearch.
I’m looking for a way that I can take advantage of the existing infrastructure, so that I don’t need to maintain two services which deal with the same thing (map data and save in Opensearch)
So far I found 1 way, and not sure if there’re some better ways: (if yes, would be highly appreciated to share with us)
Method: create an extra lambda and kinesis services (maybe Kinesis is not necessary here?)
- create a new simple lambda function which scans the entire table
- create Kinesis Data Stream, add Kinesis as a trigger to the existing lambda function
- the new simple lambda function puts record to Kinesis Data Stream after scanning the DDB table