Backfill the existing data from DynamoDB to Opensearch

Hello everyone, has anyone idea to backfill the existing data from DynamoDB to Opensearch ?

I already set up the infrastructure that every time when DynamoDB updates, DDB-stream will trigger lambda. Then the lambda function will map data and save data in Opensearch.

I’m looking for a way that I can take advantage of the existing infrastructure, so that I don’t need to maintain two services which deal with the same thing (map data and save in Opensearch)

So far I found 1 way, and not sure if there’re some better ways: (if yes, would be highly appreciated to share with us)

Method: create an extra lambda and kinesis services (maybe Kinesis is not necessary here?)

  • create a new simple lambda function which scans the entire table
  • create Kinesis Data Stream, add Kinesis as a trigger to the existing lambda function
  • the new simple lambda function puts record to Kinesis Data Stream after scanning the DDB table