Deduplication of search results

OpenDistro
,
1

I have a few indices that contain fields like the following:

"download_link" : [
  {
    "link" : "http://thingiverse.com/download-url-1",
    "domain" : "thingiverse.com"
  },
  {
    "link" : "http://thingiverse.com/download-url-2",
    "domain" : "thingiverse.com"
  },
  {
    "link" : "http://thingiverse.com/download-url-3",
    "domain" : "thingiverse.com"
  }
],

The problem is, there are several documents that include that same field and data as they are scraped from sites and when I search for a term, all the documents that contain the exact same data are returned back. Is there a way I can remove the duplicates from the results? Is there a better way to structure the documents so that removing the duplicates would be easier? I need to re-index the documents so if I need to alter them, now’s the time.

Thanks!