Approximate kNN total hits inconsistent

Hi, we have an implementation of Approximate kNN search with OpenSeaerch 1.3, with 768-dimension embedding vector in the ANN search. We observed that the total hits are not consistent when running the same query multiple times, or paginate using from and size. Sometimes the hits/total could vary by 100s when the total is thousands. We use k=100 and there are 5 shards. Any idea to make the total not changing?