[DEBUG]why a lot of logs about retention leases

hi everyone!!
Recently, my opendistro is very unstable, all indexes had retention leases error.Would you know what’s happen and the detailed DEBUG ERROR is as follows:

[2021-03-25T00:00:05,117][DEBUG][o.e.i.s.ReplicationTracker] [aaa01] [.opendistro-ism-managed-index-history-2021.03.17-000166][0] no retention leases are expired from current retention leases [RetentionLeases{primaryTerm=6, version=64,
leases={peer_recovery/48sYCzT3TfGdPKMAtDQRjg=RetentionLease{id=‘peer_recovery/48sYCzT3TfGdPKMAtDQRjg’, retainingSequenceNumber=0, timestamp=1616590954938, source=‘peer recovery’}, peer_recovery/bYfsEWRwRPWg-AOLhFsUHg=RetentionLease{id=‘peer_recovery/bYfsEWRwRPWg-AOLhFsUHg’, retainingSequenceNumber=0, timestamp=1616590954938, source=‘peer recovery’}}}]
[2021-03-25T00:00:07,504][DEBUG][o.e.i.s.ReplicationTracker] [aaa01] [.opendistro-ism-managed-index-history-2021.03.11-000160][0] no retention leases are expired from current retention leases [RetentionLeases{primaryTerm=10, version=110,
leases={peer_recovery/bYfsEWRwRPWg-AOLhFsUHg=RetentionLease{id=‘peer_recovery/bYfsEWRwRPWg-AOLhFsUHg’, retainingSequenceNumber=0, timestamp=1616587627280, source=‘peer recovery’}}}]
[2021-03-25T00:00:10,311][DEBUG][o.e.i.s.ReplicationTracker] [aaa01] [.opendistro-alerting-alerts][0] no retention leases are expired from current retention leases [RetentionLeases{primaryTerm=19, version=636,
leases={peer_recovery/z3EzFRpYRFODJawOEsTeJw=RetentionLease{id=‘peer_recovery/z3EzFRpYRFODJawOEsTeJw’, retainingSequenceNumber=107, timestamp=1616598280071, source=‘peer recovery’}, peer_recovery/bYfsEWRwRPWg-AOLhFsUHg=RetentionLease{id=‘peer_recovery/bYfsEWRwRPWg-AOLhFsUHg’, retainingSequenceNumber=107, timestamp=1616598280071, source=‘peer recovery’}}}]
[2021-03-25T00:00:11,182][DEBUG][o.e.i.s.ReplicationTracker] [aaa01] [index01][1] no retention leases are expired from current retention leases [RetentionLeases{primaryTerm=21, version=5208,
leases={peer_recovery/48sYCzT3TfGdPKMAtDQRjg=RetentionLease{id=‘peer_recovery/48sYCzT3TfGdPKMAtDQRjg’, retainingSequenceNumber=41071641, timestamp=1616588140798, source=‘peer recovery’}, peer_recovery/bYfsEWRwRPWg-AOLhFsUHg=RetentionLease{id=‘peer_recovery/bYfsEWRwRPWg-AOLhFsUHg’, retainingSequenceNumber=41071641, timestamp=1616588140798, source=‘peer recovery’}}}]
[2021-03-25T00:00:11,287][DEBUG][o.e.i.s.ReplicationTracker] [aaa01] [index02][0] no retention leases are expired from current retention leases [RetentionLeases{primaryTerm=47, version=1360,
leases={peer_recovery/48sYCzT3TfGdPKMAtDQRjg=RetentionLease{id=‘peer_recovery/48sYCzT3TfGdPKMAtDQRjg’, retainingSequenceNumber=50891, timestamp=1616591291142, source=‘peer recovery’}, peer_recovery/bYfsEWRwRPWg-AOLhFsUHg=RetentionLease{id=‘peer_recovery/bYfsEWRwRPWg-AOLhFsUHg’, retainingSequenceNumber=50891, timestamp=1616591291142, source=‘peer recovery’}}}]
[2021-03-25T00:00:11,397][DEBUG][o.e.i.s.ReplicationTracker] [aaa01] [index03][0] no retention leases are expired from current retention leases [RetentionLeases{primaryTerm=6, version=1037,
leases={peer_recovery/z3EzFRpYRFODJawOEsTeJw=RetentionLease{id=‘peer_recovery/z3EzFRpYRFODJawOEsTeJw’, retainingSequenceNumber=860529, timestamp=1616587631082, source=‘peer recovery’}, peer_recovery/bYfsEWRwRPWg-AOLhFsUHg=RetentionLease{id=‘peer_recovery/bYfsEWRwRPWg-AOLhFsUHg’, retainingSequenceNumber=860529, timestamp=1616587631082, source=‘peer recovery’}}}]

Please help me this solution ! thanks so much.

@Cindy What version are you using?

Hi~ My version is elasticsearch 7.10.2, opendistroforelasticsearch-1.13.1.
Thanks a lot.

I’m not an expert on this particular aspect of Elasticsearch (perhaps others can chime in), but it seems like this has replication and loss of communication with a shard. The ‘peer recovery’ part is the giveaway. Seems to be lots of information floating around about that. Hope it gives you a place to start looking.

@searchymcsearchface Thanks to you!