Snapshot policy fails

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
OS: 2.10
OD: 2.10

Describe the issue:
The snapshot policy keep on failing with partial status, but when i run snapshot manually it works, what could be the cause.



“description”: “Daily snapshot policy”,
“creation”: {
“schedule”: {
“cron”: {
“expression”: “0 6 * * *”,
“timezone”: “UTC”
“deletion”: {
“schedule”: {
“cron”: {
“expression”: “0 1 * * ",
“timezone”: “America/Los_Angeles”
“condition”: {
“max_age”: “3d”,
“max_count”: 21,
“min_count”: 7
“time_limit”: “1h”
“snapshot_config”: {
“date_format”: “yyyy-MM-dd-HH:mm”,
“timezone”: “America/Los_Angeles”,
“indices”: "device-
“repository”: “gcs-repo”,
“ignore_unavailable”: “true”,
“include_global_state”: “false”,
“partial”: “true”,
“metadata”: {
“any_key”: “any_value”

Relevant Logs or Screenshots:

error status

“index”: “device-2023-12-27”,
“index_uuid”: “device-2023-12-27”,
“shard_id”: 3,
“reason”: “AccessControlException[access denied ("java.lang.RuntimePermission" "accessDeclaredMembers")]”,
“node_id”: “9qXQNx1IQey2bQmemS5t6A”,

can someone please help me on this.

Hey @arun_udaiyar

Looks like you have a permission issue.

Not sure but maybe CRON doesnt have access. perhaps check the permission on the repo where the sanpshots are located.

Thanks @Gsmitt
After adding few more permission at bucket side its works fine.

but the detetion is failing without message, any advice on this

Hey @arun_udaiyar

I noticed your CRON timezone is "“timezone”: “UTC”".
But you have snapshots timezone "“timezone”: “America/Los_Angeles”,"

I would assume for deletion either its still might be a permission issue or Date/Time. Not 100% sure thou.

EDIT: Do you see anything in the logs that pertain to this issue of not deleteing your snapshot?

@arun_udaiyar Can you explain what you had to change on the bucket side to enable this permission? I am having the same issue where snapshot policies are failing to create/delete and there is no information printed to the log.