Opensearch 1.3.3 - missing indexes breaks Anomaly Detection / Alerting Kibana functionality

OpenSearch
1

HI All,

I have a problem with opensearch 1.3.3. I have it installed as below.

  • Centos 7
  • Opensearch 1.3.3 / Opensearch-Dashboards 1.3.3
  • Plugins
alertingDashboards@1.3.3.0
anomalyDetectionDashboards@1.3.3.0
ganttChartDashboards@1.3.3.0
indexManagementDashboards@1.3.3.0
observabilityDashboards@1.3.3.0
queryWorkbenchDashboards@1.3.3.0
reportsDashboards@1.3.3.0
securityDashboards@1.3.3.0

If I click on either alerting or anomaly detection in kibab an error is produced saying that the index is not found as shown below. This seems to break both the anomaly detection and alerting features with KIbana.

  log   [13:49:36.320] [info][plugins-service] Plugin "visTypeXy" is disabled.
  log   [13:49:36.430] [info][plugins-system] Setting up [44] plugins: [usageCollection,opensearchDashboardsUsageCollection,opensearchDashboardsLegacy,mapsLegacy,alertingDashboards,share,opensearchUiShared,embeddable,legacyExport,expressions,data,home,console,apmOss,management,indexPatternManagement,advancedSettings,savedObjects,dashboard,visualizations,visTypeVega,visTypeTimeline,timeline,visTypeTable,visTypeMarkdown,tileMap,regionMap,inputControlVis,visualize,ganttChartDashboards,reportsDashboards,securityDashboards,anomalyDetectionDashboards,indexManagementDashboards,queryWorkbenchDashboards,charts,visTypeVislib,visTypeTimeseries,visTypeTagcloud,visTypeMetric,discover,savedObjectsManagement,observabilityDashboards,bfetch]
  log   [13:49:36.722] [info][savedobjects-service] Waiting until all OpenSearch nodes are compatible with OpenSearch Dashboards before starting saved objects migrations...
  log   [13:49:36.978] [info][savedobjects-service] Starting saved objects migrations
  log   [13:49:37.037] [info][plugins-system] Starting [44] plugins: [usageCollection,opensearchDashboardsUsageCollection,opensearchDashboardsLegacy,mapsLegacy,alertingDashboards,share,opensearchUiShared,embeddable,legacyExport,expressions,data,home,console,apmOss,management,indexPatternManagement,advancedSettings,savedObjects,dashboard,visualizations,visTypeVega,visTypeTimeline,timeline,visTypeTable,visTypeMarkdown,tileMap,regionMap,inputControlVis,visualize,ganttChartDashboards,reportsDashboards,securityDashboards,anomalyDetectionDashboards,indexManagementDashboards,queryWorkbenchDashboards,charts,visTypeVislib,visTypeTimeseries,visTypeTagcloud,visTypeMetric,discover,savedObjectsManagement,observabilityDashboards,bfetch]
  log   [13:49:37.404] [info][listening] Server running at http://opensearch.cyberkryption.local:5601
  log   [13:49:37.513] [info][server][OpenSearchDashboards][http] http server running at http://opensearch.cyberkryption.local:5601
Anomaly detector - Unable to search detectors { Error: [index_not_found_exception] no such index [.opendistro-anomaly-detectors], with { index=".opendistro-anomaly-detectors" & resource.id=".opendistro-anomaly-detectors" & resource.type="index_or_alias" & index_uuid="_na_" }
    at respond (/usr/share/opensearch-dashboards/node_modules/elasticsearch/src/lib/transport.js:349:15)
    at checkRespForFailure (/usr/share/opensearch-dashboards/node_modules/elasticsearch/src/lib/transport.js:306:7)
    at HttpConnector.<anonymous> (/usr/share/opensearch-dashboards/node_modules/elasticsearch/src/lib/connectors/http.js:173:7)
    at IncomingMessage.wrapper (/usr/share/opensearch-dashboards/node_modules/lodash/lodash.js:4991:19)
    at IncomingMessage.emit (events.js:203:15)
    at endReadableNT (_stream_readable.js:1145:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)
  status: 404,
  displayName: 'NotFound',
  message:
   '[index_not_found_exception] no such index [.opendistro-anomaly-detectors], with { index=".opendistro-anomaly-detectors" & resource.id=".opendistro-anomaly-detectors" & resource.type="index_or_alias" & index_uuid="_na_" }',
  path: '/_plugins/_anomaly_detection/detectors/_search',
  query: {},
  body:
   { error:
      { root_cause: [Array],
        type: 'index_not_found_exception',
        reason: 'no such index [.opendistro-anomaly-detectors]',
        index: '.opendistro-anomaly-detectors',
        'resource.id': '.opendistro-anomaly-detectors',
        'resource.type': 'index_or_alias',
        index_uuid: '_na_' },
     status: 404 },
  statusCode: 404,
  response:
   '{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index [.opendistro-anomaly-detectors]","index":".opendistro-anomaly-detectors","resource.id":".opendistro-anomaly-detectors","resource.type":"index_or_alias","index_uuid":"_na_"}],"type":"index_not_found_exception","reason":"no such index [.opendistro-anomaly-detectors]","index":".opendistro-anomaly-detectors","resource.id":".opendistro-anomaly-detectors","resource.type":"index_or_alias","index_uuid":"_na_"},"status":404}',
  toString: [Function],
  toJSON: [Function] }
Alerting - MonitorService - searchMonitor: { Error: [alerting_exception] Configured indices are not found: [.opendistro-alerting-config]
    at respond (/usr/share/opensearch-dashboards/node_modules/elasticsearch/src/lib/transport.js:349:15)
    at checkRespForFailure (/usr/share/opensearch-dashboards/node_modules/elasticsearch/src/lib/transport.js:306:7)
    at HttpConnector.<anonymous> (/usr/share/opensearch-dashboards/node_modules/elasticsearch/src/lib/connectors/http.js:173:7)
    at IncomingMessage.wrapper (/usr/share/opensearch-dashboards/node_modules/lodash/lodash.js:4991:19)
    at IncomingMessage.emit (events.js:203:15)
    at endReadableNT (_stream_readable.js:1145:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)
  status: 404,
  displayName: 'NotFound',
  message:
   '[alerting_exception] Configured indices are not found: [.opendistro-alerting-config]',
  path: '/_plugins/_alerting/monitors/_search',
  query: {},
  body:
   { error:
      { root_cause: [Array],
        type: 'alerting_exception',
        reason:
         'Configured indices are not found: [.opendistro-alerting-config]',
        caused_by: [Object] },
     status: 404 },
  statusCode: 404,
  response:
   '{"error":{"root_cause":[{"type":"alerting_exception","reason":"Configured indices are not found: [.opendistro-alerting-config]"}],"type":"alerting_exception","reason":"Configured indices are not found: [.opendistro-alerting-config]","caused_by":{"type":"exception","reason":"org.opensearch.index.IndexNotFoundException: no such index [.opendistro-alerting-config]"}},"status":404}',
  toString: [Function],
  toJSON: [Function] }

I have tried manually creating an index via the api but this is missing data structures within it and causes a search failure.

Could someone in the community tell me if there is a workaround or provide a link the index structure via api indices name/_settings endpoint for both the .opendistro-anomaly-detectors and .opendistro-alerting-config

Example
curl -XGET https://127.0.0.1:9200/.opendistro-anomaly-detectors/_settings?prety=true

Thanks in advance.