# Derivative Features for Anomaly Detection Plugins

**URL:** https://forum.opensearch.org/t/derivative-features-for-anomaly-detection-plugins/14577
**Category:** Machine Learning
**Created:** [June 6, 2023, 9:06pm UTC](https://forum.opensearch.org/t/derivative-features-for-anomaly-detection-plugins/14577 "2023-06-06T21:06:03Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![ridingbikes](https://avatars.discourse-cdn.com/v4/letter/r/87869e/32.png) [@ridingbikes](https://forum.opensearch.org/u/ridingbikes)
#### Post date: [June 6, 2023, 9:06pm UTC](https://forum.opensearch.org/t/derivative-features-for-anomaly-detection-plugins/14577/1 "2023-06-06T21:06:03Z")

</div>

**Versions** (relevant - OpenSearch/Dashboard/Server OS/Browser):  
1.3.2

**Describe the issue** :  
I am trying to create an anomaly detector that generates anomalies from a derivative of aggregated counts for a specific field in my index. I am using an ElasticSearch blog post as reference ([Custom Elasticsearch Aggregations for Machine Learning Jobs | Elastic Blog](https://www.elastic.co/blog/custom-elasticsearch-aggregations-for-machine-learning-jobs)). I am able to create aggregations with the following expression. However, whenever I attempt to move this expression (without the outer-level “aggs” key) to the Feature’s Custom Expression box within the Anomaly Detector interface, I keep getting an error. I’m not sure what to make of this error or what my actual issue is. Any ideas? Thanks in advance for your help.

{  
“aggs”: {  
“changes\_in\_count”: {  
“date\_histogram”: {  
“field”: “DateTime”,  
“interval”: “month”  
},  
“aggs”: {  
“DateTime”: {  
“max”: {  
“field”: “DateTime”,  
}  
},  
“doc\_version\_count”: {  
“value\_count”: {  
“field”: “Id.keyword”  
}  
},  
“deriv”: {  
“derivative”: {  
“buckets\_path”: “\_count”  
}  
}  
}  
}  
}  
}

**Relevant Logs or Screenshots** :

 ![Screenshot 2023-06-06 at 2.03.07 PM](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/2/2ce477679f3635e4e8af49493ce0f35911694257.png)

---

<div class="post-metadata">

### Author: ![kaituo](https://avatars.discourse-cdn.com/v4/letter/k/e8c25b/32.png) [@kaituo](https://forum.opensearch.org/u/kaituo)
#### Post date: [June 12, 2023, 10:21pm UTC](https://forum.opensearch.org/t/derivative-features-for-anomaly-detection-plugins/14577/2 "2023-06-12T22:21:18Z")

</div>

I’d like to bring to your attention that the configuration format in OpenSearch’s Anomaly Detection differs from that of Elastic’s Machine Learning. In OpenSearch, we employ the concept of a “detector” instead of “job” and “data feed” used in Elastic’s ML.

For instance, consider the following Elastic’s ML aggregation:

```auto
"aggregations": {
    "buckets": {
      "date_histogram": {
        "field": "@timestamp",
        "interval": "5m",
        "time_zone": "UTC"
      },
      "aggregations": {
        "@timestamp": {
          "max": {
            "field": "@timestamp"
          }
        },
        "orders": {
          "sum": {
            "field": "events_per_min"
          }
        },
        "orders_deriv": {
          "derivative": {
            "buckets_path": "orders"
          }
        }
      }
    }
  }

```

The above can be translated to the following detector settings in OpenSearch:

```auto
POST _plugins/_anomaly_detection/detectors
{
	"name": "orders_deriv",
	"description": "Derivative of Order Volume",
	"time_field": "@timestamp",
	"indices": [
		"it_ops_kpi-2017"
	],
	"feature_attributes": [{
		"feature_name": "orders sum",
		"feature_enabled": true,
		"aggregation_query": {
			"orders": {
				"sum": {
					"field": "events_per_min"
				}
			}
		}
	}],
	"detection_interval": {
		"period": {
			"interval": 5,
			"unit": "MINUTES"
		}
	}
}

```

We encourage you to reference the OpenSearch documentation at [OpenSearch Anomaly Detection](https://opensearch.org/docs/latest/observing-your-data/ad/index/) and [Amazon OpenSearch Service](https://aws.amazon.com/blogs/big-data/detect-anomalies-on-one-million-unique-entities-with-amazon-opensearch-service/) to better understand and leverage the power of OpenSearch’s anomaly detection capabilities.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/3/33547ea01a5b12dcca2958411d3edd97ae2ea8c1.png) [@system](https://forum.opensearch.org/u/system)
#### Post date: [August 11, 2023, 10:21pm UTC](https://forum.opensearch.org/t/derivative-features-for-anomaly-detection-plugins/14577/3 "2023-08-11T22:21:32Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.
