# Unable to see prometheus data source index on the dashboard

**URL:** <https://forum.opensearch.org/t/unable-to-see-prometheus-data-source-index-on-the-dashboard/18578>\
**Category:** OpenSearch\
**Tags:** troubleshoot\
**Created:** [March 26, 2024, 7:25pm UTC](https://forum.opensearch.org/t/unable-to-see-prometheus-data-source-index-on-the-dashboard/18578 "2024-03-26T19:25:22Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![Akshat](https://avatars.discourse-cdn.com/v4/letter/a/22d042/32.png) [@Akshat](https://forum.opensearch.org/u/Akshat)\
**Post date:** [March 26, 2024, 7:25pm UTC](https://forum.opensearch.org/t/unable-to-see-prometheus-data-source-index-on-the-dashboard/18578/1 "2024-03-26T19:25:22Z")

</div>

**Versions** (relevant - OpenSearch/Dashboard/Server OS/Browser):

opensearch 2.12, ubuntu, chrome

**Describe the issue** :

The indexing on the data store for Prometheus doesn’t seem to work. is there any config that might be missing. I am using helm chart to install the opensearch and dashboard.

**Relevant Logs or Screenshots** :

 ![Screenshot 2024-03-26 at 12.24.57 PM](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/0/05399c2e79911dbbceb0da93cc2cd916375b8fa4.png)

---

<div class="post-metadata">

**Author:** ![Gsmitt](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/gsmitt/32/1718_2.png) [@Gsmitt](https://forum.opensearch.org/u/Gsmitt)\
**Post date:** [March 29, 2024, 3:08am UTC](https://forum.opensearch.org/t/unable-to-see-prometheus-data-source-index-on-the-dashboard/18578/2 "2024-03-29T03:08:23Z")

</div>

Hey @Akshat

Testing this also and kind of have the same issue I think.

I’m running OS/OSD-v2.12.0 on Ubuntu22.0.4

I connected to Prometheus in the Data source section. no issues shown.

 ![image](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/b/b9d7c23ea4dc52e96b72685ef1d129593bdad0e1.png)

Checked my datasource index set…

 ![image](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/c/c1089a5006ab66d4747dc0636472055efe26d07f.png)

On the left Pane Observability → Metrics I see this

 ![image](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/b/bd99b19b1cfedcfc7ddd9f95e6cb3ed4c1214ed2.png)

If I click on any one of those metrics I see this…

 ![image](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/f/f8eb725440c8b0fba088ae4fdc281309d3fe8b13.png)

Is this Issue you are having also?

---

<div class="post-metadata">

**Author:** ![pablo](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/pablo/32/10363_2.png) [@pablo](https://forum.opensearch.org/u/pablo)\
**Post date:** [March 29, 2024, 11:34am UTC](https://forum.opensearch.org/t/unable-to-see-prometheus-data-source-index-on-the-dashboard/18578/3 "2024-03-29T11:34:29Z")

</div>

@Akshat The provided screenshot shows only index patterns. An index pattern is not the same as an index.

You can list indices by running the following AIP call in the Dev Tools.

```auto
GET _cat/indices

```

The index pattern must be created manually.  
Take a look at one of the screenshots provided by @Gsmitt

---

<div class="post-metadata">

**Author:** ![Akshat](https://avatars.discourse-cdn.com/v4/letter/a/22d042/32.png) [@Akshat](https://forum.opensearch.org/u/Akshat)\
**Post date:** [March 29, 2024, 5:34pm UTC](https://forum.opensearch.org/t/unable-to-see-prometheus-data-source-index-on-the-dashboard/18578/4 "2024-03-29T17:34:46Z")

</div>

I used the dev tools and checked the ql-datasources in the list. The issue is that i can see the metrics visualization on the observability-\>metrics page. when I try to save the visualization and add to the dashboard, it shows me errors like below:  
`Failed to parse query due to offending symbol [ceph_bluestore_pricache:] at: 'source = cephprometheus.ceph_bluestore_pricache:' <--- HERE... More details: Expecting tokens in {'SEARCH', 'DESCRIBE', 'SHOW', 'FROM', 'WHERE', 'FIELDS', 'RENAME', 'STATS', 'DEDUP', 'SORT', 'EVAL', 'HEAD', 'TOP', 'RARE', 'PARSE', 'METHOD', 'REGEX', 'PUNCT', 'GROK', 'PATTERN', 'PATTERNS', 'NEW_FIELD', 'KMEANS', 'AD', 'ML', 'SOURCE', 'INDEX', 'D', 'DESC', 'DATASOURCES', 'SORTBY', 'STR', 'IP', 'NUM', 'KEEPEMPTY', 'CONSECUTIVE', 'DEDUP_SPLITVALUES', 'PARTITIONS', 'ALLNUM', 'DELIM', 'CENTROIDS', 'ITERATIONS', 'DISTANCE_TYPE', 'NUMBER_OF_TREES', 'SHINGLE_SIZE', 'SAMPLE_SIZE', 'OUTPUT_AFTER', 'TIME_DECAY', 'ANOMALY_RATE', 'CATEGORY_FIELD', 'TIME_FIELD', 'TIME_ZONE', 'TRAINING_DATA_SIZE', 'ANOMALY_SCORE_THRESHOLD', 'NOT', 'TRUE', 'FALSE', 'CONVERT_TZ', 'DATETIME', 'DAY', 'DAY_HOUR', 'DAY_MICROSECOND', 'DAY_MINUTE', 'DAY_OF_YEAR', 'DAY_SECOND', 'HOUR', 'HOUR_MICROSECOND', 'HOUR_MINUTE', 'HOUR_OF_DAY', 'HOUR_SECOND', 'INTERVAL', 'MICROSECOND', 'MILLISECOND', 'MINUTE', 'MINUTE_MICROSECOND', 'MINUTE_OF_DAY', 'MINUTE_OF_HOUR', 'MINUTE_SECOND', 'MONTH', 'MONTH_OF_YEAR', 'QUARTER', 'SECOND', 'SECOND_MICROSECOND', 'SECOND_OF_MINUTE', 'WEEK', 'WEEK_OF_YEAR', 'YEAR', 'YEAR_MONTH', '.', '+', '-', '(', '`', ‘AVG’, ‘COUNT’, ‘DISTINCT\_COUNT’, ‘ESTDC’, ‘ESTDC\_ERROR’, ‘MAX’, ‘MEAN’, ‘MEDIAN’, ‘MIN’, ‘MODE’, ‘RANGE’, ‘STDEV’, ‘STDEVP’, ‘SUM’, ‘SUMSQ’, ‘VAR\_SAMP’, ‘VAR\_POP’, ‘STDDEV\_SAMP’, ‘STDDEV\_POP’, ‘PERCENTILE’, ‘TAKE’, ‘FIRST’, ‘LAST’, ‘LIST’, ‘VALUES’, ‘EARLIEST’, ‘EARLIEST\_TIME’, ‘LATEST’, ‘LATEST\_TIME’, ‘PER\_DAY’, ‘PER\_HOUR’, ‘PER\_MINUTE’, ‘PER\_SECOND’, ‘RATE’, ‘SPARKLINE’, ‘C’, ‘DC’, ‘ABS’, ‘CBRT’, ‘CEIL’, ‘CEILING’, ‘CONV’, ‘CRC32’, ‘E’, ‘EXP’, ‘FLOOR’, ‘LN’, ‘LOG’, ‘LOG10’, ‘LOG2’, ‘MOD’, ‘PI’, ‘POSITION’, ‘POW’, ‘POWER’, ‘RAND’, ‘ROUND’, ‘SIGN’, ‘SQRT’, ‘TRUNCATE’, ‘ACOS’, ‘ASIN’, ‘ATAN’, ‘ATAN2’, ‘COS’, ‘COT’, ‘DEGREES’, ‘RADIANS’, ‘SIN’, ‘TAN’, ‘ADDDATE’, ‘ADDTIME’, ‘CURDATE’, ‘CURRENT\_DATE’, ‘CURRENT\_TIME’, ‘CURRENT\_TIMESTAMP’, ‘CURTIME’, ‘DATE’, ‘DATEDIFF’, ‘DATE\_ADD’, ‘DATE\_FORMAT’, ‘DATE\_SUB’, ‘DAYNAME’, ‘DAYOFMONTH’, ‘DAYOFWEEK’, ‘DAYOFYEAR’, ‘DAY\_OF\_MONTH’, ‘DAY\_OF\_WEEK’, ‘EXTRACT’, ‘FROM\_DAYS’, ‘FROM\_UNIXTIME’, ‘GET\_FORMAT’, ‘LAST\_DAY’, ‘LOCALTIME’, ‘LOCALTIMESTAMP’, ‘MAKEDATE’, ‘MAKETIME’, ‘MONTHNAME’, ‘NOW’, ‘PERIOD\_ADD’, ‘PERIOD\_DIFF’, ‘SEC\_TO\_TIME’, ‘STR\_TO\_DATE’, ‘SUBDATE’, ‘SUBTIME’, ‘SYSDATE’, ‘TIME’, ‘TIMEDIFF’, ‘TIMESTAMP’, ‘TIMESTAMPADD’, ‘TIMESTAMPDIFF’, ‘TIME\_FORMAT’, ‘TIME\_TO\_SEC’, ‘TO\_DAYS’, ‘TO\_SECONDS’, ‘UNIX\_TIMESTAMP’, ‘UTC\_DATE’, ‘UTC\_TIME’, ‘UTC\_TIMESTAMP’, ‘WEEKDAY’, ‘YEARWEEK’, ‘SUBSTR’, ‘SUBSTRING’, ‘LTRIM’, ‘RTRIM’, ‘TRIM’, ‘LOWER’, ‘UPPER’, ‘CONCAT’, ‘CONCAT\_WS’, ‘LENGTH’, ‘STRCMP’, ‘RIGHT’, ‘LEFT’, ‘ASCII’, ‘LOCATE’, ‘REPLACE’, ‘REVERSE’, ‘CAST’, ‘LIKE’, ‘ISNULL’, ‘ISNOTNULL’, ‘IFNULL’, ‘NULLIF’, ‘IF’, ‘TYPEOF’, ‘MATCH’, ‘MATCH\_PHRASE’, ‘MATCH\_PHRASE\_PREFIX’, ‘MATCH\_BOOL\_PREFIX’, ‘SIMPLE\_QUERY\_STRING’, ‘MULTI\_MATCH’, ‘QUERY\_STRING’, ‘ALLOW\_LEADING\_WILDCARD’, ‘ANALYZE\_WILDCARD’, ‘ANALYZER’, ‘AUTO\_GENERATE\_SYNONYMS\_PHRASE\_QUERY’, ‘BOOST’, ‘CUTOFF\_FREQUENCY’, ‘DEFAULT\_FIELD’, ‘DEFAULT\_OPERATOR’, ‘ENABLE\_POSITION\_INCREMENTS’, ‘ESCAPE’, ‘FLAGS’, ‘FUZZY\_MAX\_EXPANSIONS’, ‘FUZZY\_PREFIX\_LENGTH’, ‘FUZZY\_TRANSPOSITIONS’, ‘FUZZY\_REWRITE’, ‘FUZZINESS’, ‘LENIENT’, ‘LOW\_FREQ\_OPERATOR’, ‘MAX\_DETERMINIZED\_STATES’, ‘MAX\_EXPANSIONS’, ‘MINIMUM\_SHOULD\_MATCH’, ‘OPERATOR’, ‘PHRASE\_SLOP’, ‘PREFIX\_LENGTH’, ‘QUOTE\_ANALYZER’, ‘QUOTE\_FIELD\_SUFFIX’, ‘REWRITE’, ‘SLOP’, ‘TIE\_BREAKER’, ‘TYPE’, ‘ZERO\_TERMS\_QUERY’, ‘SPAN’, ‘MS’, ‘S’, ‘M’, ‘H’, ‘W’, ‘Q’, ‘Y’, ID, INTEGER\_LITERAL, DECIMAL\_LITERAL, DQUOTA\_STRING, SQUOTA\_STRING, BQUOTA\_STRING}`

---

<div class="post-metadata">

**Author:** ![Akshat](https://avatars.discourse-cdn.com/v4/letter/a/22d042/32.png) [@Akshat](https://forum.opensearch.org/u/Akshat)\
**Post date:** [March 29, 2024, 5:40pm UTC](https://forum.opensearch.org/t/unable-to-see-prometheus-data-source-index-on-the-dashboard/18578/5 "2024-03-29T17:40:23Z")

</div>

Did you create the data source from the Dashboard UI or with API endpoint? Lets see if this solves the issue [Opensearch dashboard goes blank with Json parse error - #6 by Akshat](https://forum.opensearch.org/t/opensearch-dashboard-goes-blank-with-json-parse-error/18383/6)

---

<div class="post-metadata">

**Author:** ![Gsmitt](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/gsmitt/32/1718_2.png) [@Gsmitt](https://forum.opensearch.org/u/Gsmitt)\
**Post date:** [March 29, 2024, 9:05pm UTC](https://forum.opensearch.org/t/unable-to-see-prometheus-data-source-index-on-the-dashboard/18578/6 "2024-03-29T21:05:07Z")

</div>

> [@Akshat](#):
>
> Did you create the data source from the Dashboard UI or with API endpoint?

I did both. Or I should say I tried both. right now I’m just using API.

EDIT: I do have key made, I saw that when I was troubleshooting.

---

<div class="post-metadata">

**Author:** ![Akshat](https://avatars.discourse-cdn.com/v4/letter/a/22d042/32.png) [@Akshat](https://forum.opensearch.org/u/Akshat)\
**Post date:** [April 4, 2024, 4:08pm UTC](https://forum.opensearch.org/t/unable-to-see-prometheus-data-source-index-on-the-dashboard/18578/7 "2024-04-04T16:08:09Z")

</div>

I have the same issue showing up again. I tried 2.11.1 and used default admin admin user and added the master key and it was working. Now when I upgraded 2.12.0 the page goes blank again. I see that there is a new version out yesterday 2.13 and I am going to give this 1 more go. I am getting out of ideas and I hope someone from the community or the contributors look into this soon.

---

<div class="post-metadata">

**Author:** ![Gsmitt](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/gsmitt/32/1718_2.png) [@Gsmitt](https://forum.opensearch.org/u/Gsmitt)\
**Post date:** [April 4, 2024, 9:24pm UTC](https://forum.opensearch.org/t/unable-to-see-prometheus-data-source-index-on-the-dashboard/18578/8 "2024-04-04T21:24:23Z")

</div>

Hey @Akshat

Yeah there is a issue, I posted in Slack and GitHub.

> <https://github.com/opensearch-project/sql/issues/2541>
>
> \*\*What is the bug?\*\*
> With a Prometheus data source added the Observability -\> M…etrics will show the available metrics from this data source. However when a metric is selected the page becomes mostly empty and errors are thrown in the browser console.
> 
> \*\*How can one reproduce the bug?\*\*
> Steps to reproduce the behavior:
> 1. Create a Prometheus data source (below ran from Dev Tools)
> \`\`\`
> POST \_plugins/\_query/\_datasources
> {
> "name": "prometheus",
> "connector": "prometheus",
> "properties": {
> "prometheus.uri": "https://prometheus.url.com/",
> "prometheus.auth.type": "basicauth",
> "prometheus.auth.username": "username",
> "prometheus.auth.password": "password"
> },
> "allowedRoles": \[\]
> }
> \`\`\`
> 2. Navigate to Observability -\> Metrics
> 3. Wait for the list of available metrics to populate
> !\[image\](https://github.com/opensearch-project/sql/assets/101164522/d1fa132f-8674-42f9-89ce-f93058bc7bee)
> 
> 4. Click on any metric, a moment later the page clears
> !\[image\](https://github.com/opensearch-project/sql/assets/101164522/fcfaa8b5-8020-4ff7-8c2f-591e5cd6cf1c)
> 
> When the page clears the following errors present in the browser console
> !\[image\](https://github.com/opensearch-project/sql/assets/101164522/003814ac-193f-43ab-9b3e-68a286e84f25)
> 
> \*\*What is the expected behavior?\*\*
> When a metric is selected from Observability -\> Metrics I expect to see a summary of the metric similar to the screenshots in https://opensearch.org/docs/latest/observing-your-data/prometheusmetrics/
> 
> \*\*What is your host/environment?\*\*
> - OS: Docker deployed on CentOS7
> - Version 2.12.0
> - Browser: Chrome version 122.0.6261.95 (Official Build) (64-bit)
> - Relevant Plugins:
> opensearch-sql version 2.12.0.0
> opensearch-observability version 2.12.0.0

---

<div class="post-metadata">

**Author:** ![Gsmitt](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/gsmitt/32/1718_2.png) [@Gsmitt](https://forum.opensearch.org/u/Gsmitt)\
**Post date:** [April 5, 2024, 2:46am UTC](https://forum.opensearch.org/t/unable-to-see-prometheus-data-source-index-on-the-dashboard/18578/9 "2024-04-05T02:46:53Z")

</div>

Hey @Akshat

I tried version 2.13 same issue

> [@Akshat](#):
>
> am getting out of ideas and I hope someone from the community or the contributors look into this soon.

On OpenSearch [Slack.com](http://Slack.com) I was just talking to someone about this. I think this maybe fix in version 2.14.

You can see it here…

> **[Slack is your productivity platform](https://opensearch.slack.com/?redir=%2Farchives%2FC01QENNTGUD%2Fp1711760080485209%3Fname%3DC01QENNTGUD%26perma%3D1711760080485209)**
>
> Slack is a new way to communicate with your team. It’s faster, better organized, and more secure than email.

---

<div class="post-metadata">

**Author:** ![Akshat](https://avatars.discourse-cdn.com/v4/letter/a/22d042/32.png) [@Akshat](https://forum.opensearch.org/u/Akshat)\
**Post date:** [April 5, 2024, 4:23pm UTC](https://forum.opensearch.org/t/unable-to-see-prometheus-data-source-index-on-the-dashboard/18578/10 "2024-04-05T16:23:02Z")

</div>

Do any one over there has idea about when this bug was introduced? it would be great to get an invite for the slack channel.

---

<div class="post-metadata">

**Author:** ![Gsmitt](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/gsmitt/32/1718_2.png) [@Gsmitt](https://forum.opensearch.org/u/Gsmitt)\
**Post date:** [April 6, 2024, 12:59am UTC](https://forum.opensearch.org/t/unable-to-see-prometheus-data-source-index-on-the-dashboard/18578/11 "2024-04-06T00:59:50Z")

</div>

> [@Akshat](#):
>
> it would be great to get an invite for the slack channel.

try this

> **[Slack workspace](https://opensearch.org/slack.html)**
>
> OpenSearch is a community-driven, Apache 2.0-licensed open source search and analytics suite that makes it easy to ingest, search, visualize, and analyze data.

---

<div class="post-metadata">

**Author:** ![Akshat](https://avatars.discourse-cdn.com/v4/letter/a/22d042/32.png) [@Akshat](https://forum.opensearch.org/u/Akshat)\
**Post date:** [April 15, 2024, 4:55pm UTC](https://forum.opensearch.org/t/unable-to-see-prometheus-data-source-index-on-the-dashboard/18578/12 "2024-04-15T16:55:05Z")

</div>

do we have any idea if this will be fixed on 2.14? I was reading the channel and looks like there is going to be a new release soon.

---

<div class="post-metadata">

**Author:** ![Gsmitt](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/gsmitt/32/1718_2.png) [@Gsmitt](https://forum.opensearch.org/u/Gsmitt)\
**Post date:** [April 15, 2024, 8:56pm UTC](https://forum.opensearch.org/t/unable-to-see-prometheus-data-source-index-on-the-dashboard/18578/13 "2024-04-15T20:56:57Z")

</div>

Not sure, you can check GitHub. I think they have a road map.

---

<div class="post-metadata">

**Author:** ![Gsmitt](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/gsmitt/32/1718_2.png) [@Gsmitt](https://forum.opensearch.org/u/Gsmitt)\
**Post date:** [April 15, 2024, 9:54pm UTC](https://forum.opensearch.org/t/unable-to-see-prometheus-data-source-index-on-the-dashboard/18578/14 "2024-04-15T21:54:23Z")

</div>

Check here…

> **[Build software better, together](https://github.com/orgs/opensearch-project/projects/1)**
>
> GitHub is where people build software. More than 100 million people use GitHub to discover, fork, and contribute to over 420 million projects.