Document fields do not always get *.keyword entries

Open Source Elasticsearch and Kibana
1

Hi

I am logging my exceptions and adding my strack trace to as error.stack_trace when I display this I use error.stack_trace.keyword. I can see my old entries fine but I can not see any new stack_traces. If I use the discover functionality I can see them all under error.stack_trace but not under error.stack_trace.keyword. Is there something that I need to do to make sure my string entries gets “keyworded” automatically?

2

You can check the mappings of the index to see if the field error.stack_trace.keyword exists, you may add the field’s mapping to the index template(if not exist you can create one) to make sure that every new index contains that field.

3

That does not seem to be it. It seems that some of the logs with error.stack_trace get properly handled and get the error.stack_trace.keyword entry but not all.

4

You mean that in Discover you want to see the value of the .keyword subfield? I don’t think you can do that normally, because subfields aren’t stored in _source, you only get the parent field (in your case, error.stack_trace stored.

Though the API, you can get the contents of the .keyword subfield by default if you ask for it in docvalue_fields.

5

Not quite. I mean that when I filter in the Discover view on error.stack_trace.message not everything from error.stack_trace shows. Especially newer message doesn’t have the .message.

6

I don’t quite get how your mapping looks like. Can you share a relevant snippet?

7

Hi again! Sorry for the incredibly late response, have been busy with other parts of this project.

I think the mapping looks like it should.

            "stack_trace": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },

When I search in Discover for error.stack_trace.keyword: exists I get 2 hits vs 74 hits for error.stack_trace: exists

8

No worries about the delay, it’s all best-effort here :slight_smile:

I think it’s because of ignore_above. If your string is more than 256 characters (and most stacktraces are), it gets ignored (i.e. not indexed). At least that’s my understanding. So when you search the keyword you don’t get anything.

But wait, exists is a keyword that you’re searching in the stacktrace? In that case you want to search in the text field, because that is tokenized. The keyword field is indexed as one big token.

If you want to search for docs that have any value in a field, you’d search for field:*, but I still think you wouldn’t get documents where the value is ignored, I don’t now for sure.

9

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.