Hello,
Im using the opensearch-data-prepper-jdk-2.8.0 version
I have dataprepper up and running wiht a pipeline.
The issue is when I use this grok option that is the grok_when it doesn’t run. Meaning after restart daraprepper it gives this error
JAVA_HOME is set to /opt/opensearch/data-prepper/openjdk/jdk-17.0.10+7
2024-07-10T13:48:47,317 [main] INFO org.opensearch.dataprepper.DataPrepperArgumentConfiguration - Command line args: /opt/opensearch/data-prepper/pipelines,/opt/opensearch/data-prepper/config/data-prepper-config.yaml
2024-07-10T13:48:47,408 [main] WARN org.springframework.context.support.AbstractApplicationContext - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name ‘dataPrepper’ defined in URL [jar:file:/opt/opensearch/data-prepper/lib/data-prepper-core-2.8.0.jar!/org/opensearch/dataprepper/DataPrepper.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name ‘pipelineParser’ defined in org.opensearch.dataprepper.parser.config.PipelineParserConfiguration: Unsatisfied dependency expressed through method ‘pipelineParser’ parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name ‘pipelinesDataFlowModel’ defined in org.opensearch.dataprepper.parser.config.PipelineParserConfiguration: Unsatisfied dependency expressed through method ‘pipelinesDataFlowModel’ parameter 1; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘preTransformedDataFlowModel’ defined in org.opensearch.dataprepper.parser.config.PipelineParserConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.opensearch.dataprepper.model.configuration.PipelinesDataFlowModel]: Factory method ‘preTransformedDataFlowModel’ threw exception; nested exception is org.opensearch.dataprepper.pipeline.parser.ParseException: Failed to parse the configuration
working configurations:
log-pipeline:
source:
http:
port: 8080
ssl: false
processor:
- grok:
match:
request_url: [‘%{WORD:http_method} (?:%{URIPATH:request_endpoint}(?:?%{GREEDYDATA:request_params})?) HTTP/%{NUMBER:client_version}’]
- parse_json:
source: json
- convert_entry_type:
key: “request_status”
type: “integer”
- convert_entry_type:
key: “upstream_status”
type: “integer”
- convert_entry_type:
key: “bytes_sent”
type: “integer”
- convert_entry_type:
key: “request_time”
type: “double”
- convert_entry_type:
key: “upstream_connect_time”
type: “double”
- convert_entry_type:
key: “upstream_header_time”
type: “double”
- convert_entry_type:
key: “upstream_response_time”
type: “double”
- convert_entry_type:
key: “connections_active”
type: “integer”
- delete_entries:
with_keys: [“http_method”]
sink:
- stdout: {}
Bad configuration
log-pipeline:
source:
http:
port: 8080
ssl: false
processor:
- grok:
grok_when: ‘contains(request_url, “/ui/api/”)’
match:
request_url: [‘%{WORD:http_method} (?:%{URIPATH:request_endpoint}(?:?%{GREEDYDATA:request_params})?) HTTP/%{NUMBER:client_version}’]
- parse_json:
source: json
- convert_entry_type:
key: “request_status”
type: “integer”
- convert_entry_type:
key: “upstream_status”
type: “integer”
- convert_entry_type:
key: “bytes_sent”
type: “integer”
- convert_entry_type:
key: “request_time”
type: “double”
- convert_entry_type:
key: “upstream_connect_time”
type: “double”
- convert_entry_type:
key: “upstream_header_time”
type: “double”
- convert_entry_type:
key: “upstream_response_time”
type: “double”
- convert_entry_type:
key: “connections_active”
type: “integer”
- delete_entries:
with_keys: [“http_method”]
sink:
- stdout: {}
Can you help?
Thanks in advance