Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
Logstash v 7.10.0
Describe the issue:
Hi, I am new to logstash, but am trying to read in a non-formatted log with different information I need scattered around. I use the Grok filter to grab that data just fine, but it is spread across multiple json output objects. I need my output to have all of the information I grab in one single json.
Configuration:
I have 3 different grok filters inside of if statements so that I do not end up withhundreds of lines of output that are useless to me like so:
filter {
if ([message] =~ /^Job/) {
grok {
match => { "message" => ['Job \<%{NUMBER:job_id}\> is submitted to queue \<%{WORD:queue}\>\.'] }
add_field => {
"group" => "1"
}
}
}
else if ([message] =~ /^Running test/) {
grok {
match => { "message" => ['Running test %{WORD:test} on block %{WORD:Block} with seed %{WORD:Seed}'] }
add_field => {
"group" => "1"
}
}
}
Relevant Logs or Screenshots:
Log file lines:
Job <4471520> is submitted to queue <hw_queue>.
<<Waiting for dispatch ...>>
<<Starting on server>>
LSB_JOBNAME is /path/tests/test_2m_4s/511863904/simulate
presim
Running test test_2m_4s on block BLOCK with seed 511863904
Running this command:
<command>
TOOL: xrun(64) 21.12-a071: Started on Jan 31, 2023 at 12:52:33 CST
--------------------------------------------------------------------
Name Type Size Value
--------------------------------------------------------------------
...
Current output looks like:
{"queue":"hw_queue","job_id":"4471520"}
{"Block":"BLOCK","test":"test_2m_4s","Seed":"511863904"}
{"Year":"2023","Month":"Jan","Day":"31","TimeZone":"CST"}
Desired output:
{"queue":"hw_queue","job_id":"4471520", "Block":"BLOCK","test":"test_2m_4s","Seed":"511863904", "Year":"2023","Month":"Jan","Day":"31","TimeZone":"CST"}
I have tried a few things, but nothing has been working for me, any help would be appreciated.