Multiple :sql_last_value for Logstash JDBC?

Open Source Elasticsearch and Kibana
1

Hi all. Does anyone know if it’s possible to have OS Logstash store more than one :sql_last_value if you’re using more than one JDBC input in the config file? If not, it seems we’re either limited to a single JDBC, or have to tolerate the :sql_last_value being captured only for the last JDBC it ran. That might be a substantial number of rows to miss.

I found where it stores that value. But it looks like it will only store one value–not one per JDBC input:

logstash@995dfd795f4d:~/data/plugins/inputs/jdbc$ cat logstash_jdbc_last_run
--- 2023-11-13 20:40:00.891320000 Z

Thanks!

2

You can use specify custom path to store the sql_last_value, just like this:

input {
  jdbc {
    statement => "SELECT * FROM mgd.seq_sequence WHERE _sequence_key > ? AND _sequence_key < ? + ? ORDER BY _sequence_key ASC"
    last_run_metadata_path => "/my_path/test-jdbc-int-sql_last_value1.yml"
    # ... other configuration bits
  }
jdbc {
    statement => "SELECT * FROM mgd.seq_sequence WHERE _sequence_key > ? AND _sequence_key < ? + ? ORDER BY _sequence_key ASC"
    last_run_metadata_path => "/my_path/test-jdbc-int-sql_last_value2.yml"
    # ... other configuration bits
  }
}