Continuous transform stuck in "initializing"

Plugins Index Management
1

I have a series of large indexes which hold a dataset that we are attempting to pivot using a continuous transform. I have been able to get the previous “non-continuous” transform to run over the data in about 7 - 10 days. However, I have not been able to get the continuous transform to run. It still says “initializing” after 5 or 6 days. The _explain endpoint does show that the transform is many objects behind on the source indexes, but not much else of value. Any assistance with getting this transform to actually run continuously would be great.

For reference, here is the output from the _explain endpoint.

"transform-2y_v1" : {
  "metadata_id" : "Vftr1N7GIbe2bnr9TuD_DQ",
  "transform_metadata" : {
    "transform_id" : "transform-2y_v1",
    "last_updated_at" : 1648105507920,
    "status" : "init",
    "failure_reason" : null,
    "stats" : {
      "pages_processed" : 0,
      "documents_processed" : 0,
      "documents_indexed" : 0,
      "index_time_in_millis" : 0,
      "search_time_in_millis" : 0
    },
    "continuous_stats" : {
      "documents_behind" : {
        "2019-000006" : 163389464,
        "2018-000004" : 132498845,
        "2018-000003" : 130003533,
        "2019-000007" : 163840084,
        "2019-000004" : 152933355,
        "2018-000005" : 107461609,
        "2019-000005" : 164770719,
        "2019-000002" : 142111682,
        "2019-000003" : 146678088,
        "2018-000002" : 125507755,
        "2018-000001" : 125309600,
        "2019-000001" : 144003746,
        "2022-000003" : 258400419,
        "2021-000002" : 182019854,
        "2020-000001" : 199180443,
        "2021-000001" : 174520029,
        "2021-000004" : 862156913,
        "2020-000002" : 192869187,
        "2022-000006" : 257450152,
        "2020-000003" : 194221140,
        "2022-000007" : 252686088,
        "2021-000003" : 189643358,
        "2019-000008" : 160414127,
        "2022-000004" : 255916855,
        "2020-000004" : 190731072,
        "2022-000005" : 256935329,
        "2019-000009" : 149586918,
        "2020-000005" : 194679349,
        "2021-000005" : 0,
        "2020-000006" : 139235889,
        "2022-000008" : 251051311,
        "2022-000009" : 251498822,
        "pre-2018-000001" : 241379636,
        "pre-2018-000002" : 197650317,
        "2022-000010" : 260072274,
        "2022-000011" : 20892538
      }
    }
  }
}
2

Hi, were you able to get the continuous transform to run? Could you provide me with some more information to help get to the bottom of this:
Can you check your logs to see if there is anything related to the continuous transform job? If you grep for your transform job ID, in your case transform-2y_v, that should come up with most relevant logs.
What is your job interval for this continuous transform job? If possible, would you be able to post the transform job definition?
What I immediately think of is that this could be an issue with how the continuous transform is handling locking if the job interval is short but the execution takes a really long time. Could you try setting the interval to something large like 7 days and see what happens?

3

I’m sorry I abandoned this thread over the spring and summer. The jobs continued to fail and I needed to attend to other issues. Once we upgrade the production cluster to a recent version, I will be re-attempting this.