Any idea how to upgrade the version of an existed index

OpenSearch
1

Describe the issue:
The derived field type was introduced from 2.15.
I think we can use the feature to new index created by OpenSearch 2.15.

If we have indices that created by before 2.14, how do we use the feature with old indices?
What is the easiest way to upgrade version of indices?

I have 3 options.

  1. Using Reindex API / Update by query
  2. Using Shring/Split, but it requires new index
  3. Snapshot and Restore, but it requires new index

Other any idea?

Thanks,

2

Have you tried that indices created before 2.15.0 cannot use derived field type? I donā€™t thing we have this limitation, once you upgrade your cluster to 2.15.0, all existing old indices can use the new field types.

1 Like
3

Thanks for your response!

And sorry for my mistakesā€¦
You are right.

I have mistaken my sample request.
I suceeded the following requests.

  1. Update the index mapping with derived fields
  2. Search using derived fields to fields parameter

But I got an error using the search request with derived fields sometimes, not always.

Here is the error. I will analyze it and make small reproduce example. Thanks again

org.opensearch.action.search.SearchPhaseExecutionException: all shards failed
        at org.opensearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:770) [opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:395) [opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:810) [opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:548) [opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.action.search.AbstractSearchAsyncAction$1.onFailure(AbstractSearchAsyncAction.java:316) [opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.action.search.SearchExecutionStatsCollector.onFailure(SearchExecutionStatsCollector.java:104) [opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:75) [opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.action.search.SearchTransportService$ConnectionCountingHandler.handleException(SearchTransportService.java:766) [opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.transport.TransportService$9.handleException(TransportService.java:1729) [opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1515) [opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:1629) [opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1603) [opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:81) [opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.transport.TransportChannel.sendErrorResponse(TransportChannel.java:75) [opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.action.support.ChannelActionListener.onFailure(ChannelActionListener.java:70) [opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.action.ActionRunnable.onFailure(ActionRunnable.java:104) [opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:54) [opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.threadpool.TaskAwareRunnable.doRun(TaskAwareRunnable.java:78) [opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) [opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:59) [opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:941) [opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) [opensearch-2.15.0.jar:2.15.0]
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) [?:?]
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) [?:?]
        at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
Caused by: org.opensearch.OpenSearchException$3
        at org.opensearch.OpenSearchException.guessRootCauses(OpenSearchException.java:710) ~[opensearch-core-2.15.0.jar:2.15.0]
        at org.opensearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:393) [opensearch-2.15.0.jar:2.15.0]
        ... 23 more
Caused by: java.util.ConcurrentModificationException
        at java.base/java.util.HashMap$HashIterator.remove(HashMap.java:1619) ~[?:?]
        at org.opensearch.index.mapper.ParametrizedFieldMapper$Builder.parse(ParametrizedFieldMapper.java:732) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.index.mapper.ParametrizedFieldMapper$TypeParser.parse(ParametrizedFieldMapper.java:769) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.index.mapper.ParametrizedFieldMapper$TypeParser.parse(ParametrizedFieldMapper.java:754) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.index.mapper.ObjectMapper$TypeParser.parseDerived(ObjectMapper.java:384) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.index.mapper.ObjectMapper$TypeParser.parseObjectOrDocumentTypeProperties(ObjectMapper.java:298) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.index.mapper.RootObjectMapper$TypeParser.parse(RootObjectMapper.java:185) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:146) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.index.mapper.DefaultDerivedFieldResolver.getAllDerivedFieldTypeFromObject(DefaultDerivedFieldResolver.java:194) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.index.mapper.DefaultDerivedFieldResolver.initDerivedFieldTypes(DefaultDerivedFieldResolver.java:181) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.index.mapper.DefaultDerivedFieldResolver.<init>(DefaultDerivedFieldResolver.java:64) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.index.mapper.DefaultDerivedFieldResolver.<init>(DefaultDerivedFieldResolver.java:45) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.index.mapper.DerivedFieldResolverFactory.createResolver(DerivedFieldResolverFactory.java:49) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.search.SearchService.createSearchContext(SearchService.java:1122) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.search.SearchService.createContext(SearchService.java:1046) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.search.SearchService.executeQueryPhase(SearchService.java:649) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.search.SearchService$2.lambda$onResponse$0(SearchService.java:622) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:74) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.action.ActionRunnable$2.doRun(ActionRunnable.java:89) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) ~[opensearch-2.15.0.jar:2.15.0]
        ... 8 more
4

Maybe the error is fixed in 2.16

1 Like