Describe the issue:
As the Java high-level REST Client is getting deprecating, I am planning to use Java Client version 2.4.0 (which is the latest) to connect to my AWS OpenSearch Service (OpenSearch 2.5.0). OpenSearch doc gives an example to connect to Amazon OpenSearch Service which uses AwsSdk2Transport transport. AwsSdk2Transport uses AWS SDK for Java 2.x. My company still uses AWS SDK for Java 1.x and has no recent plans to move to AWS SDK for Java 2.x. Is there any other way to connect to AWS OpenSearch Service using Java Client or Java high-level REST Client is the only option?
Or can I use Apache HttpClient 5 Transport as it’s mentioned in the doc and make it work? I am not sure if this is a better idea compared to using Java high-level REST Client. Also, this seems to me that I would require a good understanding of SSL and TLS technology. Thank you!
I was able to connect to AWS OpenSearch service hosted in an AWS VPC from a service running on AWS EC2 instance using both Apache HttpClient 5 Transport (Preferred way as mentioned here and Apache RestClient Transport.
For more information, check here.
Using Apache HttpClient 5 Transport
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLEngine;
import org.apache.hc.client5.http.auth.AuthScope;
import org.apache.hc.client5.http.auth.UsernamePasswordCredentials;
import org.apache.hc.client5.http.impl.auth.BasicCredentialsProvider;
import org.apache.hc.client5.http.impl.nio.PoolingAsyncClientConnectionManager;
import org.apache.hc.client5.http.impl.nio.PoolingAsyncClientConnectionManagerBuilder;
import org.apache.hc.client5.http.ssl.ClientTlsStrategyBuilder;
import org.apache.hc.core5.function.Factory;
import org.apache.hc.core5.http.HttpHost;
import org.apache.hc.core5.http.nio.ssl.TlsStrategy;
import org.apache.hc.core5.reactor.ssl.TlsDetails;
import org.apache.hc.core5.ssl.SSLContextBuilder;
import org.opensearch.client.opensearch.OpenSearchClient;
import org.opensearch.client.transport.OpenSearchTransport;
import org.opensearch.client.transport.httpclient5.ApacheHttpClient5TransportBuilder;
...
...
final int port = 443;
final Set<String> nodeAddresses = Set.of("*your vpc/search endpoint for AWS OpenSearch Service*");
final HttpHost[] hosts = nodeAddresses.stream().map(address -> new HttpHost("https", address, port)).toArray(HttpHost[]::new);
final BasicCredentialsProvider credentialsProvider = new BasicCredentialsProvider();
// Only for demo purposes. Don't specify your credentials in code.
// credentialsProvider.setCredentials(new AuthScope(host), new UsernamePasswordCredentials("admin", "admin".toCharArray()));
// I created AWS OpenSearch cluster without username and password, so I didn't need to provide that.
// You need to make sure that the EC2 instance that you will be sending request from has access (that is defined in AWS OpenSearch Cluster security group).
final SSLContext sslcontext = SSLContextBuilder
.create()
.loadTrustMaterial(null, (chains, authType) -> true)
.build();
final ApacheHttpClient5TransportBuilder builder = ApacheHttpClient5TransportBuilder.builder(hosts);
builder.setHttpClientConfigCallback(httpClientBuilder -> {
TlsStrategy tlsStrategy = null;
try {
tlsStrategy = ClientTlsStrategyBuilder.create()
.setSslContext(SSLContextBuilder.create().build())
// Uncomment this if using Java 9 or older version. See https://issues.apache.org/jira/browse/HTTPCLIENT-2219
// .setTlsDetailsFactory(new Factory<SSLEngine, TlsDetails>() {
// @Override
// public TlsDetails create(final SSLEngine sslEngine) {
// return new TlsDetails(sslEngine.getSession(), sslEngine.getApplicationProtocol());
// }
// })
.build();
} catch (NoSuchAlgorithmException e) {
e.printStackTrace();
} catch (KeyManagementException e) {
e.printStackTrace();
}
final PoolingAsyncClientConnectionManager connectionManager = PoolingAsyncClientConnectionManagerBuilder
.create()
.setTlsStrategy(tlsStrategy)
.build();
return httpClientBuilder
.setDefaultCredentialsProvider(credentialsProvider)
.setConnectionManager(connectionManager);
});
final OpenSearchTransport transport = ApacheHttpClient5TransportBuilder.builder(hosts).build();
final OpenSearchClient client = new OpenSearchClient(transport);
Using RestClient Transport
import org.apache.hc.client5.http.auth.AuthScope;
import org.apache.hc.client5.http.auth.UsernamePasswordCredentials;
import org.apache.hc.client5.http.impl.async.HttpAsyncClientBuilder;
import org.apache.hc.client5.http.impl.auth.BasicCredentialsProvider;
import org.apache.hc.core5.http.HttpHost;
import org.opensearch.client.RestClient;
import org.opensearch.client.RestClientBuilder;
import org.opensearch.client.json.jackson.JacksonJsonpMapper;
import org.opensearch.client.opensearch.OpenSearchClient;
import org.opensearch.client.transport.OpenSearchTransport;
import org.opensearch.client.transport.rest_client.RestClientTransport;
...
...
final int port = 443;
final Set<String> nodeAddresses = Set.of("*your vpc/search endpoint for AWS OpenSearch Service*");
final HttpHost[] hosts = nodeAddresses.stream().map(nodeAddress -> new HttpHost(nodeAddress, port, "https")).toArray(HttpHost[]::new);
final BasicCredentialsProvider credentialsProvider = new BasicCredentialsProvider();
// Only for demo purposes. Don't specify your credentials in code.
// credentialsProvider.setCredentials(new AuthScope(host), new UsernamePasswordCredentials("admin", "admin".toCharArray()));
// I created AWS OpenSearch cluster without username and password, so I didn't need to provide that.
// You need to make sure that the EC2 instance that you will be sending request from has access (that is defined in AWS OpenSearch Cluster security group).
final RestClient restClient = RestClient.builder(hosts).
setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
@Override
public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
}
}).build();
final OpenSearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
final OpenSearchClient client = new OpenSearchClient(transport);