Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
opensearch java client 2.8.1
Describe the issue:
Is there an easy way to sign the request when using Apache HttpClient 5 Transport?
Is there an existing example somewhere?
@opensearchlearner if you mean signing with AWS Sig v4, see please opensearch-java/guides/auth.md at main · opensearch-project/opensearch-java · GitHub (TLDR; this is AWS SDK based transport). If you really need to use Apache HttpClient 5 Transport, please check GitHub - acm19/aws-request-signing-apache-interceptor
@reta ,
Thanks for the response,
Yes you are right, I mean signing with AWS Sig v4.
I investigated using AwsSdk2Transport as well, As I asked here, I have 2 issues using it which has not been addressed yet.
The good thing about using ApacheHttpClient5Transport was that I had option to choose between http and https for running local integration tests by http.
I tried to use GitHub - acm19/aws-request-signing-apache-interceptor,
It is working fine for GET requests but ES returning 403 for PUT requests and it looks like signing is not happening properly.
When I am using GitHub - acm19/aws-request-signing-apache-interceptor with ApacheHttpClient5Transport as mentioned, get requests are fine but for the PUT request /i receive this error:
Caused by: org.opensearch.client.transport.httpclient5.ResponseException: method [PUT], host [], URI [], status line [HTTP/1.1 403 Forbidden]
{“message”:"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.
The Canonical String for this request should have been
'PUT
accept:application/json; charset=UTF-8
content-length:47
content-type:application/json; charset=UTF-8
host:search-catalogue-data-s–1817108109-rccx5nu236eebbvl4mrzwgwhvq.ap-southeast-2.es.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20240312T030211Z
x-amz-security-token:IQoJb3JpZ2luX2VjEDMaDmFwLXNvdXRoZWFzdC0yIkcwRQIgGzc/Is/bMwbyz1qGxwgHINfduFEuDhsNSwbweO+9gd4CIQD6MjY/9FbaUEySm4MEZgqgOnUWvielLFRfJPGytopEtSrKBQg8EAAaDDkzNjY2NDQ2Mzg1NiIMNuVyrsSnflj7w9TyKqcFJwp4G4w7mETv0KS6zjtJWOwXexZaK2Ho9XbuJzSUioO4yYrQKEzAIKaDJlrbDihjBbFgp//oq7/L2EkgvGrOXoKcSvl5xCTERnyo2urX/ImyzGC9RmvzbYt4rQaKzQRssZGP5TwGs1XPDpIcu1iVnUsfu0xNZG2YhtOuLardM7yQcn5Jqt8FXOR4NZZlzjBsa/FfcpymwdbdS0TrKPpTG24s8H84HyfoSMYlPk1PpgMt1ZVJWiKRsKWquesLcpGCiF6utK++L8cimFAsEf4LxEZ4ST/lrdq8unupDN+Edm862yiC96u1FB/GdASwWHaKt3285Vau3Ce/ipjFzUEpXcs3Rd6itcS3OOIU5IrQiqgXYxtBLLbFliztDxQ+CeLvAstOB7oh2i8tyi+5dDH2ZRX5JeR0NitNJQF3plpe5ktuXjtVqZn1Wwpk4UmykJPuVfUK+CKCIebMScma0JOa589JHB/ngWbkHVZCHzIEuDrwkOK2mA7mbCXEjChg8rV244CoWg6XwEGk/+8KowL8L2CL66HJx0CLkLZ6opwmnrUlmwlsSPrbbPqU95zVQ4aOKSLjt6w7/kQTUq0TKTh0fWoPuhuU8axHfUv3JU4NUNzvyocJW16F6iajrZ+mcne5a4/0Y3MNAE4Ch4dDrKGkonWqClOeXPuttvKS0JlTI3qRbyerTMYweUPUFKmmrA6IKlog5a9l4U7xUu84l1Vt0redHPyNyImSiyWT9f73PaYEDYyGCBRFFUe9MVWgE5jSxEJ3M6Q4CvhKBfX0mwQBHXiVBBcCObCkRNAWbGORNr1++RjA5hoDEEuyEw0napxgIDawHSW5tp3rJzo5TMFzz/T1/5hVXsQKQS8uEBYk5jzBgdJqK6My4WLVOjq8L8hOjocw6rz8izCGib+vBjqxAah0pFU06yYGC7BLy56IgKp9J3I5cw6+mRgLBSPFZiCZfS3XPHL7XT2FGwkEK6gbDDg8qqT/tl2ZfGD6N853GJ97MQD2Gpm8S0vzeR3/odCftDvXXNZJxf5tMXtnXtGNcOVBr6rs4KSSGEOkVKeAk8xA62hoz2DDDzActYxXbB6myfa8+q3dXQN0XhgYuw27onJJ5faPSkG4FHiJHuXdUSQKz5OUOsw4AFzeQDYPhQsNyg==
accept;content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-security-token
4db4276a52606b18093df202d794d9cebaae2e1785ef411ca549520d7a2be21a’
The String-to-Sign should have been
‘AWS4-HMAC-SHA256
20240312T030211Z
20240312/ap-southeast-2/es/aws4_request
2a099805b1c4f747bdc82cc40cde2826d8e703817ea6a280eeba2c5d7907cf60’"}
… 30 common frames omitted
When I am using ApacheHttpClient5Transport with GitHub - acm19/aws-request-signing-apache-interceptor, I’m receiving this error:
org.opensearch.client.transport.httpclient5.ResponseException: method [PUT], host [], URI [], status line [HTTP/1.1 403 Forbidden]
{“message”:"The request signature we calculated does not match the signature you provided.
Check your AWS Secret Access Key and signing method. Consult the service documentation for details.
The Canonical String for this request should have been
'PUT
accept:application/json; charset=UTF-8
content-length:47
content-type:application/json; charset=UTF-8
host:*****
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20240312T030211Z
x-amz-security-token:IQoJb3JpZ2luX2VjEDMaDmFwLXNvdXRoZWFzdC0yIkcwRQIgGzc/Is/bMwbyz1qGxwgHINfduFEuDhsNSwbweO+9gd4CIQD6MjY/9FbaUEySm4MEZgqgOnUWvielLFRfJPGytopEtSrKBQg8EAAaDDkzNjY2NDQ2Mzg1NiIMNuVyrsSnflj7w9TyKqcFJwp4G4w7mETv0KS6zjtJWOwXexZaK2Ho9XbuJzSUioO4yYrQKEzAIKaDJlrbDihjBbFgp//oq7/L2EkgvGrOXoKcSvl5xCTERnyo2urX/ImyzGC9RmvzbYt4rQaKzQRssZGP5TwGs1XPDpIcu1iVnUsfu0xNZG2YhtOuLardM7yQcn5Jqt8FXOR4NZZlzjBsa/FfcpymwdbdS0TrKPpTG24s8H84HyfoSMYlPk1PpgMt1ZVJWiKRsKWquesLcpGCiF6utK++L8cimFAsEf4LxEZ4ST/lrdq8unupDN+Edm862yiC96u1FB/GdASwWHaKt3285Vau3Ce/ipjFzUEpXcs3Rd6itcS3OOIU5IrQiqgXYxtBLLbFliztDxQ+CeLvAstOB7oh2i8tyi+5dDH2ZRX5JeR0NitNJQF3plpe5ktuXjtVqZn1Wwpk4UmykJPuVfUK+CKCIebMScma0JOa589JHB/ngWbkHVZCHzIEuDrwkOK2mA7mbCXEjChg8rV244CoWg6XwEGk/+8KowL8L2CL66HJx0CLkLZ6opwmnrUlmwlsSPrbbPqU95zVQ4aOKSLjt6w7/kQTUq0TKTh0fWoPuhuU8axHfUv3JU4NUNzvyocJW16F6iajrZ+mcne5a4/0Y3MNAE4Ch4dDrKGkonWqClOeXPuttvKS0JlTI3qRbyerTMYweUPUFKmmrA6IKlog5a9l4U7xUu84l1Vt0redHPyNyImSiyWT9f73PaYEDYyGCBRFFUe9MVWgE5jSxEJ3M6Q4CvhKBfX0mwQBHXiVBBcCObCkRNAWbGORNr1++RjA5hoDEEuyEw0napxgIDawHSW5tp3rJzo5TMFzz/T1/5hVXsQKQS8uEBYk5jzBgdJqK6My4WLVOjq8L8hOjocw6rz8izCGib+vBjqxAah0pFU06yYGC7BLy56IgKp9J3I5cw6+mRgLBSPFZiCZfS3XPHL7XT2FGwkEK6gbDDg8qqT/tl2ZfGD6N853GJ97MQD2Gpm8S0vzeR3/odCftDvXXNZJxf5tMXtnXtGNcOVBr6rs4KSSGEOkVKeAk8xA62hoz2DDDzActYxXbB6myfa8+q3dXQN0XhgYuw27onJJ5faPSkG4FHiJHuXdUSQKz5OUOsw4AFzeQDYPhQsNyg==
accept;content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-security-token
4db4276a52606b18093df202d794d9cebaae2e1785ef411ca549520d7a2be21a’
The String-to-Sign should have been
‘AWS4-HMAC-SHA256
20240312T030211Z
20240312/ap-southeast-2/es/aws4_request
2a099805b1c4f747bdc82cc40cde2826d8e703817ea6a280eeba2c5d7907cf60’
"}
… 30 common frames omitted
Thanks @opensearchlearner , opening the issue for GitHub - acm19/aws-request-signing-apache-interceptor would be the best way to solve the problem. Thank you.