generating aws signature

Q&A's, tips, howto's

generating aws signature

Postby joejoe » Sun Oct 23, 2016 4:38 am

Hi,

I am following this php code and am trying to reproduce it with nL.

http://webtutsdepot.com/2009/10/13/amaz ... quest-php/

I cannot figure out if this is a two part transaction or just a single query.

Would anyone know if I need an initial aws server response from the get-url command in order to complete the rest of building the url that will make the request with the aws signature?

Like do I need to say 'knock knock', and use part of their 'whos there' response in order to generate my signature?

Thanks for help!

joejoe
joejoe
 
Posts: 153
Joined: Thu Jun 25, 2009 5:09 pm
Location: Denver, USA

Re: generating aws signature

Postby ralph.ronnquist » Sun Oct 23, 2016 5:07 am

I'm no expert on AWS API, but from reading the PHP, it looks like it's a single handshake, but it relies on a prior agreement between you and Amazon about the thingies called "$publicKey" and "$privateKey". Basically it seems to be a matter of scrambling the original request with the private key, then issue the compound request of that scramble together with the public key, for getting the request to be serviced.
ralph.ronnquist
 
Posts: 209
Joined: Mon Jun 02, 2014 1:40 am
Location: Melbourne, Australia

Re: generating aws signature

Postby joejoe » Sun Oct 23, 2016 5:12 am

Ok great, that is what I had hoped, thank you very much Ralph!
joejoe
 
Posts: 153
Joined: Thu Jun 25, 2009 5:09 pm
Location: Denver, USA

Re: generating aws signature

Postby joejoe » Sun Oct 23, 2016 5:16 am

And would I be correct to use this module that Lutz has already created?

http://www.newlisp.org/code/modules/cry ... rypto_hmac

or just use the

http://www.newlisp.org/code/modules/cry ... pto_sha256

Thanks again!
joejoe
 
Posts: 153
Joined: Thu Jun 25, 2009 5:09 pm
Location: Denver, USA

Re: generating aws signature

Postby ralph.ronnquist » Sun Oct 23, 2016 5:22 am

I'm pretty sure it'd be the hash_hmac step by using those two, yes, as in:
Code: Select all
(crypto:hmac crypto:sha256 message key)

Then it needs base64-enc and url-encode and replace "%7E" with "~"...
Last edited by ralph.ronnquist on Sun Oct 23, 2016 5:32 am, edited 1 time in total.
ralph.ronnquist
 
Posts: 209
Joined: Mon Jun 02, 2014 1:40 am
Location: Melbourne, Australia

Re: generating aws signature

Postby joejoe » Sun Oct 23, 2016 5:28 am

Got it, thanks Ralph! Much appreciated!! :D
joejoe
 
Posts: 153
Joined: Thu Jun 25, 2009 5:09 pm
Location: Denver, USA

Re: generating aws signature

Postby joejoe » Sat Nov 05, 2016 11:03 am

Hi and thanks!

I think I got to the signature:

Code: Select all
("131 37 166 173 32 246 42 23 198 156 244 102 148 91 171 80 252 115 124 141 246 64
 19 94 85 112 145 181 2 189 98 73")

When I use an online hex decoder I get this: qf3"FB#V$A‰q€%!AA$fA™HQQ!‰˜s

Looks like a signature!

How would I decode the hex in nL to produce this signature?

I have tried different things with char and format but am still swinging at it.

Thank you very much for the help! :D

If I helps anyone, here is how I have gotten this far:

Code: Select all
#!/usr/bin/newlisp

(module "crypto.lsp")

(set 'output1 (crypto:hmac crypto:sha256 "GET
ecs.amazonaws.com
/onca/xml
AWSAccessKeyId= AAAAAAAAAAAAAAAAAAA&AssociateTag=PutYourAssociateTagHere&Keywords=newlisp&Operation=ItemSearch&SearchIndex=Books&Service=AWSECommerceService&Timestamp=2016-11-05T10%3A25%3A14.000Z&Version=2011-08-01" "BBBBBBBBBBBBBBBBBBBBBBBBBBBBB"))

(set 'output2 (unpack (dup "b" (length output1)) output1))

(set 'output3 (crypto:hmac crypto:sha256 "GET
ecs.amazonaws.com
/onca/xml
AWSAccessKeyId= AAAAAAAAAAAAAAAAAAA&AssociateTag=PutYourAssociateTagHere&Keywords= newlisp&Operation=ItemSearch&SearchIndex=Books&Service=AWSECommerceService&Timestamp=2016-11-05T10%3A25%3A14.000Z&Version=2011-08-01" (string output2)))

(set 'signature-hex (unpack (dup "b" (length output3)) output3))

(println signature-hex)

(exit)
joejoe
 
Posts: 153
Joined: Thu Jun 25, 2009 5:09 pm
Location: Denver, USA

Re: generating aws signature

Postby ralph.ronnquist » Sun Nov 06, 2016 2:33 am

Wouldn't the "hexdigest" merely be something like this?
Code: Select all
(join (map (curry format "%02x") signature-hex))
ralph.ronnquist
 
Posts: 209
Joined: Mon Jun 02, 2014 1:40 am
Location: Melbourne, Australia

Re: generating aws signature

Postby joejoe » Sun Dec 11, 2016 10:57 am

Hi and thank you for help,

Here is what I am running:

Code: Select all
#!/usr/bin/newlisp

(module "crypto.lsp")

(set 'pubkey "aa")
(set 'privkey "bb")

(set 'timestamp (string (date (date-value) 320 "%Y") "-" (date (date-value) 320 "%m") "-" (date (date-value) 320 "%d") "T" (date (date-value) 320 "%H") "\%3A" (date (date-value) 0 "%M") "\%3A" (date (date-value) 320 "%S") "Z"))

(println timestamp)

(set 'output1 (crypto:hmac crypto:sha256 "GET
ecs.amazonaws.com
/onca/xml
AWSAccessKeyId=" (silent pubkey) "&Keywords=newlisp&Operation=ItemSearch&SearchIndex=Books&Service=AWSECommerceService&Timestamp=" (silent timestamp) "&Version=2011-08-01" (silent privkey)))

(set 'output2 (unpack (dup "b" (length output1)) output1))

(set 'output3 (crypto:hmac crypto:sha256 "GET
ecs.amazonaws.com
/onca/xml
AWSAccessKeyId=" (silent pubkey) "&Keywords= newlisp&Operation=ItemSearch&SearchIndex=Books&Service=AWSECommerceService&Timestamp=" (silent timestamp) "&Version=2011-08-01" (string output2)))

(set 'signature-hex (unpack (dup "b" (length output3)) output3))

(println (join (map (curry format "%02x") signature-hex)))

(exit)


I get a new timestamp but the signature doesn't change when I re-execute it.

Thanks for any tip!
joejoe
 
Posts: 153
Joined: Thu Jun 25, 2009 5:09 pm
Location: Denver, USA

Re: generating aws signature

Postby ralph.ronnquist » Mon Dec 12, 2016 9:27 am

Maybe you meant to have (string ...) embeddings for the third argument to crypto:hmac calls?
ralph.ronnquist
 
Posts: 209
Joined: Mon Jun 02, 2014 1:40 am
Location: Melbourne, Australia

Re: generating aws signature

Postby joejoe » Thu Dec 15, 2016 2:26 am

Hi and thanks,

I want to back up to make sure I have the HMAC SHA256 signature calculating correctly.

I am trying this:

Code: Select all
(set 'output1 (crypto:hmac crypto:sha256 (string "GET
webservices.amazon.co.uk
/onca/xml
AWSAccessKeyId=AKIAIOSFODNN7EXAMPLE&Actor=Johnny%20Depp&AssociateTag=mytag-20&Operation=ItemSearch&Operation=ItemSearch&ResponseGroup=ItemAttributes%2COffers%2CImages%2CReviews%2CVariations&SearchIndex=DVD&Service=AWSECommerceService&Sort=salesrank&Timestamp=2016-12-15T01%3A42%3A01.000Z&Version=2013-08-01") "1234567890"))

(set 'signature-hex (unpack (dup "b" (length output1)) output1))
(println (join (map (curry format "%02x") signature-hex)))


I get this:

Code: Select all
786c6ac1346d09aeb7fc4d158e70201141ee483d9e338958bf8492740969a997


On the AWS tester tool[1], it says the HMAC signature should be this:

Code: Select all
eGxqwTRtCa63%2FE0VjnAgEUHuSD2eM4lYv4SSdAlpqZc%3D


With an online HMAC generator tester tool[2] set to SHA256, it says I should get this signature:

Code: Select all
9f4defac2a33e7ce3e585c0515bdba65d272fd8852a3b40e5549f8ad9104c4ac


Anyone know which is correct? Thanks! :D

[1] https://associates-amazon.s3.amazonaws. ... index.html
[2] https://www.freeformatter.com/hmac-gene ... #ad-output
joejoe
 
Posts: 153
Joined: Thu Jun 25, 2009 5:09 pm
Location: Denver, USA

Re: generating aws signature

Postby joejoe » Wed Dec 21, 2016 3:32 pm

I think the issue may be with line breaks.

When I do this:

Code: Select all
(set 'output1 (crypto:hmac crypto:sha256 "onetwo" "1234567890"))

(set 'signature-hex (unpack (dup "b" (length output1)) output1))

(println (join (map (curry format "%02x") signature-hex)))


I get this:

Code: Select all
47d07edd67e5cca3bb98c5cf4cca73459dd8a89afaa555ac7b4ce475c6ae6c27


The online HMAC generator tester confirms the same:

Code: Select all
47d07edd67e5cca3bb98c5cf4cca73459dd8a89afaa555ac7b4ce475c6ae6c27


However, when I try to put a line break in the string with:

Code: Select all
(set 'output1 (crypto:hmac crypto:sha256 "one
two" "1234567890"))


I get this:

Code: Select all
b599058300d5ed4e5f160c30745c74f3e6c5cfc6683cc9d36b9a279f8955077e


However, the online HMAC tool shows this:

Code: Select all
e16a940481b8b9dd18211f5fb637bd5d926ac9d6e446e6489f030d66ea17fb91


And in the online HMAC tool I am inputting in the string so it has the line break:

Code: Select all
one
two


Getting close!!

Would there be any clues out there?

Thank you!!!
joejoe
 
Posts: 153
Joined: Thu Jun 25, 2009 5:09 pm
Location: Denver, USA

Re: generating aws signature

Postby ralph.ronnquist » Wed Dec 21, 2016 9:10 pm

An online tool, using a form, would typically digest line ends as \r\j, whereas a *nix system would prefer using \j only, and a mac system would rather favour \r for line endings. The AWS tool you pointed at some posts earlier appears to compute the signature using \j line endings. And it also scrambles the hmac output by both base64 encoding then url-encoding.
ralph.ronnquist
 
Posts: 209
Joined: Mon Jun 02, 2014 1:40 am
Location: Melbourne, Australia

Re: generating aws signature

Postby joejoe » Wed Dec 21, 2016 9:47 pm

Ok gotcha thanks.

I got the signatures generating correctly with your message help, Ralph.

Code: Select all
(string (chop (base64-enc (crypto:hmac ....)) "%3D")


This did it and a few tweaks to finalize the url formatting got successful api calls.

Thanks all for the help!
joejoe
 
Posts: 153
Joined: Thu Jun 25, 2009 5:09 pm
Location: Denver, USA


Return to newLISP in the real world

Who is online

Users browsing this forum: No registered users and 1 guest

cron