Jamie Kitson followed the instructions to integrate their software with a new payment provider. The payment API was fairly straight forward, mostly a straightforward call to a web endpoint. As an error check, the request required an base-64 encoded, MD5 hash of its contents appended to the end of it.

Jamie did just that, in C#. And the payment processor balked: the hash was wrong. There was no information beyond that, just "bad hash".

Jamie checked the output, hashed many different possible values, confirmed that a different MD5 hashing library generated the same results, and did all of the sane things one might to do check and see if you were correctly hashing an input. They checked the documentation, confirmed that they were hashing the right contents, confirmed that there wasn't any salting, confirmed that nothing they were doing on their end was wrong.

Eventually, Jamie tried the JavaScript sample code provided by the vendor. And it gave a different result.

var hashVal = CryptoJS.MD5(hStr)
var hashVal = window.btoa(hashVal)

This seems pretty straightforward, right? We hash the content, and then Base64 encode it. It looks nearly identical to the C# code that Jamie was using. And requests generated using this method worked. So what was wrong?

Jamie checked out the docs for CryptoJS. The hashing functions didn't return strings, they returned WordArray objects- arrays of 32-bit integers. But, when you attempt to use a WordArray as if it were a string, well: "When you use a WordArray object in a string context, it's automatically converted to a hex string."

The payment provider wasn't Base64 encoding the hash. They were Base64 encoding the hex string representing the hash. That wasn't just in their sample code, that was their actual implementation.

The WordArray object has its own Base64 conversion (hash.toString(CryptoJS.enc.Base64)), which generated output identical to Jamie's.

So the payment provider used a cryptographic library without a full understanding of its interface, ended up treating hashes like hexadecimal strings, not binary data, and then required all customers to do the same- without documenting that requirement. Oh, and since the string is a hex string, you don't need to Base64 encode it in the first place, making the whole thing extra silly.

And they're handling payments, which raises all sorts of questions.

[Advertisement] Keep the plebs out of prod. Restrict NuGet feed privileges with ProGet. Learn more.