TIL: Setting multi-line secrets in AWS Secrets Manager
How the AWS Console can quietly mess up formatting of your values
Recently I had to store a PGP private key inside a JSON secret in AWS Secrets Manager, as one of the key/value pairs. Sounds pretty easy, right? Just paste it into the AWS Console's UI. After trial and error, and spending a bit more time that I'd like to admit, I can pretty confidently say - please don't do this. I tried the following, which I though should "just" work:
- Pasting the raw key directly into the JSON value field. The newlines got stripped, so my app got a single line that PGP library being used could not parse at all.
- Running the value through
JSON.stringifyfirst so the newlines became\n. That also didn't help.
The annoying part is that the Console gives no indication that anything is wrong. The secret looks fine in the UI, and the failure only shows up at runtime when whatever is consuming the value tries to actually use it.
The fix is to keep the value as far as possible from the Console's UI. Instead, use awscli, with a little help from jq for formatting in our case:
jq -n --rawfile key ./private.asc '{ pgp_key: $key, other_value: "..." }' > secret.json
aws secretsmanager put-secret-value \
--secret-id my/secret \
--secret-string file://secret.jsonjq --rawfile reads the key file verbatim and handles JSON escaping, and file:// skips any shell-quoting nonsense. To verify, fetch the secret back and inspect it:
aws secretsmanager get-secret-value --secret-id my/secret \
--query SecretString --output text | jq .Now everything is formatted beautifully and the application reading the secret has no problem with messed up formatting. Thanks a lot for reading!