Why is my Lambda function getting retried when getting synchronously called from the command line?

I recently ran into a surprising behavior when calling a Lambda function from the command line that I thought I'd share.

I have a dumb little function that splits up newsletters into an RSS feed of individual entries (because that's how I prefer to consume Hacker Newsletter and Golang Weekly). It only runs on Tuesdays and Fridays, and Golang Weekly was a little late this week, so I had to run it by hand:

$ aws lambda invoke --function-name newsletter-expander --log-type Tail /dev/stdout | jq -r 'objects | .LogResult' | base64 -d

It ran, but much to my surprise, when I looked at the logs, I saw it ran three times, and each execution started roughly one minute after the previous (the function does things like resolving link redirects, so it can take over a minute to run). Now, I know that when Lambda functions are invoked asynchronously and encounter errors, you can configure them retry - but here I'm invoking mine synchronously, and I saw no evidence of errors in either the logs or the metrics for the function. So what gives?

Well, it turns out that what was happening was aws was waiting for the function to finish and timing out after a minute (which is the default), and then retrying upon error (which it does 4 times by default, for a total of 5 tries). So all I had to do was change the timeout via --cli-read-timeout and that allowed the function to complete without the client timing out!