Did some more testing, and it's not an imagemagick problem (Or it is, in a roundabout way)
convert wasn't in the path of the webserver, so putting a full path for CONVERT_COMMAND fixed it.

Aha. sips is the problem. According to sips --help:
-z, --resampleHeightWidth pixelsH pixelsW

Height, then width. No, I don't know why either.
But just switching the order makes it work fine:
sips -z $height $width -s formatOptions $quality -s format jpeg $filename --out $thumbnail >/dev/null; # quality setting doesn't seem to work

Also, even when it's working it gives this error in the logs:
Warning: File format jpeg does not support option 70
It looks like the way to do quality is either "formatOptions" or "quality", but neither of them supports integer settings, only low/normal/high (ewww)
I found this from doing "sips --helpProperties" (which is bloody hard to find out about)