upload-cert is exporting cert in binary form, but current instructions on google play console are providing commands to generate it in base encoded format (old binary format is still working when you switch from old signing to new one)
You can wait for this fix ([expo-cli] export upload certificate in RFC format by wkozyra95 · Pull Request #594 · expo/expo-cli · GitHub) or export cert manually with
keytool -export -rfc
-keystore your-upload-keystore.jks
-alias upload-alias
-file output_upload_certificate.pem