Push notifications failing

Please provide the following:

  1. SDK Version:35
  2. Platforms(Android/iOS/web/all):android/iOS

We’ve had push notifications running without any issues for a while now. Starting today we are seeing a lot of failures. Not all notifications are failing, however, which is what’s confusing me. Any ideas?

The error message is:

Failed to generate token: error:0909006C:PEM routines:get_name:no start line

InvalidCredentials

The Apple Push Notification service key for your account could not be loaded, likely due to a malformed p8 file. Expo needs a valid PEM key in order to deliver notifications to APNs.

If it was a bad p8 file, wouldn’t all push messages be failing?

Thanks for any insights you can provide.

2 Likes

Jeff, I’m having the same issue. Things were running great, no changes were made in the last week and now suddenly as of this morning, only some are going through. As much as this sucks, it’s good to know that someone else is dealing with the exact same problem currently.

1 Like

Exactly the same issue here and symptoms. About 50% failures or so. We tried to deploy a new SDK 36 build yesterday, so I thought it was related to that, but now I’m not so sure.

@jeffws did you try deploying yesterday?

Nothing change on Expo’s end this morning to our knowledge. We’re rolling back the iOS push notification service to an older revision just in case to see if that helps.

@jeffws did you try deploying yesterday?

We did a small OTA update on a staging version of our app yesterday but did not deploy anything to our production app where the failures are occurring. It’s possible that this affected something with the push notifications on prod but not seeing any failures with push notifications on our staging app.

UPDATE – did just see some push failures on our staging app as well.

Thanks, we’re monitoring this so we’ll see if there’s any improvement. (It won’t be immediately obvious since the failures are ~50%).

If it helps, the first failure we saw was around UTC 2020-01-02 10:21 , although we don’t send many notifications in that period so it probably happened well before that.

We think we’ve found the root cause of this. There was one push notification service instance that was running a different copy of the credential decryption code that was out of sync with the protocol all the other servers use. This is why there was a credentials decryption error just some of the time. Really sorry for the outage and expect it to be resolved now.

2 Likes

This seems to be fixed from our repro attempts. Thanks for the prompt turnaround on the investigation!

@ide

Appears to be fixed on our side as well. Thanks for looking into it! Will continue to monitor.

hi guys,
I have the same error right now:

  1. SDK Version:35
  2. Platforms(iOS): iOS 13.3

is a Builded version.

The Apple Push Notification service key for your account could not be loaded, likely due to a malformed p8 file. Expo needs a valid PEM key in order to deliver notifications to APNs.

Is a staging problem ?
Let me know. regards.

@gex.rossi everything has been working fine for us since the fix.

And How can i solve this problem ?

We upload the right distribution p12 also we upload the aps.p12 for the push notification service key.
And then we upload the provision.mobile created with distribution.p12.

Currently expo describe so our credential:

Application credentials

Provisioning profile is missing. It will be generated during the next build
Apple Team ID: XXXXXX,  Apple Team Name: YYYYY

Provisioning profile is missing. It will be generated during the next build
Apple Team ID: ZZZZZZ,  Apple Team Name: ---------

User credentials

Push Notifications Key - Key ID: ZZZZZZZZ
Apple Team ID: XXXXXX, Apple Team Name: ---------
used by

Distribution Certificate - Certificate ID: -----
Apple Team ID: MMMMMMMM, Apple Team Name: ---------
used by

Below the build creation process:
? Path to P12 file: mypath\idistribution.p12
? P12 password: [hidden]
Please provide your Apple Push Notifications service key:
? Path to P8 file: mypath\aps.p12
? Key ID: ZZZZZZZZ
Please provide your Apple Provisioning Profile:
? Path to .mobile provisioning profile: mypath.mobileprovision

Build OK!

check the receipt:

{“data”:{“635c4748-44a3-472f-bef6-6238c11f9e48”:{“status”:“error”,“message”:“The Apple Push Notification service key for your account could not be loaded, likely due to a malformed p8 file. Expo needs a valid PEM key in order to deliver notifications to APNs.”,“details”:{“error”:“InvalidCredentials”,“sentAt”:1578835410},"__debug":{“internalError”:“Failed to generate token: error:0909006C:PEM routines:get_name:no start line”}}}}

that’s all