App crashes after CI publish but fine when locally

Diagnostics from local machine. I can’t run diagnostics on the pipeline - the whole thing freezes when I add this step.

Expo CLI 4.1.6 environment info:
    System:
      OS: macOS Mojave 10.14.6
      Shell: 5.3 - /bin/zsh
    Binaries:
      Node: 14.15.5 - /usr/local/bin/node
      Yarn: 1.22.4 - /usr/local/bin/yarn
      npm: 6.14.11 - /usr/local/bin/npm
      Watchman: 4.9.0 - /usr/local/bin/watchman
    Managers:
      CocoaPods: 1.9.3 - /usr/local/bin/pod
    SDKs:
      iOS SDK:
        Platforms: iOS 13.2, DriverKit 19.0, macOS 10.15, tvOS 13.2, watchOS 6.1
      Android SDK:
        API Levels: 26, 27, 28, 29
        Build Tools: 26.0.3, 27.0.3, 28.0.3, 29.0.2, 30.0.0
        System Images: android-24 | Google APIs Intel x86 Atom, android-28 | Google APIs Intel x86 Atom, android-28 | Google APIs Intel x86 Atom_64, android-30 | Google APIs Intel x86 Atom_64, android-30 | Google Play Intel x86 Atom, android-30 | Google Play Intel x86 Atom_64
    IDEs:
      Android Studio: 4.1 AI-201.8743.12.41.7042882
      Xcode: 11.3.1/11C504 - /usr/bin/xcodebuild
    npmPackages:
      expo: ^39.0.0 => 39.0.3
      react: 16.13.1 => 16.13.1
      react-dom: 16.13.1 => 16.13.1
      react-native: https://github.com/expo/react-native/archive/sdk-39.0.3.tar.gz => 0.63.2
      react-navigation: ^4.4.0 => 4.4.0
    Expo Workflow: managed

I’m trying to setup a CI pipeline on BitBucket. After a successful publish the app works as intended on android but on iOS crashes immediately, even before the splash screen. No errors, no logs (only the cryptic iOS crash report). If I run expo publish locally on my machine everything works fine.
Here’s the pipeline config:

image: node:alpine
definitions:
  caches:
    npm: ~/.npm
pipelines:
  branches:
    master:
      - step:
          name: Deploy to Expo
          deployment: test
          caches:
            - npm
          script:
            - apk add --no-cache bash
            - touch .env
            - echo REACT_APP_BASE_URL=$REACT_APP_BASE_URL >> .env
            - echo PRODUCTION_WEB_APP=$PRODUCTION_WEB_APP >> .env
            - echo DEV_WEB_APP="" >> .env
            - echo DEV_PN_TOKEN="" >> .env
            - npm install -g expo-cli
            - npm ci
            - npm run find-themes
            - expo login -u $EXPO_USERNAME -p $EXPO_PASSWORD
            - expo publish --non-interactive

I tried the official expo command to publish the app in the CI env:

$ npx expo publish --non-interactive

but it was failing. It was asking do I want to install the cli globally. So I installed the cli manually and run the commands as I would normally do.
Here’s the pipeline log:

+ umask 000

+ GIT_LFS_SKIP_SMUDGE=1 retry 6 git clone --branch="master" --depth 50 https://x-token-auth:$REPOSITORY_OAUTH_ACCESS_TOKEN@bitbucket.org/$BITBUCKET_REPO_FULL_NAME.git $BUILD_DIR
Cloning into '/opt/atlassian/pipelines/agent/build'...

+ git reset --hard 3aec4f38b8d0a720bd51f0c5ee78e3cc110a157a
HEAD is now at 3aec4f3 even more pipeline changes

+ git config user.name bitbucket-pipelines

+ git config user.email commits-noreply@bitbucket.org

+ git config push.default current

+ git config http.${BITBUCKET_GIT_HTTP_ORIGIN}.proxy http://localhost:29418/

+ git remote set-url origin http://bitbucket.org/$BITBUCKET_REPO_FULL_NAME

+ git reflog expire --expire=all --all

+ echo ".bitbucket/pipelines/generated" >> .git/info/exclude

+ chmod 777 $BUILD_DIR

Cache "npm": Downloading
Cache "npm": Not found

Images used:
    build : docker.io/library/node@sha256:db3f9c88223ba2491e7d2846a18efe46ecb4b60bf2bcf4b53cdcd39e5f6888b5
+ apk add --no-cache bash
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
(1/4) Installing ncurses-terminfo-base (6.1_p20200118-r4)
(2/4) Installing ncurses-libs (6.1_p20200118-r4)
(3/4) Installing readline (8.0.1-r0)
(4/4) Installing bash (5.0.11-r1)
Executing bash-5.0.11-r1.post-install
Executing busybox-1.31.1-r9.trigger
OK: 9 MiB in 20 packages

+ touch .env

+ echo REACT_APP_BASE_URL=$REACT_APP_BASE_URL >> .env

+ echo PRODUCTION_WEB_APP=$PRODUCTION_WEB_APP >> .env

+ echo DEV_WEB_APP="NONE" >> .env

+ echo DEV_PN_TOKEN="NONE" >> .env

+ npm install -g expo-cli
npm WARN deprecated urix@0.1.0: Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated hoek@4.2.1: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated topo@2.0.2: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated fsevents@1.2.13: fsevents 1 will break on node v14+ and could be using insecure binaries. Upgrade to fsevents 2.
npm WARN deprecated chokidar@2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
npm WARN deprecated har-validator@5.1.5: this library is no longer supported
npm WARN deprecated joi@11.4.0: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated @hapi/formula@2.0.0: Moved to 'npm install @sideway/formula'
npm WARN deprecated @hapi/pinpoint@2.0.0: Moved to 'npm install @sideway/pinpoint'
npm WARN deprecated @hapi/address@4.1.0: Moved to 'npm install @sideway/address'
npm WARN deprecated fsevents@1.2.13: fsevents 1 will break on node v14+ and could be using insecure binaries. Upgrade to fsevents 2.
npm WARN deprecated chokidar@2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
npm WARN deprecated @hapi/joi@17.1.1: Switch to 'npm install joi'
npm WARN deprecated core-js@2.6.12: core-js@<3 is no longer maintained and not recommended for usage due to the number of issues. Please, upgrade your dependencies to the actual version of core-js@3.

added 1900 packages, and audited 1901 packages in 47s

110 packages are looking for funding
  run `npm fund` for details

4 high severity vulnerabilities

To address all issues, run:
  npm audit fix

Run `npm audit` for details.

+ npm ci

added 2838 packages, and audited 2839 packages in 52s

32 vulnerabilities (28 low, 2 moderate, 2 high)

To address issues that do not require attention, run:
  npm audit fix

To address all issues possible (including breaking changes), run:
  npm audit fix --force

Some issues need review, and may require choosing
a different dependency.

Run `npm audit` for details.

+ npm run find-themes

> find-themes
> node ./themeFinder

Found 3 file(s). Created theme file: ./native-base-theme/loadCustomThemes.js

+ expo login -u $EXPO_USERNAME -p $EXPO_PASSWORD
[16:56:49] 
Success. You are now logged in as fuappl.

+ expo publish --non-interactive

[16:56:50] - Expo SDK: 39.0.0
[16:56:50] - Release channel: default
[16:56:50] - Workflow: Managed

[16:56:51] Building optimized bundles and generating sourcemaps...
[16:56:52] Starting Metro Bundler
[16:56:52] Building iOS bundle
[16:59:08] Finished building JavaScript bundle in 125738ms.
[16:59:08] Building Android bundle
[17:00:55] Finished building JavaScript bundle in 107110ms.
[17:00:55] Building source maps
[17:00:55] Finished building JavaScript bundle in 600ms.
[17:00:56] Finished building JavaScript bundle in 452ms.
[17:00:56] Building asset maps
[17:01:01] Finished building JavaScript bundle in 4787ms.
[17:01:06] Finished building JavaScript bundle in 4841ms.
[17:01:06] 
[17:01:06] Bundle                     Size
┌ index.ios.js          7.29 MB
├ index.android.js      7.42 MB
├ index.ios.js.map      16.7 MB
└ index.android.js.map  16.9 MB
[17:01:06] 
[17:01:06] 💡 JavaScript bundle sizes affect startup time. Learn more: https://expo.fyi/javascript-bundle-sizes
[17:01:06] 
[17:01:06] Analyzing assets
[17:01:06] Saving assets
[17:01:06] No assets changed, skipped.
[17:01:06] 
Processing asset bundle patterns:
[17:01:06] - /opt/atlassian/pipelines/agent/build/**/*
[17:01:06] 
[17:01:06] Uploading JavaScript bundles
[17:01:10] Running postPublish hook: sentry-expo/upload-sourcemaps
[17:01:11] Publish complete

[17:01:11] 📝  Manifest: https://exp.host/XXXXXX/index.exp?sdkVersion=39.0.0 Learn more: https://expo.fyi/manifest-url
[17:01:11] ⚙️   Project page: https://expo.io/XXXXXXX Learn more: https://expo.fyi/project-page

[17:01:11] Created release 1.0.36-r.M5wPLkByJ.

[17:01:18] > Found 4 release files
> Analyzing 4 sources
> Rewriting sources
> Adding source map references
> Bundled 4 files for upload
> Uploaded release files to Sentry
> File upload complete

Source Map Upload Report
  Minified Scripts
    ~/main.android.bundle (sourcemap at main.android.map)
    ~/main.ios.bundle (sourcemap at main.ios.map)
  Source Maps
    ~/main.android.map
    ~/main.ios.map

[17:01:18] Finalized release 1.0.36-r.M5wPLkByJ.

Assembling contents of new cache 'npm'
Cache "npm": Compressing
Cache "npm": Compressed in 12 seconds
Cache "npm": Uploading 183.2 MiB
Cache "npm": Uploaded in 10 seconds

Searching for test report files in directories named [test-results, failsafe-reports, test-reports, surefire-reports] down to a depth of 4
Finished scanning for test reports. Found 0 test report files.
Merged test suites, total number tests is 0, with 0 failures and 0 errors.

Does anyone has an idea what might be wrong? Or at least how to get some useful logs to debug this issue?

Are there any sentry errors logged? It’s unlikely this is a bug with Expo, and is actually a bug hidden somewhere in your app, but of course these are the most difficult bugs to root out :sweat_smile:

First thing I would recommend is combing through the iOS device logs, and from there I would suggest taking out pieces of your app one by one to try and isolate the code triggering the crash. Here’s a helpful guide on debugging expo apps- fyi/manual-debugging.md at master · expo/fyi · GitHub