`expo publish` intermittently failing with "Error: write EPIPE" or "ENOMEM: not enough memory"

Hey – we run expo publish on CircleCI and recently have been running into intermittent failures. Anyone else seeing this, and/or have any idea what may be causing this?

We run expo publish --non-interactive --max-workers 1 --release-channel <RELEASE_CHANNEL>.

I saw older posts saying this may be related to outages on Expo’s end, but not seeing anything like that on https://status.expo.io/

Also, we recently upgraded from Expo 36 to 39. Were there significant changes in how expo publish works between these versions, or in the amount of memory used?

Error output for Error: write EPIPE:

[05:10:56] - Expo SDK: 39.0.0
[05:10:56] - Release channel: <redacted>
[05:10:56] - Workflow: Managed

[05:10:56] Building optimized bundles and generating sourcemaps...
[05:10:58] Starting Metro Bundler.
[05:10:58] Building iOS bundle
[05:16:44] Finished building JavaScript bundle in 338554ms.
[05:16:44] Building Android bundle
[05:19:41] events.js:200
[05:19:41]       throw er; // Unhandled 'error' event
[05:19:41]       ^
[05:19:41] 
[05:19:41] Error: write EPIPE
[05:19:41]     at ChildProcess.target._send (internal/child_process.js:806:20)
[05:19:41]     at ChildProcess.target.send (internal/child_process.js:676:19)
[05:19:41]     at ChildProcessWorker.send (/home/circleci/project/mobile/node_modules/metro/node_modules/jest-worker/build/workers/ChildProcessWorker.js:286:17)
[05:19:41]     at WorkerPool.send (/home/circleci/project/mobile/node_modules/metro/node_modules/jest-worker/build/WorkerPool.js:32:34)
[05:19:41]     at Farm._process (/home/circleci/project/mobile/node_modules/metro/node_modules/jest-worker/build/Farm.js:129:10)
[05:19:41]     at onEnd (/home/circleci/project/mobile/node_modules/metro/node_modules/jest-worker/build/Farm.js:122:12)
[05:19:41]     at ChildProcessWorker._onProcessEnd (/home/circleci/project/mobile/node_modules/metro/node_modules/jest-worker/build/workers/ChildProcessWorker.js:280:14)
[05:19:41]     at ChildProcessWorker.onMessage (/home/circleci/project/mobile/node_modules/metro/node_modules/jest-worker/build/workers/ChildProcessWorker.js:220:14)
[05:19:41]     at ChildProcess.emit (events.js:223:5)
[05:19:41]     at emit (internal/child_process.js:876:12)
[05:19:41] Emitted 'error' event on ChildProcess instance at:
[05:19:41]     at internal/child_process.js:810:39
[05:19:41]     at processTicksAndRejections (internal/process/task_queues.js:76:11) {
[05:19:41]   errno: 'EPIPE',
[05:19:41]   code: 'EPIPE',
[05:19:41]   syscall: 'write'
[05:19:41] }
[05:19:41] 
[05:19:42] socket hang up
[05:19:42] Error: socket hang up
    at connResetException (internal/errors.js:570:14)
    at Socket.socketOnEnd (_http_client.js:440:23)
    at Socket.emit (events.js:228:7)
    at endReadableNT (_stream_readable.js:1185:12)
    at processTicksAndRejections (internal/process/task_queues.js:81:21)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Exited with code exit status 1
CircleCI received exit code 1

Error output for ENOMEM: not enough memory:

[06:41:29] Building optimized bundles and generating sourcemaps...
[06:41:30] Starting Metro Bundler.
[06:41:30] Building iOS bundle
[06:42:13] node_modules/react-native/Libraries/Components/TextInput/TextInputState.js: ENOMEM: not enough memory, read
[06:42:13] Failed building JavaScript bundle.
[06:42:13] node_modules/react-native/Libraries/Components/TextInput/TextInputState.js: ENOMEM: not enough memory, read
[06:42:14] Packager URL http://127.0.0.1:19001/node_modules/expo/AppEntry.bundle?dev=false&minify=true&hot=false&platform=ios returned unexpected code 500. Please open your project in the Expo app and see if there are any errors. Also scroll up and make sure there were no errors or warnings when opening your project.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Exited with code exit status 1
CircleCI received exit code 1

Output of expo diagnostics (run in CircleCI):

  Expo CLI 3.28.2 environment info:
    System:
      OS: Linux 4.15 Debian GNU/Linux 9 (stretch) 9 (stretch)
      Shell: 4.4.12 - /bin/bash
    Binaries:
      Node: 12.14.1 - /usr/local/bin/node
      Yarn: 1.21.1 - ~/project/mobile/node_modules/.bin/yarn
      npm: 6.13.4 - /usr/local/bin/npm
    npmPackages:
      expo: ^39.0.0 => 39.0.3 
      react: 16.13.1 => 16.13.1 
      react-dom: 16.13.1 => 16.13.1 
      react-native: https://github.com/expo/react-native/archive/sdk-39.0.3.tar.gz => 0.63.2 
      react-native-web: ~0.13.7 => 0.13.18 
      react-navigation: ~4.0.10 => 4.0.10 
    Expo Workflow: managed

Thanks!

Hey @chris-dharma, there haven’t been any significant changes to the publishing functionality between SDK36 and 39. Is it possible to update to the latest version of the cli (3.28.5) in your CI env? If so, can you see if the problem persists?

Also, do you have any issues publishing the project locally?

Cheers,
Adam

Hey @adamjnav

We’ve updated to expo-cli 3.28.5 and the problem persists :confused:

No issues publishing the project locally. The output of expo diagnostics locally is:

  Expo CLI 3.28.5 environment info:
    System:
      OS: macOS 10.15.7
      Shell: 5.7.1 - /bin/zsh
    Binaries:
      Node: 12.14.1 - ~/.nvm/versions/node/v12.14.1/bin/node
      Yarn: 1.21.1 - ~/dev/dharma/dharma-clients/mobile/node_modules/.bin/yarn
      npm: 6.13.4 - ~/.nvm/versions/node/v12.14.1/bin/npm
    Managers:
      CocoaPods: 1.9.3 - /usr/local/bin/pod
    SDKs:
      iOS SDK:
        Platforms: iOS 14.1, DriverKit 19.0, macOS 10.15, tvOS 14.0, watchOS 7.0
    IDEs:
      Android Studio: 3.5 AI-191.8026.42.35.6010548
      Xcode: 12.1/12A7403 - /usr/bin/xcodebuild
    npmPackages:
      expo: ^39.0.0 => 39.0.3
      react: 16.13.1 => 16.13.1
      react-dom: 16.13.1 => 16.13.1
      react-native: https://github.com/expo/react-native/archive/sdk-39.0.3.tar.gz => 0.63.2
      react-native-web: ~0.13.7 => 0.13.18
      react-navigation: ~4.0.10 => 4.0.10
    npmGlobalPackages:
      expo-cli: 3.28.2
    Expo Workflow: managed

is this only happening on CI for you? maybe something changed on circle’s side of things, or they’re having issues.

one thing you might want to try, which we will be rolling out to everyone in SDK 40, is to opt in to a new way of starting the packager with EXPO_USE_DEV_SERVER=1 expo publish

Yeah, it’s only happening on CI for us – we’ll follow up with Circle as well.

What brought me here first is that this started happening as soon as we upgraded to Expo 39.

We’ll give EXPO_USE_DEV_SERVER=1 expo publish a go and report back!

Unfortunately, EXPO_USE_DEV_SERVER=1 expo publish doesn’t seem to solve the problem.

I’ve also posted in Circle’s forum (currently under spam review; hopefully reviewed by their staff soon).

The failures seem to be most often (though not always) related to memory errors. Is there anything that’s changed between Expo 36 -> 39 concerning memory usage?

Also, we typically publish to several release channels at the same time (from separate machines). This previously never caused us any issues. Could this affect whether publishes succeed?

It seems like the issue may be that expo publish is running out of memory.

We’re running expo publish in a 2 CPU / 4 GB RAM Docker environment. Would that cause us any memory usage issues?

Hey @adamjnav @notbrent – we’re seeing that expo publish is consistently using all 4GB of RAM available to it in CircleCI, and often failing because it is exceeding the available 4GB of memory.

We’ve tried adjusting expo’s --max-workers (currently set to 1) as well as node’s --max-old-space-size (currently not adjusted), all to no avail.

Is it typical of expo publish to use that much memory? Are there common mistakes that would cause an app to require that much memory to publish?

Thanks for the tips so far – I really appreciate you guys sharing your time! :hugs:

i can’t say that i’ve ever heard of this happening before. are you maybe using some babel transform that could be misbehaving? could you try running the same circle workflow on a blank new project to see if you’re experiencing the issue?