Questions about ExponentAudio API

Expo SDK
1

First off, thanks for adding this feature, and making it so simple to use.

My main question is this, I realize that the audio player was deliberately designed to stop playing when the conditions change, but I am wondering if it is possible at this point to tweak those settings?

I am creating an audio player app and ideally I’d like the app to keep playing the music and continue down the playlist when the app is in the background and/or the device is locked. It’s a not a huge deal as this is just going to be a demo app for me to show off in interviews, but in the future I’d like to expand the app into something more fully functional and it’d be great if I could do that without ejecting from Expo and using another audio package.

It seems that playing media files in the mobile environment is quite a bit more complex than I had originally assumed.

Question two is, will the app start playing the audio file before it is fully loaded into the cache? I know I can find out for myself but it’d be nice to know ahead of time what to expect.

And sort of off topic and no required answer, but I am curious… but has anyone tried installing and using the React-Howler package in an Expo app? I know that in the future, I want to use some packed wrapper for the Web Audio Api, but it is frustratingly difficult to find any information on the usage of the API outside of the web context. I know it can play audio without the HTML5

Ultimately I want to grab frequency data from the audio file and display it on the screen using D3 to to make some fancy visuals, and ensure that the audio player transitions from one track into the next seamlessly.

I’m also curious how ExponentAudio becomes one of the NativeModules in the Expo XDE.

I know this is a long post and I’ve opened a deep rabbit hole… I’d be perfectly happy with a simple and direct answer to my first question. Thanks!

Jack

2

Also, in using the demo playlist app on mu iPhone, it seems that the audio won’t play through the phone’s speakers. The track will be “playing”, but no sound, and it works fine with my bluetooth headphones.

Is that a deliberate feature, or a bug?

3

Sound was off… yup, that. Did I mention the sound of my phone is always off? Imagine that.

Btw throwing me into the ring against ES7 when I’ve just managed to stay standing with ES6 is really putting me through the ringer. I guess we’ll call it a character building experience eh? :sweat_smile:

BTW it looks like the react-howler module is throwing out html as soon as it’s imported so that’s a big NO on that one. Maybe just pulling a few individual methods our of the Howler package itself… The web audio API documentation is big and scary and hard to understand. Howler says it’s pure JS and has no dependencies, which makes it sound like it could work, but what is it exactly that is delivering the generated waveform to the speakers? Even if it is on a website? I know I’m just rambling to myself at this point. It’d just be really cool if I could get all the features I want from this thing without ejecting:

  1. Password login/account a la Meteor - verified and working, so far

  2. The ability to take huge lists of tracks (from a site like archive.org) and make smaller lists of tracks that you can save in your account as “collections.” - Slightly more complicated, but a good opportunity to learn redux maybe? I guess the pub/sub from meteor would work fine until there is a volume of users.

  3. The ability to make playlists on the fly from said collections, play tracks continuously from that playlist (gapless, which I think is possible with what Expo already has set up, since I could make two tracks play simultaneously, well, the same track, so it just needs a bit of logic thrown in there), save the playlist to your account, and download the tracks onto your device for offline listening (assets folder?).

  4. Fancy visualizations of the music using D3 to fill up all the extra screen real estate, and add one more key word to my resume.

Stretch goal… shareable playlists that update in real-time on all clients as people add tracks to them.

4

Hi Jack!!

Thank you so much for your detailed query regarding the Audio API – it’s fantastic to hear so much information from someone using our product!

I’m going to respond linearly to your messages.

[1]

My main question is this, I realize that the audio player was deliberately designed to stop playing when the conditions change, but I am wondering if it is possible at this point to tweak those settings?

I’m assuming that you are referring to this sentence in the docs:

“We designed the Audio API to be as non-intrusive as possible, so audio will automatically stop if other audio starts playing on the device, if the device is locked, if the app / experience is backgrounded, or if headphones / bluetooth audio devices are disconnected.”

This will be getting somewhat relaxed in the upcoming SDK release, but we will still not support backgrounding. We realize that it is a huge desire of many of our users, and will hopefully have better news for you on that front soon.

It seems that playing media files in the mobile environment is quite a bit more complex than I had originally assumed.

Yes! You’re unfortunately right, but that’s a problem we are working really hard to solve.

[2]

Question two is, will the app start playing the audio file before it is fully loaded into the cache? I know I can find out for myself but it’d be nice to know ahead of time what to expect.

Yes, the library will by default start playing the audio file before it is fully loaded in cache if you call playAsync() and the audio file is only partially loaded. We are working now on adding more state information to the audio API regarding buffering and loaded duration. If you would like, let me know if there is a specific API call you would like to see regarding this and I can try to ensure that we implement it.

[3]

I want to use some packed wrapper for the Web Audio Api, but it is frustratingly difficult to find any information on the usage of the API outside of the web context.

Using web audio is not possible on Expo at the moment, but we are looking into it. Your musings in your third message are spot on regarding a possible implementation-- fingers crossed for an update soon!

[4]

I want to grab frequency data from the audio file and display it on the screen

This is not possible with the current implementation, but this is another thing that we are looking into!!

[5]

I’m also curious how ExponentAudio becomes one of the NativeModules in the Expo XDE.

It was developed internally by us at Expo!

[6] Feature list

This sounds like a great app! It seems to me that (1), (2), and (3) are perfectly possible with our API as it stands, but as I mentioned earlier, (4) is currently not possible.

Let me know if you have any more questions or clarifications-- I am available for real time chat pretty often on Expo Community Slack at @greg if you want to talk there as well!

Greg