Problem solved. Before sending the contact object to my component that renders it with FlatList, I parsed it into a simpler object with just the fields I needed (because, as per here, using the API fields parameter does nothing, and getContactsAsync always returns everything).
Now the FlatList is rendered very fast in its entirety. I updated my snack to show the new code.
Not sure how to mark a topic as closed, but this one is.