Unable to apply image to a texture to be used in ThreeViews

#1

I am working on implementing a texture from an image using createTHREEViewClass. I want to map this texture to a materaial to be used on the inside of a sphere mesh to make a panorama.

I am using Node v7.10.0

Here are my project dependencies:

"@expo/vector-icons": "^5.0.0",
"directory": "^0.1.0",
"expo": "^16.0.0",
"lodash": "^4.17.4",
"prop-types": "^15.5.9",
"react": "16.0.0-alpha.6",
"react-native": "^0.43.4",
"react-navigation": "^1.0.0-beta.9",
"react-redux": "^5.0.4",
"redux-logger": "^3.0.1",
"redux-multi": "^0.1.12",
"redux-thunk": "^2.2.0",
"three": "^0.85.2"

Here is the project path for the image I am trying to load:

src/components/TestView/equirectangle.jpg

Here is the app’s folder structure:

|- src
  |- components
    |- TestView
      |- equirectange.jpg
      |- TestThreeJS.js
|- App.js

Here is my class:

import React, {Component} from 'react'
import {Dimensions} from 'react-native'
import Expo from 'expo'
import * as THREE from 'three'

const {
  Matrix4,
  MeshBasicMaterial,
  PerspectiveCamera,
  Scene,
  SphereGeometry,
  Vector3
} = THREE

export default class TestThreeJS extends Component {
  ThreeView = Expo.createTHREEViewClass(THREE)
  
  componentWillMount () {
    const {width, height} = Dimensions.get('window')

    this.scene = new Scene()
    this.scene.name = 'scene'

    this.camera = new PerspectiveCamera(75, width / height, 1, 1000)
    this.camera.name = 'perspective-camera'
    this.camera.target = new Vector3(0, 0, 0)

    this.scene.add(this.camera)

    const panoGeometry = new SphereGeometry(500, 60, 40)
    panoGeometry.applyMatrix(new Matrix4().makeScale(-1, 1, 1))

    const panoMaterial = loadImageMaterial('pano', this.ThreeView)

    const panoMesh = new THREE.Mesh(panoGeometry, panoMaterial)
    panoMesh.material.side = THREE.DoubleSide
    panoMesh.name = 'pano'

    this.scene.add(panoMesh)
  }

  update = (dt) => {
    
  }

  render () {
    const {ThreeView, update, scene, camera} = this

    return (
      <ThreeView
        style={{flex: 1}}
        backgroundColor={'#FFFFFF'}
        scene={scene}
        camera={camera}
        tick={update}
      />
    )
  }
}

// The code below was suggested by Jim

const modules = {
  'pano': require('./equirectangle.jpg')
}

const Assets = Object.assign({},
  ...Object.keys(modules).map(name => ({
    [name]: Expo.Asset.fromModule(modules[name])
  }))
)

const loadImageMaterial = (assetName, threeView) => {
  const asset = Assets[assetName]
  asset.localUri = asset.uri
  const texture = threeView.textureFromAsset(asset)
  texture.minFilter = texture.magFilter = THREE.NearestFilter
  texture.needsUpdate = true

  const material = new MeshBasicMaterial({
    map: texture,
    transparent: true // Use the image's alpha channel for alpha.
  })

  return material
}

I got an error when running the aboce code: Asset ./equirectangle.jpg needs to be downloaded before being used as an OpenGL texture.

It is thrown from createTHREEViewClass ln 39. The asset I am passing does not have a localUri property, just a uri.

I updated the asset object to set the localUri property from the uri and here is my asset object logged out in textureFromAsset:

// asset object
{
  "name": "equirectangle",
  "type": "jpg",
  "hash": "27bb232b523c0038ce850bd343e9366e",
  "uri": "http://localhost:19001/assets/src/components/TestView/equirectangle.jpg?platform=ios&hash=27bb232b523c0038ce850bd343e9366e",
  "width": 5376,
  "height": 2688,
  "downloading": false,
  "downloaded": false,
  "downloadCallbacks": [],
  "localUri": "http://localhost:19001/assets/src/components/TestView/equirectangle.jpg?platform=ios&hash=27bb232b523c0038ce850bd343e9366e"
}

I now get an error Cant find variable: document

I checked the meta object that Assets.fromModule is returning and here is what I get:

// meta obj
{
  "__packager_asset": true,
  "httpServerLocation": "/assets/src/components/TestView",
  "width": 5376,
  "height": 2688,
  "scales": [
    1
  ],
  "hash": "27bb232b523c0038ce850bd343e9366e",
  "name": "equirectangle",
  "type": "jpg",
  "fileHashes": [
    "27bb232b523c0038ce850bd343e9366e"
  ]
}

Which is turned into the asset I listed above.

This is where I am stuck. A couple of things of interest:

  1. Expo.Asset.fromModule() does not set a localUri property even though it is expected in threeView.textureFromAsset()
  2. The uri and now localUri path seem to be looking in an assets directory that does not exist. I looked into Asset.fromModule and its using the react-native AssetRegistry which must be making asset locations in an asset directory
0 Likes

#2

Hi Jordan! I think the issue is that you didn’t download the assets in modules. Running this in componentWillMount should fix this issue:

componentWillMount() {
   await this.load();
    ...other stuff
  }
async load() {
    try {
      // Load assets
      await Promise.all(
        Object.keys(Assets).map(name => Assets[name].downloadAsync())
      );
    } catch (e) {
      Alert.alert('Error when loading', e.message);
    }
  }
0 Likes

#3

Hi!

Thank you for the reply :slight_smile:

I am on the very new side of async and await so apologies. When I run your code snippet, the await in componentWillMount throws an error: “await is a reserved word” in my editor as well as the ios simulator.

The await within the load function is works fine though. If I pull await from within willMount I get the the "Asset ‘fileName’ needs to be downloaded before being used as and OpenGL texture.

I feel like this is soooo close.

0 Likes

#4

Apologies, made a mistake here. Putting your other calls under the await Promise.all should fix this. This should work (missing a line since cursor is blocking):

componentWillMount() {
   this.load();
}
async load() {
    try {
      // Load assets
      await Promise.all(
        Object.keys(Assets).map(name => Assets[name].downloadAsync())
      );
     // put your other stuff here
     this.loadPano();
     this.loadHotspots();
    } catch (e) {
      Alert.alert('Error when loading', e.message);
    }
  }
0 Likes

#5

Error gone! Thanks.

Unfortunately nothing is now rendering to the ThreeView.

I am going to put together a GH repo for this and add it to this forum post.

0 Likes

#6

Here is the git repo where I am hitting errors:

So the wall currently is in App.js ln 75. I think I am able to create the texture, apply it to a material, and even create a mesh from it. But… when I go to add it to the scene, I get an error saying “Can’t find variable: document” It then references many three.js files in the stack trace

As always, very grateful for your time and ideas.

0 Likes

#7

Yeah seems like it’s trying to make some web-related calls here that use the DOM. You could try to see what the function on document it is that it’s trying to call and see if you can polyfill that. Try doing global.document = {}; and seeing what the next error is that you get. :open_mouth:

0 Likes

#8

So this has come full circle. The initial issue I had with mapping an image to a texture to a material was the use of document.createElementNS by THREEJS in the loading process of an image. The clamptoMaxSize function creates a canvas to make sure the image does not exceed a maximum size. I went to where is was being called (three.js uploadTexture) and just use the image provided instead of the clamped image

var image = texture.image // clampToMaxSize( texture.image, capabilities.maxTextureSize );

The document error goes away but now the entire expo app just crashes. I have put a week into this issue and I need an honest answer here because I’m going to get in trouble pretty quick. Does EXPO support loading images into textures to be used in a material? If so, is there any way you can use the super simple repo above to demonstrate this feature?

Im at that critical crossroad where I may need to drop this effort and find something else. You guys have been great and I hope it works :slight_smile:

Thanks.

0 Likes

#9

@jordanpapaleo is this on iOS or Android?

0 Likes

#10

If you post a minimal project that reproduces this error on GitHub or paste its code here or make a https://snack.expo.io of it, that could be a quick way to track down the issue!

0 Likes

#11

There is a github link at post 5/10 which is the bug :slight_smile:

I dont think I can do the snack.expo.io because I needed to install threejs as a project dependency.

0 Likes

#12

Ok great thanks! I’ll try this out tonight or in the day tomorrow!

1 Like

#13

Greetings, found any answer to this yet?

I tried overwriting the createElementNS function like so:

var DOMParser = require('react-native-html-parser').DOMParser;
var myDomParser = new DOMParser();
window.document = {};

window.document.createElementNS = (x,y) => {
    if (y === "img") {
        let doc = myDomParser.parseFromString("<html><head></head><body><img class='image' src=''></img></body></html>");
        return doc.getElementsByAttribute("class", "image")[0];
    }
}

But i get: TypeError: image.addEventListener is not a function

So this is probably not an option

0 Likes

#14

“Textured Solid” doesn’t show the material texture, but rather the image you assign to it in the UV/Image editor. Also if the texture doesn’t render, it probably means there are no UV coords or its using a different mapping method like “Generated”.
https://downloader.vip/minecraft-pocket-edition/
https://downloader.vip/counter-strike/

0 Likes