This shows a few interesting experiments with UMAJIN.
First is projecting a 3d Mesh from a homography extracted from a scene using a FAST feature detector and a FLANN high dimension matching algorithm to match the features we had previously captured from our natural image. That means you can learn and then find any image with sufficiently interesting corners in it.
The next experiment is using the incoming video stream as a pseudo environment map for the specular elements of the scene. This means reflections on our 3D object look more natural as they are in the same tones as the rest of the image.
Finally tweaking our paint and material shaders for running on mobile devices like the android device in this case or iOS in real time.