About an execution error

[Segfred]

Hello, Haarbach, I am Robert. I find your work great but I encounter a problem when running your code. I am using Mac Yosemite and following exactly your instructions about how to run the code. The thing is when I run ./ppf-reconstruction, everything is OK and I get a sparse reconstruction. But when I run ./ppf_reconstruction -dir ../samples/Bunny_Sphere/ -knn 5 -nFrames 5, the program just aborts and shows that "libc++abi.dylib: terminating with uncaught exception of type Iex::EnosysExc: Cannot initialize semaphore (Function not implemented)." I am wondering if you know how to fix that problem?
With Best!

[adrelino]

Hi Robert

sorry for my late reply, I was quite busy recently.

Even though I could not yet reproduce your bug, I think we can narrow it down to what is actually failing.

The only difference between running
./ppf-reconstruction (wich defaults to -dir ../samples/Bunny_RealData/)
and
./ppf_reconstruction -dir ../samples/Bunny_Sphere/ -knn 5 -nFrames 5
is that
the first one (that works for you) uses real data and real depth maps grabbed from Kinect and saved in .png format
and the latter one (which fails for you) uses synthetic data and generated depth maps from Blender saved in .exr format.

I also googled a little bit for your error, which hinted towards OpenCV in combination with OpenEXR loading issues.

To be sure, could you check out my code again because I added some more verbosity output before and after calls to cv::imread (which loads the .png or .exr files). If everything there works fine, you should now get the following output for each depth image loaded:

./ppf_reconstruction -dir ../samples/Bunny_Sphere/ -knn 5 -nFrames 5
 PointCloud fromDepthMap constructor
LoadingSaving::loadPointCloudFromDepthMap before cv::imread ../samples/Bunny_Sphere//depth_000000.exr
LoadingSaving::loadPointCloudFromDepthMap after cv::imread ../samples/Bunny_Sphere//depth_000000.exr     type: CV_32FC3 channels:3
DownSampled 11215->109 pts with voxelSize:0.015 minPtsPerVoxel:20

if you only see
LoadingSaving::loadPointCloudFromDepthMap before cv::imread
but not the log after calling the function, we know it has to do with OpenCV + OpenEXR

could you please check that for me?

Also, please send me the output of running
brew info opencv
maybe something changed in the brew opencv package in the meantime which caused the bug

[Segfred]

Hi, Haarbach. Thank you very much for your detailed explanation and I have retried your code in another iMac( from my laboratory). It turns out that both of your commands can run perfectly on the iMac.

Part1: on the Macbook (where the second command cannot run)
But as for my own Macbook, I even cannot see the following result
「LoadingSaving::loadPointCloudFromDepthMap before cv::imread 」All I can see is 「PointCloud fromDepthMap constructor」,then the program just aborts.
Besides, I installed the opencv 2.4 by source before and it exists in the usr/local/include address now, instead of in usr/local/Cellar (default address by brew install opencv). When I type brew info opencv, what I see is as below: (it seems my opencv cannot be found by this program).

cmd: brew info opencv
「homebrew/science/opencv: stable 2.4.12 (bottled), HEAD
Open source computer vision library
http://opencv.org/
Not installed
From: https://github.com/homebrew/homebrew-science/blob/master/opencv.rb
==> Dependencies
Build: cmake ✔, pkg-config ✘
Required: jpeg ✘, libpng ✘, libtiff ✘
Recommended: eigen ✔, openexr ✘, homebrew/python/numpy ✘
Optional: gstreamer ✘, jasper ✘, libdc1394 ✘, openni ✘, qt ✘, tbb ✘, vtk ✘, ffmpeg ✘
==> Options
--32-bit
Build 32-bit only
--c++11
Build using C++11 mode
--universal
Build a universal binary
--with-cuda
Build with CUDA support
--with-ffmpeg
Build with ffmpeg support
--with-gstreamer
Build with gstreamer support
--with-jasper
Build with jasper support
--with-java
Build with Java support
--with-libdc1394
Build with libdc1394 support
--with-opengl
Build with OpenGL support
--with-openni
Build with openni support
--with-qt
Build the Qt4 backend to HighGUI
--with-quicktime
Use QuickTime for Video I/O instead of QTKit
--with-tbb
Enable parallel code in OpenCV using Intel TBB
--with-vtk
Build with vtk support
--with-ximea
Build with XIMEA support
--without-eigen
Build without eigen support
--without-numpy
Use a numpy you've installed yourself instead of a Homebrew-packaged numpy
--without-opencl
Disable GPU code in OpenCV using OpenCL
--without-openexr
Build without openexr support
--without-python
Build without Python support
--without-tests
Build without accuracy & performance tests
--HEAD
Install HEAD version
AsmitadeMacBook-Pro:build asmita$ 」

In usr/local/Cellar:
there exist brew-cask, cmake, eigen, free glut
In usr/local/include:
there exist EXTERNAL, eigen3, opencv, pcl-1.8, GL, g2o, OPENCV2

Sorry about the mess in my Macbook since I installed many libraries before.
Then I switch to the iMac in my laboratory and got the following result

Part2: on the iMac (where both commands are OK)
cmd: brew info opencv
homebrew/science/opencv: stable 2.4.12 (bottled), HEAD
Open source computer vision library
http://opencv.org/
/usr/local/Cellar/opencv/2.4.12_2 (225 files, 35.6M) *
Poured from bottle
From: https://github.com/Homebrew/homebrew-science/blob/master/opencv.rb
==> Dependencies
Build: cmake ✔, pkg-config ✘
Required: jpeg ✔, libpng ✔, libtiff ✔
Recommended: eigen ✔, openexr ✔, homebrew/python/numpy ✔
Optional: gstreamer ✘, jasper ✘, libdc1394 ✘, openni ✘, qt ✘, tbb ✔, vtk ✘, ffmpeg ✘
==> Options
--32-bit
Build 32-bit only
--c++11
Build using C++11 mode
--universal
Build a universal binary
--with-cuda
Build with CUDA support
--with-ffmpeg
Build with ffmpeg support
--with-gstreamer
Build with gstreamer support
--with-jasper
Build with jasper support
--with-java
Build with Java support
--with-libdc1394
Build with libdc1394 support
--with-opengl
Build with OpenGL support
--with-openni
Build with openni support
--with-qt
Build the Qt4 backend to HighGUI
--with-quicktime
Use QuickTime for Video I/O instead of QTKit
--with-tbb
Enable parallel code in OpenCV using Intel TBB
--with-vtk
Build with vtk support
--with-ximea
Build with XIMEA support
--without-eigen
Build without eigen support
--without-numpy
Use a numpy you've installed yourself instead of a Homebrew-packaged numpy
--without-opencl
Disable GPU code in OpenCV using OpenCL
--without-openexr
Build without openexr support
--without-python
Build without Python support
--without-tests
Build without accuracy & performance tests
--HEAD
Install HEAD version
==> Caveats
Python modules have been installed and Homebrew's site-packages is not in your Python sys.path, so you will not be able to import the modules this formula installed. If you plan to develop with these modules, please run:
mkdir -p /Users/lab/Library/Python/2.7/lib/python/site-packages
echo 'import site; site.addsitedir("/usr/local/lib/python2.7/site-packages")' >> /Users/lab/Library/Python/2.7/lib/python/site-packages/homebrew.pth

By the way, I am wondering have you ever thought about taking ordinary pictures instead of range images as input? And have you ever thought about model fitting (surface reconstruction) as well as texture mapping after getting the dense point cloud? Could you be so kind to tell me your recent focus?

Best Wishes!
Robert

[lironesamoun]

I got the same issue concerning the command
./ppf_reconstruction -dir ../samples/Bunny_Sphere -knn 5 -nFrames 5

I got this message :

Found 42 files in ../samples/Bunny_Sphere that match the pattern: depth.png|.jpg|.tif|.exr|.bmp . first is: ../samples/Bunny_Sphere/depth_000000.exr
Found 0 files in ../samples/Bunny_Sphere that match the pattern: mask.png|.jpg|.tif|.exr|.bmp .
Found 1 files in ../samples/Bunny_Sphere that match the pattern: Intrinsic_.txt|.xyz . first is: ../samples/Bunny_Sphere/Intrinsic.txt
Found 42 files in ../samples/Bunny_Sphere that match the pattern: pose_.txt|.xyz . first is: ../samples/Bunny_Sphere/poses_000000.txt
loaded 42 poses from dir ../samples/Bunny_Sphere with prefix pose
PointCloud fromDepthMap constructor
LoadingSaving::loadPointCloudFromDepthMap before cv::imread ../samples/Bunny_Sphere/depth_000000.exr
libc++abi.dylib: terminating with uncaught exception of type Iex::EnosysExc: Cannot initialize semaphore (Function not implemented)

Concerning the command line
brew info opencv

I got

homebrew/science/opencv: stable 2.4.13 (bottled), HEAD
Open source computer vision library
http://opencv.org/
/usr/local/Cellar/opencv/2.4.12_2 (345 files, 108.2M)
Poured from bottle on 2016-02-15 at 09:47:01
/usr/local/Cellar/opencv/2.4.13 (276 files, 35.8M)
Poured from bottle on 2016-07-01 at 09:50:56
From: https://github.com/Homebrew/homebrew-science/blob/master/opencv.rb
==> Dependencies
Build: cmake ✔, pkg-config ✔
Required: jpeg ✔, libpng ✔, libtiff ✔
Recommended: eigen ✔, openexr ✔, homebrew/python/numpy ✔
Optional: gstreamer ✔, jasper ✘, libdc1394 ✘, openni ✘, qt ✔, tbb ✔, vtk ✔, ffmpeg ✘

Segfred rise an interesting question ! Is it possible to the reconstruction using view from camera ? Thank

[adrelino]

Hi

The error

Iex::EnosysExc

Is defined by OpenEXR.
http://stackoverflow.com/questions/22688117/writing-a-float-image-in-opencv-with-pixel-values-bigger-than-1/22689373#22689373
https://github.com/openexr/openexr/blob/master/IlmBase/Iex/IexErrnoExc.h#L129

Have you tried reinstalling opencv and openexr?
brew uninstall opencv openexr && brew install openexr opencv && brew doctor
Maybe something went wrong when linking them (e.g. maybe they were compiled using different compilers and std libs).

[adrelino]

By the way, I am wondering have you ever thought about taking ordinary pictures instead of range images as input?
Segfred rise an interesting question ! Is it possible to the reconstruction using view from camera ? Thank

No, it is a purely geometry based approach, thus range images are used and unprojected using the pinhole camera model to get points in 3d space. If you have a stereo camera system one could use stereo vision techniques to get these 3d points from 2 RGB images, but that was out of the scope here.

And have you ever thought about model fitting () as well as texture mapping after getting the dense point cloud? Could you be so kind to tell me your recent focus?

The final result of my pipeline is just the optimized trajectory / camera pose of each frame.

The next steps in a standard scan-to-model / surface reconstruction pipeline would be to:

1. project the points from each scan into a common reference frame.
2. Build a volumetric model, e.g. using a signed distance volume
3. Extract the isocontour and build a triangle mesh using the Marching Cubes Algorithm
4. use the RGB images and some sort of ray-casting to color these triangles. Need a smart way to integrate the color of a point in 3D when seen multiple times from different angles and lightning conditions.

But this is not in my focus right now, there are already lots of solutions who deal with this.

[swframe]

I had this error and the info here helped.
I thought I would add a little extra info to make it more clear how to fix the problem.
If you build opencv you have to disable the posix semaphore api calls.
The following page explains how to build opencv on mac os:
http://www.pyimagesearch.com/2015/06/15/install-opencv-3-0-and-python-2-7-on-osx/
It gives a cmake command and you need to add "-D HAVE_POSIX_SEMAPHORES=0"