.hdr files from equirectangular images?


#1

My Ricoh Theta spherical panorama camera makes equirectangular images of its environment. It also allows exposure bracketing, albeit with a twist: The geometry in the images does not overlap 100%, because the camera’s built-in stitching algorithm introduces local distortions which vary depending on exposure. But that should be a minor issue. So:

What tools / workflows can I use to create .hdr files from a set of exposure bracketed equirectangular images?

Update: Documentation

Hugin documentation gives a list of tools for creating .hdr files.They mention Photoshop, Photomatix, FDRTools, and pfstools.

Photomatix offers export to .hdr if one checks “Show 32-bit unprocessed image” upon loading bracketed photos. However, with Photomatix 5.0, I had the problem that one HDRI turned out very dark.

Update: Workflow

What follows is my workflow, which resulted from the discussion in this thread.

  1. Create a series of exposure bracketed images, for example using my app Theta Control.

  2. Download the images from the camera, and create a backup: The original images will be edited.

  3. Using exiftool on the command line, give all images the same orientation meta data:

    >exiftool -overwrite_original -tagsFromFile R0010030.jpg -MakerNotes *.jpg
    

    This step is necessary because the Theta, even when sitting still on a tripod, does not always record the same orientation information.

  4. Open every image in the Theta desktop app and export it with top/bottom correction.

  5. Load the corrected images into FDRTools Basic 2.6. Save to .hdr. Optionally, create a tone-mapped preview JPEG.

    There are various other tools for creating the .hdr file. Due to lack of experience I cannot recommend one over another. @gpalov did a comparison: HDR Software - the panoramic overview

  6. Open .hdr file in IrfanView 4, to check brightness.


(Brian James) #2

Check out http://www.hdrlabs.com/ and I’d recommend Christian Bloch’s book too which is listed there ( I actually need to pick up the second version). I learned everything I know about HDRI photography from him!


#3

Thanks for the links! I guess with the Theta I could skip many steps: There is no stitching necessary as that’s done in camera.


(Brian James) #4

I’d be interested in your results with the Theta. Stitching is complicated and even more so when merging brackets. There are many variables involved such as the nodal point of the lens, distance from surrounding objects and lens length to name a few.


#5

Quick’n’dirty test:

http://www.food4rhino.com/materials-textures/private-rose-baden-baden

I imported three bracketed equirectangular Theta images into Photomatix Pro and exported to .hdr. There has been no additional processing. The preview image is a screenshot from Photomatix Pro.

Note that while the Theta’s exposure compensation can be adjusted, there is no control over how it is achieved. The camera may vary shutter speed and/or ISO. The lens’ aperture apparently is fixed.


(Brian James) #6

I’m denied access to the link on F4R even though I’m logged in… but I am curious how it handles the poles of the sphere.


#7

The above link should work now. I also uploaded a new HDRI, shot yesterday, this time with a bit less contrast:

http://www.food4rhino.com/materials-textures/meadow-lichtentaler-allee-baden-baden

If there wasn’t the 2 MB upload limit on discourse.mcneel.com, I would be happy to share the source images, which are slightly above that limit.


Is the 2Mb limit on uploading files new?
(Brian James) #8

Nice! I lowered the multiplier and saturation and it works pretty well here in a quick tested using Neon with Brazil, thanks for sharing it!

The horizon line is uneven and the bottom of the pano looks like it is getting a reflection off of something maybe a tripod or the camera lens bezel at a guess. But aside from that it looks good to me for a fast environment. I would next test how it does with objects closer to the camera such as your typical kitchen. This is usually where auto-stitching algorithms have the most trouble.


#9

Although simple, I really enjoy the rendering!

The horizon is automatically leveled using sensors in the camera. Also note that the meadow is not flat.

What you see is the camera itself! As a tripod I used the Novoflex STASET rod system, stuck into the ground. It’s invisible because the footprint is less than that of the camera.

Already here the auto-stitching algorithm had trouble: Some of the distant trees don’t overlap, although there has been no wind and the tripod was steady. I could have enabled de-ghosting in Photomatix, but I didn’t since I’m worried about loss of color depth: I can imagine that the de-ghosting algorithm works by discarding information from individual images.

Just checked: Still there is the 2 MiB limit, and so I cannot upload the individual images.

While waiting for the possibility to upload the source images, I uploaded the EV+0 exposure for interactive viewing: https://theta360.com/s/Lbv


#11

http://www.food4rhino.com/materials-textures/bathroom-mixed-lighting-indoor-outdoor

No kitchen, but a bathroom. How’s that?

Inside it was quite dark, and so I turned on the ceiling light, which is on the warm side. Of course that’s not such a great idea. However, 1. otherwise the Theta would chose high ISO, and 2. it demonstrates how well the Theta’s auto white balance works. I also tried turning on the fluorescent lamp above the mirror, but that gave me zebra stripes in the dark images, i.e. the shots with high shutter speed.


(Brian James) #13

That room’s a good test and the bathroom’s not bad from a stitching point of view especially considering the proximity of the surroundings. I really only picked up on a slight mismatch of the framing around the door.

The main issue for me with this one is that the equirectangular projection is not what you see in the Theta online viewer. I tested it in Octane, Maxwell and Brazil/Neon and all three apply it to an environment as you can see in the back of the two samples here. Adjustment of the horizon or equator of the pano is usually done during the stitching phase so I’m not sure what options you have there. You’ll want a nice horizontal line across the middle of the image with the distortion happening at the top and bottom of the image (i.e. the poles of the sphere)

One other small problem I found was that the intensity had to be increased by 50-100x depending on the renderer used. Not a big deal but if you don’t know how to do that, you wouldn’t be able to use the HDRI.


#14

The stitching algorithm is a bit unpredictable. In other cases, I didn’t see mismatches at that distance.

I had attached the camera on a pole to the small table, and the pole was oriented diagonally. The easiest solution is probably to orient the camera upright on a tripod. Alternatively it should be possible to remap the source images using Hugin, though possibly with loss of quality.

This I don’t understand. To me the exposure of the source images looks correct, and the rest was done by Photomatix. I am not aware of options for adjusting intensity when creating a .hdr file.


#15

I just remembered that the Theta desktop viewer can export with top/bottom correction!

The result are leveled JPEGs. I just tried it, but then the images don’t overlap anymore. Apparently, the sensor data does not match exactly between images. Orientation is stored in a proprietary EXIF extension. So I tried copying EXIF data from one image to another. Unfortunately, none of the tools that I tried were able to handle the data. Here’s what exiftool says:

>exiftool -TagsFromFile EV+0.jpg EV+2.jpg
Warning: Invalid RicohSubdir data - EV+0.jpg
    1 image files updated

(Brian James) #16

I don’t have much experience with editing EXIF data to help with that bit but thanks for explaining what you found. I’m curious about the Theta and if their developers are responsive you might try asking them.

It’s interesting that the exterior scene didn’t need the multiplier tweak to work well. It might be due to the ISO setting or white balance that you mentioned as being decided for you. In any case, that one’s an easy fix. Open it up in any 32 bit capable image editor and adjust the exposure. Then save it out as an HDR again. One of my favorite things about this format is that pixel luminance is independent of pixel resolution. In short, here’s a wicked small one that works.

badezimmer_adjusted.zip (1.3 MB)


#17

From my experience, Ricoh isn’t overly responsive concerning internals of the camera. I once asked them for information concerning remote controlling the Theta, but they said that there’s nothing public. Then I found that various Japanese hackers have published quite a lot about the Theta. This information allowed me to get started writing my app Theta Control.

Anyhow, concerning the proprietary EXIF extension, someone seems to have decoded it, at least in part:

http://xanxys.hatenablog.jp/entry/20131110/1384094832#20131110f1

Based on that documentation, someone else wrote an online tool to get orientation information stored in JPEGs:

Possibly. For the EV+2 image, the camera went from ISO 200 to ISO 560. Unfortunately, there is no way to set ISO to a fixed value and opt for lower shutter speeds.


#18

Finally, the images straight off camera:


#19

I discovered that the latest exiftool, version 9.68, now allows copying the Ricoh specific meta data, which includes information about the camera and about orientation of pictures:

[MakerNotes]    Maker Note Type                 : Rdc
[MakerNotes]    Firmware Version                : 1.30
[MakerNotes]    Serial Number                   : (00000000)00110463
[MakerNotes]    Recording Format                : JPEG
[MakerNotes]    Accelerometer                   : 325 -41
[MakerNotes]    Compass                         : 337.5

This allowed me to correct the orientation of my bathroom panorama. Corrected equirectangular images:





Because of the problem with low brightness, I switched from Photomatix Pro to FDRTools Basic. The latter gives more control for creating an .hdr, and the default result looks OK, at least to my untrained eye. Possibly a lot can be improved by by spending actual thought on what options there are for merging exposures.

All my food4Rhino submission are now orientation corrected:

I added the workflow to my initial post in this thread.


#20

Hi Felix, how did you corrected the 2:1 images? What software did you use.
I have not tested Theta for 32 bit environmental lighting, as I think noise is too much and it can give uncorrect results for lighting. still haven’t tested that though. For my 32 bit environments I use Nikon DSLR and Promote remote control 7x1.7 EVs or 5x2EVs


#21

@gpalov See my initial post. Here I explain that I used the Theta desktop app to fix orientation (top/bottom correction), after giving all images the same orientation meta data.

Let me know, if you need more information.

The problem with the Theta, of course, is that it needs a lot of light, or else it will choose high ISO especially for the overexposed images in the series. High ISO images, obviously, don’t really add much information. In fact, it is possible that with ISO 200, the camera simply pushes the exposure in software by one stop, causing banding and noise. As the Theta is likely doing that on raw data, there still is some information gain compared to just pushing exposure of an ISO 100 JPEG.