textures – Matthew Petroff https://mpetroff.net mpetroff.net Wed, 30 Nov 2016 02:08:23 +0000 en-US hourly 1 Bandwidth-efficient HDR WebGL Textures https://mpetroff.net/2015/11/bandwidth-efficient-hdr-webgl-textures/ https://mpetroff.net/2015/11/bandwidth-efficient-hdr-webgl-textures/#comments Sun, 15 Nov 2015 17:08:47 +0000 http://mpetroff.net/?p=2003 Continue reading →]]> Storing high dynamic range (HDR) images is a solved problem in general, such as using the excellent OpenEXR format. When is comes to the web, however, there are two major problems. The first is that web browsers don’t support any of the standard HDR image formats, and the second is that HDR images tend to be rather large, which is a problem for conserving bandwidth and reducing load times. Although there are ways to load HDR textures into WebGL using pre-encoded textures and WebGL extensions, this is very bandwidth-inefficient and not supported by all WebGL devices. Thus, the first problem has to be worked around by encoding the HDR image information into standard web images, e.g. JPEG and PNG. The earliest example of this that I can find is the pre-WebGL pfstools HDR HTML viewer, which layers multiple JPEG images at different exposures and blends them with different opacity settings.1 This method was good for its time, since before WebGL image processing calculations were difficult to perform in the browser. However, the method is bandwidth-inefficient as it requires downloading the same image at multiple exposures—five images in the example. This brings us to WebGL, where a method PNG-based method was developed by SpiderGL (and copied by three.js). Although the provided encoder is undocumented, it appears to be based on JPEG-HDR, which uses a tone-mapped base image combined with a subband image that encodes the ratio between the tone-mapped image and the original HDR image.2 The method encodes the tone-mapped image in the RGB channels of a PNG and the ratio in the alpha channel. This works well when lossless compression is desired but is bandwidth-inefficient for photographic textures.

After fiddling around with encoding RGBE as a JPEG and PNG and porting the SpiderGL technique to two separate JPEG images instead of a single PNG image, I stumbled upon a paper describing a better method (which happens to be written by the same person who created the SpiderGL HDR example). The “BoostHDR” technique described in the paper segments the image based on luminance, creating a compression-driven map (CDM), and tone maps the image with different parameters in each segment.3 With the spatial and tone map parameters contained in the CDM, the original HDR image can be recovered from the tone mapped image. The map is stored as a PNG, and the tone mapped image is stored as a JPEG. See the paper for details and figures. I implemented the paper’s encoding scheme in Python, although I left out some of the filtering steps to reduce the script’s external dependencies since the filtering didn’t help much. To display the HDR image in WebGL, both the tone mapped image and CDM image are first loaded using JavaScript, and the CDM is then loaded into the tone mapped image’s alpha channel using the Canvas API. This RGBA texture is then loaded into WebGL, where a fragment shader function is used to recover the HDR image. As one would expect from a photograph encoded as a JPEG versus a PNG, this technique results in files sizes a few times smaller than the SpiderGL method. For lack of a better name, I’m calling this technique “WebHDR.”

A demo can be viewed here. Both the encoder and demo viewer are available on GitHub and have been released into the public domain.

  1. Rafał Mantiuk and Wolfgang Heidrich. “Visualizing high dynamic range images in a web browser”. In: Journal of Graphics, GPU, and Game Tools 14.1 (2009), pp. 43–53. DOI: 10.1080/2151237X.2009.10129276. URL: http://pfstools.sourceforge.net/papers/mantiuk09vhdri.pdf.  ↩

  2. Greg Ward and Maryann Simmons. “JPEG-HDR: A backwards-compatible, high dynamic range extension to JPEG”. In: ACM SIGGRAPH 2006 Courses. ACM, 2006, p. 3. DOI: 10.1145/1185657.1185685. URL: http://www.anyhere.com/gward/papers/cic05.pdf.  ↩

  3. Francesco Banterle and Roberto Scopigno. BoostHDR: a novel backward-compatible method for HDR images. 2012. DOI: 10.1117/12.931504. URL: http://vcg.isti.cnr.it/Publications/2012/BS12/spie_2012_compression_hdr.pdf.  ↩

]]> https://mpetroff.net/2015/11/bandwidth-efficient-hdr-webgl-textures/feed/ 12