WebGL – Matthew Petroff

Color Cycle Picker

Matthew Petroff — Sat, 31 Mar 2018 01:36:29 +0000

The “category10” color palette, originally developed by Tableau, was adopted as the default color cycle for Matplotlib 2.0 and is also used by default by D3.js and Vega, along with other software packages. While more aesthetically pleasing than the old Matplotlib default, it is unfortunately not colorblind-friendly.¹ In an effort to improve this and promote the development of colorblind-friendly color cycles for scientific visualization, I built a color cycle picker that incorporates color vision deficiency simulation and enforces a minimum perceptual distance between colors, for both normal and anomalous trichromats. This is accomplished by performing color vision deficiency simulations^{doi:10.1109/TVCG.2009.113 " rel="footnote">2} for various types of deficiencies and enforcing a minimum perceptual difference for the simulated colors using the CAM02-UCS^{doi:10.1007/978-1-4419-6190-7_2 " rel="footnote">3} perceptually uniform color space (each type of deficiency is treated separately). Additionally, a minimum lightness distance is enforced, for better grayscale printability. The tool allows colors to be picked from a visualization of the CAM02-UCS color gamut and assembled into a color cycle. This visualization is performed using hardware-accelerated WebGL to allow for real-time interactive adjustment of parameters; the resulting palette is also visualized. The minimum perceptual color distance, lightness distance, and color vision deficiency simulation parameters are all adjustable. A hosted copy is provided, and the code is available in a repository on GitHub.

Personally, I have difficulty telling the second and third colors apart. ↩
G. M. Machado, M. M. Oliveira and L. A. F. Fernandes, “A Physiologically-based Model for Simulation of Color Vision Deficiency,” in IEEE Transactions on Visualization and Computer Graphics, vol. 15, no. 6, pp. 1291-1298, Nov.-Dec. 2009. doi:10.1109/TVCG.2009.113 ↩
Luo M.R., Li C. (2013) CIECAM02 and Its Recent Developments. In: Fernandez-Maloigne C. (eds) Advanced Color Image Processing and Analysis. Springer, New York, NY. doi:10.1007/978-1-4419-6190-7_2 ↩

Bandwidth-efficient HDR WebGL Textures

Matthew Petroff — Sun, 15 Nov 2015 17:08:47 +0000

Storing high dynamic range (HDR) images is a solved problem in general, such as using the excellent OpenEXR format. When is comes to the web, however, there are two major problems. The first is that web browsers don’t support any of the standard HDR image formats, and the second is that HDR images tend to be rather large, which is a problem for conserving bandwidth and reducing load times. Although there are ways to load HDR textures into WebGL using pre-encoded textures and WebGL extensions, this is very bandwidth-inefficient and not supported by all WebGL devices. Thus, the first problem has to be worked around by encoding the HDR image information into standard web images, e.g. JPEG and PNG. The earliest example of this that I can find is the pre-WebGL pfstools HDR HTML viewer, which layers multiple JPEG images at different exposures and blends them with different opacity settings.^{http://pfstools.sourceforge.net/papers/mantiuk09vhdri.pdf. " rel="footnote">1} This method was good for its time, since before WebGL image processing calculations were difficult to perform in the browser. However, the method is bandwidth-inefficient as it requires downloading the same image at multiple exposures—five images in the example. This brings us to WebGL, where a method PNG-based method was developed by SpiderGL (and copied by three.js). Although the provided encoder is undocumented, it appears to be based on JPEG-HDR, which uses a tone-mapped base image combined with a subband image that encodes the ratio between the tone-mapped image and the original HDR image.^{http://www.anyhere.com/gward/papers/cic05.pdf. " rel="footnote">2} The method encodes the tone-mapped image in the RGB channels of a PNG and the ratio in the alpha channel. This works well when lossless compression is desired but is bandwidth-inefficient for photographic textures.

After fiddling around with encoding RGBE as a JPEG and PNG and porting the SpiderGL technique to two separate JPEG images instead of a single PNG image, I stumbled upon a paper describing a better method (which happens to be written by the same person who created the SpiderGL HDR example). The “BoostHDR” technique described in the paper segments the image based on luminance, creating a compression-driven map (CDM), and tone maps the image with different parameters in each segment.^{http://vcg.isti.cnr.it/Publications/2012/BS12/spie_2012_compression_hdr.pdf. " rel="footnote">3} With the spatial and tone map parameters contained in the CDM, the original HDR image can be recovered from the tone mapped image. The map is stored as a PNG, and the tone mapped image is stored as a JPEG. See the paper for details and figures. I implemented the paper’s encoding scheme in Python, although I left out some of the filtering steps to reduce the script’s external dependencies since the filtering didn’t help much. To display the HDR image in WebGL, both the tone mapped image and CDM image are first loaded using JavaScript, and the CDM is then loaded into the tone mapped image’s alpha channel using the Canvas API. This RGBA texture is then loaded into WebGL, where a fragment shader function is used to recover the HDR image. As one would expect from a photograph encoded as a JPEG versus a PNG, this technique results in files sizes a few times smaller than the SpiderGL method. For lack of a better name, I’m calling this technique “WebHDR.”

A demo can be viewed here. Both the encoder and demo viewer are available on GitHub and have been released into the public domain.

Rafał Mantiuk and Wolfgang Heidrich. “Visualizing high dynamic range images in a web browser”. In: Journal of Graphics, GPU, and Game Tools 14.1 (2009), pp. 43–53. DOI: 10.1080/2151237X.2009.10129276. URL: http://pfstools.sourceforge.net/papers/mantiuk09vhdri.pdf. ↩
Greg Ward and Maryann Simmons. “JPEG-HDR: A backwards-compatible, high dynamic range extension to JPEG”. In: ACM SIGGRAPH 2006 Courses. ACM, 2006, p. 3. DOI: 10.1145/1185657.1185685. URL: http://www.anyhere.com/gward/papers/cic05.pdf. ↩
Francesco Banterle and Roberto Scopigno. BoostHDR: a novel backward-compatible method for HDR images. 2012. DOI: 10.1117/12.931504. URL: http://vcg.isti.cnr.it/Publications/2012/BS12/spie_2012_compression_hdr.pdf. ↩

Print Maps

Matthew Petroff — Mon, 09 Feb 2015 00:19:35 +0000

Web maps, e.g. Google Maps, normally don’t print well, as their resolution is much lower than normal print resolution, not to mention the various other unwanted text and elements that print along with the map. While the unwanted elements can be cropped out, the only fix for the low resolution is to render a higher resolution image (or use vectors). Formerly, this required installing GIS software, which also requires a suitable data source. Print Maps changes that by leveraging Mapbox GL JS and OpenStreetMap data to render print resolution maps in the browser. After the user selects the map size, zoom, location, style, resolution, and output format, PNG or PDF, Mapbox GL JS is configured as if it was being used on a very high pixel density display and used to render the map output. To use Print Maps, visit printmaps.mpetroff.net.

The site’s source code is available on GitHub. Also, slides from my HopHacks presentation on the project.

Edit (2017-01-01): Changed printmaps.org to printmaps.mpetroff.net as I’m going to let the printmaps.org domain registration expire.

Pannellum 2.0

Matthew Petroff — Sat, 23 Aug 2014 02:17:42 +0000

Two years in the making, I finally released Pannellum 2.0, which is a near complete rewrite. The renderer was replaced with raw WebGL, and multiresolution panorama support was added, along with a fallback CSS 3D renderer. Other additions include support for JSON configuration files, hotspots, tours, compass headings, CORS, partial panoramas, and cubic panoramas. I also put together a website for the viewer, registering pannellum.org. Unfortunately, there is still a dearth of documentation—something I need to work on. The below example demonstrates the multiresolution, hotspot, compass heading, and tour functionality using panoramas of the George Peabody Library and a JSON configuration file.

Pannellum 1.2

Matthew Petroff — Tue, 28 Aug 2012 16:34:42 +0000

With a few new features, a few improvements, and a bug fix, Pannellum 1.2 has been released. The new release includes keyboard panning controls, support for a fallback URL if WebGL is not supported by a browser, clarified load button text, and vector icons. Also included is a workaround for a, now resolved, regression in WebKit’s fullscreen API support. Enjoy!

Update: Pannellum 1.2.1 has been released with a fix for a minor mouse dragging bug.

The new release can be downloaded here, or one can see the project on Github.