Bandwidth-efficient HDR WebGL Textures

Posted on November 15, 2015

Storing high dynamic range (HDR) images is a solved problem in general, such as using the excellent OpenEXR format. When is comes to the web, however, there are two major problems. The first is that web browsers don’t support any of the standard HDR image formats, and the second is that HDR images tend to be rather large, which is a problem for conserving bandwidth and reducing load times. Although there are ways to load HDR textures into WebGL using pre-encoded textures and WebGL extensions, this is very bandwidth-inefficient and not supported by all WebGL devices. Thus, the first problem has to be worked around by encoding the HDR image information into standard web images, e.g. JPEG and PNG. The earliest example of this that I can find is the pre-WebGL pfstools HDR HTML viewer, which layers multiple JPEG images at different exposures and blends them with different opacity settings.1 This method was good for its time, since before WebGL image processing calculations were difficult to perform in the browser. However, the method is bandwidth-inefficient as it requires downloading the same image at multiple exposures—five images in the example. This brings us to WebGL, where a method PNG-based method was developed by SpiderGL (and copied by three.js). Although the provided encoder is undocumented, it appears to be based on JPEG-HDR, which uses a tone-mapped base image combined with a subband image that encodes the ratio between the tone-mapped image and the original HDR image.2 The method encodes the tone-mapped image in the RGB channels of a PNG and the ratio in the alpha channel. This works well when lossless compression is desired but is bandwidth-inefficient for photographic textures.

Continue reading →

  1. Rafał Mantiuk and Wolfgang Heidrich. “Visualizing high dynamic range images in a web browser”. In: Journal of Graphics, GPU, and Game Tools 14.1 (2009), pp. 43–53. DOI: 10.1080/2151237X.2009.10129276. URL: http://pfstools.sourceforge.net/papers/mantiuk09vhdri.pdf.  ↩

  2. Greg Ward and Maryann Simmons. “JPEG-HDR: A backwards-compatible, high dynamic range extension to JPEG”. In: ACM SIGGRAPH 2006 Courses. ACM, 2006, p. 3. DOI: 10.1145/1185657.1185685. URL: http://www.anyhere.com/gward/papers/cic05.pdf.  ↩

Posted in | Tagged , , , | 12 Comments

MacBook Webcam Teardown

Posted on October 29, 2015

With the release of the mid-2013 MacBook models, Apple stopped using the standard USB video device class interface for the laptops’ webcams. On these laptops, the webcam connects via the PCIe bus, enumerating as a Broadcom 1570 device. Unfortunately, this interface is completely undocumented, and there is no Linux driver, although an effort is being made to reverse engineer one. Since I thought understanding the device hardware might aid this driver development effort, I bought a replacement webcam to disassemble. The front of the camera, from left to right, contains a 12-pin connector; power management circuitry, consisting of some sort of crystal or resonator, a Texas Instruments TPS657091 power management IC, and assorted passives; an ambient light sensor (with decoupling capacitor); the image sensor and fixed-focus optics; and an LED. The back of the device contains identification markings, which include a DataMatrix barcode and text, both of which read CC2 3222 2UYE F9T9CF on my particular camera.

Front of Camera
Back of Camera Continue reading →

Posted in | Tagged , , , , , , , | 2 Comments

An Efficient Method of Serving Many Small Files

Posted on September 30, 2015

Certain types of web content, such as map tiles and image pyramids1, require serving many small files, but this can often be inconvenient. Transferring a large number of small files over SCP or SFTP is very slow, and managing these files on disk can also be unpleasant. The transfer problem can be solved by adding the files to a tarball before transferring them, transferring the tarball, and untarring them once transfered. However, this is an extra step, and it doesn’t address the file management problem. Mapbox devised a solution for this problem for map tiles with their MBTiles format. This container format stores map tiles in a SQLite database, and a server implementation is then used to serve tiles directly from the container. This cleanly solves the aforementioned transfer and storage problems, but it is not general purpose and only works for map tiles. This specialization allows for additional optimizations such as deduplicating identical map tiles, but it means it can’t be used for storing image pyramids or other uses.

The use of a SQLite database container format can also work as a general purpose solution, provided a general purpose database key is used. Thus, I propose a general purpose “FilesDB” format as a generic solution. This format consists of an SQLite database containing a files table, which in turn contains a filename column of type text and a data column of type blob. A directory of files is stored in the format by storing each files in the directory in the database with the file’s path relative to the the base directory2 as the filename and the file’s contents as the data. As a proof of concept, I wrote a Python script for generating a .filesdb container and a rudimentary server for serving the files from the format in Go. These, along with a basic specification document, are available from a repository on GitHub.

  1. Used by Pannellum’s multires format, for example. ↩

  2. Using Unix directory separators (/) ↩

Posted in | Tagged , , , | 1 Comment

Automatic Camera Clock Synchronization under Linux

Posted on August 23, 2015

The one feature I really wish my DSLR had was geotagging. Since my camera lacks this feature, I need to record a GPS track with an external device from which positions can be extracted based on timestamps for geotagging. This requires the camera’s clock to be set accurately, which I want anyway, but doing so manually in the camera’s menu is a bit of a pain. In the past I’ve only recorded GPS tracks for geotagging sporadically, as it required carrying around a dedicated GPS receiver. However, I finally bought a smartphone a few months ago, so I now always carry a device that’s capable of recording GPS tracks.1 This caused me to revisit the clock synchronization problem.

Under Linux, gPhoto2 supports synchronizing the camera’s internal clock with the computer’s clock for many cameras, including mine, a Canon EOS Rebel T2i. As long as one’s computer is configured to use NTP, this results in quite accurate timestamps on photos. In my case, under Linux Mint 17, running this synchronization manually involves plugging in the camera, unmounting the camera after it gets automounted so gPhoto2 can access it, and then running the appropriate gPhoto2 command to synchronize the camera’s clock. To automate this process, one just needs to add a udev rule to run the clock synchronization command automatically, before the camera is mounted. I wrote such a rule. Since the rule responsible for mounting the camera is in 40-libgphoto2-6.rules, the new rule that synchronizes the camera’s clock should be saved as /etc/udev/rules.d/39-sync-camera-times.rules so that it runs right before the camera is automounted. The contents of this file are as follows:

ACTION!="add", GOTO="sync_camera_time_rules_end"
SUBSYSTEM!="usb", GOTO="sync_camera_time_rules_end"
ENV{ID_USB_INTERFACES}=="", IMPORT{builtin}="usb_id"
ENV{ID_USB_INTERFACES}=="*:060101:*", RUN+="/usr/bin/gphoto2 --set-config syncdatetime=1"

LABEL="sync_camera_time_rules_end"

Now the camera’s internal clock will be synchronized with the computer’s clock any time the camera is plugged in. Note that this sets the camera’s clock to UTC, which makes the most sense anyway as the EXIF time data doesn’t include a time zone.2 I’ve tested the rule with a Canon EOS Rebel T2i under Linux Mint 17, but it should also work for any other camera for which gPhoto2 supports clock synchronization and under Ubuntu 14.04 and similar Linux distros. Obviously, gPhoto2 needs to be installed.

Edit (2017-03-11): The above no longer works on Linux Mint 18 / Ubuntu 16.04. The following contents of /etc/udev/rules.d/39-sync-camera-times.rules should be used instead:

ACTION!="add", GOTO="sync_camera_time_rules_end"
SUBSYSTEM!="usb", GOTO="sync_camera_time_rules_end"
ENV{ID_USB_INTERFACES}=="", IMPORT{builtin}="usb_id"
ENV{ID_USB_INTERFACES}=="*:060101:*", ENV{TZ}="Etc/UTC", RUN+="/usr/bin/gphoto2 --set-config datetime=now"

LABEL="sync_camera_time_rules_end"

For some reason, gPhoto now insists on doing a time zone conversion, which is why the time zone has to be explicitly set to UTC.

  1. Transferring the recorded tracks to a computer is easier too. ↩

  2. In my opinion, this is a significant improvement over the automatic clock synchronization in Canon’s EOS Utility for Windows, which insists on syncing the camera’s clock to local time. ↩

Posted in | Tagged , , , , , , , , | 1 Comment

Decoding a Midea Air Conditioner Remote

Posted on July 26, 2015

Last month, I purchased a 6000 BTU Midea window air conditioner (branded Arctic King WWK+06CR5) and thought it would be convenient if I could control it remotely. Doing so would involve decoding the remote’s IR signals; for this, I used a USB Infrared Toy and the PyIrToy Python library. Control signals for other Midea air conditioners have previously been decoded, providing a starting point. Although the signals transmitted by my air conditioner’s R09B/BGCE remote are similar to these previous remotes, they are also sufficiently different such that the actual data transmitted shares little in common. The signal is transmitted on a 38 kHz carrier, with a time base, T, of 21 carrier cycles, approximately 1.1 ms. Each bit consists of the IR transmitter off for 1T followed by it turned on for either 1T for 0 or 3T for 1. Each frame consists of a start pulse, six bytes of data, a middle pulse, and then the inverse of the six data bytes. The start pulse consists of the transmitter off for 8T and then on for 8T; the middle pulse consists of the transmitter off for 1T, on for 9.5T, off for 8T, and then on for 8T.

R09B/BGCE Remote Continue reading →

Posted in | Tagged , , , , , | 11 Comments