The development of the projection was motivated as a method to store spherical panoramic images, for potential use as a new format for the Pannellum panorama viewer. Since GPU texture limits are defined in terms of square textures, a square projection is ideal for such a use, and a quincuncial arrangement will in general have lower distortion than something like a cylindrical projection, since the sphere is cut in more places. A quincuncial arrangement is also seamlessly tileable, which can help to avoid sampling issues at the image edges. Additionally, to avoid wasting pixels due to varying levels of spatial detail, an equal-area projection is preferred, eliminating the Peirce quincuncial projection from contention, since it is a conformal projection.² For existing projections, this left the Collignon quincuncial projection and the Gringorten projection. However, the Gringorten projection does not have closed-form solutions, which prevents its use in a GPU shader. This left just the Collignon quincuncial projection, but it has significant angular distortion and cusps, which will increase visual artifacts in the projected panorama view and will likely decrease the effectiveness of lossy image compression, e.g., JPEG compression. Thus, the goal was to develop a new projection that had relatively low angular distortion like the Gringorten projection but had closed-form solutions, allowing for a GPU shader implementation.

After going through many, many pages of scratch paper, I developed the new projection described in the paper, which met the design goals. It has closed-form solutions and has comparable—although slightly higher—average angular distortion to that of the Gringorten projection. However, it has lower maximum angular distortion and has only almost-imperceptible cusps. As it does not have a large, arguably-unsightly cusp along the equator like the Gringorten projection, it is probably a better projection even for cartographic purposes, where the lack of closed-form solutions does not present an issue. A comparison of the new projection with the Collignon quincuncial projection and the Gringorten projection is shown below.

The paper’s supplementary information contains all the code necessary to reproduce the paper’s figures, including a Python implementation of the projection and a D3.js implementation. Before including the projection in Pannellum, a detailed comparison of the new projection to existing cube-based equal-area and near-equal-area projections needs to be completed.

1. Gringorten, Irving I. “A square equal-area map of the world.” Journal of Applied Meteorology and Climatology 11, no. 5 (1972): 763–767.  ↩

2. A projection cannot be simultaneously conformal (angle preserving) and equal area.  ↩

I broke the process into four steps: acquiring the data, extracting L^aT_eX commands from caption environments, finding potential figure caption candidates, and verifying these candidates. As the arXiv source archive is well over 1 TB in size, it is provided in an AWS S3 bucket configured such that the requester pays for bandwidth, which would result in a bandwidth bill of >$100 if downloaded directly. As I was only interested in the T_eX source and not the figures, which account for most of the total file size, and since AWS does not charge to transfer between S3 buckets and EC2 instances in the same region, I first ran a script on an EC2 instance to download from arXiv’s S3 bucket and extract and repackage just the T_eX source files. This allowed me to greatly reduce the amount of data transfer required and allowed me to download the full T_eX source file corpus for <$5. Next, I used the TexSoup Python package to process the T_eX files and produce a list of L^aT_eX commands used in the caption environment. I then used a final script to search for papers that used command names that referenced colors or shapes to compile a list of likely paper candidates and produced HTML files for each year containing a link to the PDF for each candidate paper as well as the full T_eX source for the identified caption, with the matching commands highlighted. Finally, I manually verified the papers using the HTML files that were produced. Except for trivial false positivies, which could be identified by looking at the included caption source, I manually looked at the PDF for each candidate paper, verified that it included a visual caption indicator, and classified the caption indicator if it had one. For papers that included indicators, I then attempted to locate the published version of record of the paper and did the same for it.

Through this process, my scripts located around ~5100 paper candidates from the beginning of arXiv in 1992 through the end of June 2020. I manually verified these candidates for papers submitted prior to the end of 2016; these accounted for ~2000 candidates, of which I verified ~1100 papers to have some sort of visual caption indicator. For ~700 of these, I was able to verify the presence of some form of visual caption indicator in the published version of record. Of these, ~60% included a black shape or line indicator, ~25% included a color shape or line indicator, and the remainder included colored text. The fraction of papers with color shape or line indicators was higher in the pre-prints, since it was not uncommon for the published version to include a black indicator when the pre-print included a colored indicator. I stopped at the end of 2016 since the verification process was quite time consuming, and I could only look at so many papers before giving up.

These findings show that the idea of using figure color caption indicators is by no means a new idea. However, it’s still quite rare in relative terms, since at most a couple thousand out of arXiv’s ~1.7 million pre-prints include such indicators. Most of the examples I found used a colored shape (■) or line (—) in parentheses, or both in cases where both a line and marker were used. My proposal to use a colored underline does still appear to have been a novel concept, but it proved quite complicated to implement, so using shapes or lines in parentheses is much more practical, since it is simpler and is evidentially compatible with many publishers’ workflows. Furthermore, the existing examples can be used as evidence when complaining about paper proofs, after the typesetter predictably removes the indicators, to show that the indicators are possible and that they can and should be included in the final published version of the paper.

One color indicator that I recommend against using is colored text, since it can be difficult to read and often violates WCAG contrast guidelines. Its use seems particularly common in the computer vision literature and, to a lesser degree, the machine learning literature. It is often used to highlight table entries, a purpose much better served by using italic, bold, or bold–italic text.

I have made the scripts used for this analysis, the paper candidates, and the final verified results available. The final verified results are also available separately for easy viewing. Note that the verified results are incomplete and may contain errors.

I have only made minor changes to the previously detailed analysis procedures (see previous set ranking and order ranking posts for details), but there are now ~50% more responses, which has helped with training stability and has reduced uncertainty between different models in the network ensemble. The figure below shows the fifteen lowest ranked six-color color sets on the left and the fifteen highest ranked six-color color sets on the right.

The accuracy for both the training and tests sets remained at 58%. The plot below shows the average six-color color set scores as a function of rank, with a 1-sigma error band.

Using the highest-ranked six-color set, the figure below shows the fifteen lowest ranked orderings on the left and the fifteen highest ranked orderings on the right.

Accuracy was similar to before, with an accuracy of 55% on the training set and an accuracy of 54% on the test set. The plot below shows the average ordering scores as a function or rank, with a 1-sigma error band.

Next, the same technique was extended to the eight-color color sets. The figure below shows the fifteen lowest ranked eight-color color sets on the left and the fifteen highest ranked eight-color color sets on the right.

The accuracy was 57% for both the training and test sets. The plot below shows the average eight-color color set scores as a function of rank, with a 1-sigma error band.

Using the highest-ranked eight-color set, the figure below shows the fifteen lowest ranked orderings on the left and the fifteen highest ranked orderings on the right.

The accuracy was 55% on the training set and 53% on the test set. The plot below shows the average ordering scores as a function or rank, with a 1-sigma error band.

This is an incremental improvement over the previous results, as it just used extra data, while keeping the analysis procedure the same. The fact that accuracy was similar when the analysis was extended to eight-color color sets and color cycles is promising. I’d like to devise a method that combines both the six-color and eight-color color sets in the training process to maximize the use of the response data; I have a few ideas on how to do this but nothing concrete yet. I’ve also looked more into the idea of devising a color namability criterion by reanalyzing the xkcd Color Survey results. While my reanalysis has led to some interesting tidbits about color names, it didn’t really pan out as far as becoming a useful criterion for ranking the color sets at hand. I’ve been trying to clarify the licensing on the raw xkcd Color Survey responses database dump before writing up my findings, but so far, I have not received a reply from Randall Munroe (which is understandable). As always, more responses would be helpful. I had not originally intended for the survey to go on as long as it has, but as I’ve been busy with my normal (cosmology-related) research and as I’ve not received as many responses as I had hoped for, the survey remains open to responses. I plan on leaving it open until the analysis is close to final (at least a few more months), after which I’ll close the survey to responses and execute the final analysis runs.

Earlier this year, I became aware of a feature in GitHub-flavored Markdown that displays a colored square inline when HTML color codes are surrounded by backticks, e.g., #1f77b4. Although I only recently became aware of this feature, it dates back to at least 2017 and is similar to a feature that Slack has had since at least 2014. When I saw this inline color presentation, I immediately thought of its applicability to figure captions, particularly in academic papers; as a colorblind individual, matching colors referenced in figure captions to features in the figures themselves can be challenging at times due to difficulties with naming colors. Thus, I added similar annotations to figure captions in my recently submitted paper, Two-year Cosmology Large Angular Scale Surveyor (CLASS) Observations: A First Detection of Atmospheric Circular Polarization at Q Band:

Fig. 2. Frequency dependence of polarized atmospheric signal at zenith for the CLASS observing site, both for circular polarization (, shown in blue) and linear polarization (, shown in orange). The light gray bands indicate CLASS observing frequencies, with the lowest frequency band corresponding to the Q-band telescope.

Fig. 5. Example binned azimuth profiles are shown…angle cut. The profile in blue is from a zenith angle of 43.9° and a boresight rotation angle of −45°, the profile in orange is from a zenith angle of 46.7° and a boresight rotation angle of 0°, and the profile in red is from a zenith angle of 52.8° and a boresight rotation angle of +45°.

The first caption refers to a line plot, while the second caption refers to a scatter plot with best fit lines. These examples, as well as underlining examples elsewhere in this post, display best in a browser that supports changing the underline thickness via the text-decoration-thickness CSS property. At the time of writing, this includes Firefox 70+ and Safari 12.2+ but does not include any version of Chrome; however, browser underlining support is still subpar to the underline rendered by , so the reader is encouraged to view the figures in the paper.

While the primary purpose of these annotations is to improve accessibility for individuals with color vision deficiencies, they are also helpful when a paper is printed or displayed in grayscale. For example, it is much easier to distinguish blue and orange in grayscale with the annotations than without.

As this was an experiment, I included two different methods for visualizing the color, a thick colored underline under and a colored square following the color name. Since the colors are referring to solid lines in the plot, the underlines make sense because they match the plot features, e.g., a solid blue line. Likewise, a dotted underline might make sense for a dotted blue line, although it is more difficult to discern the color of the dotted line than the solid line. I am undecided as to whether or not including the colored square is a good idea. While it adds an additional visual cue, the main reason I included it was to increase the chances of at least one of the indicators making it past the editors and into the final published paper; as the paper is currently under review, it remains to be seen if either indicator survives the publication process.

For scatter plots, however, colored shapes make perfect sense. A scatter plot with red squares (), blue diamonds (), and orange circles () should include such shapes in the figure caption when the caption refers to the corresponding points. I am undecided as to whether or not the color names in such cases should be underlined, just as I am undecided as to whether or not line plots should included a colored square. Although I have not seen any color indicators, for either lines or scatter points, in the scientific literature, the use of shapes in figure captions is not a new practice. I have found examples dating from the mid-1950s through the early 2000s. The closest example I have found is in a 1997 paper¹ that refers to a symbol with both its name and a graphical representation:

Fig. 5. Couette-Taylor experiments. Logarithmic…number. The black triangles (▲) are the results obtained with smooth cylinders, and the open ones (△) correspond to those obtained with the ribbed ones. The crosses (×) show for comparison…and Swinney [8].

Other examples include a 1967 paper² (and a 1968 paper³) that uses graphical representations inline instead of symbol names:

Fig. 13. Additional…symmetries. Points marked with ■ are the excess…nuclei, points marked with □ the excess…N = Z. The points ▼ show the differences…larger Z-values. The points △ are the differences…for even-Z–odd-N nuclei.

and a 1955 paper⁴ that puts the figure legend inline in the figure caption:

Fig. 1. (p,n) cross sections in millibarns. ○—measured total…isotope; □—partial…isotope; ×—observed…estimate. Curves…of r₀. The dotted bands indicate…energy.

There are other examples, e.g., this 1960 paper,⁵ that put the legend on separate lines at the end of the caption, but doing so isn’t really the same idea. There are also papers that treated line styles in the same manner as scatter plot symbols, such as this 1962 paper:⁶

Fig. 1. Counting rate…in pulses per cm² sec. Maximum…is indicated by broken lines (– – –). The zone…has been shaded.

These examples should not be considered by any means exhaustive, since searching for this sort of thing is extremely difficult.⁷ In particular, while I don’t know of any prior publications that include color indicators, this does not mean that they do not exist. If anyone reading this is aware of any such examples, or of other interesting figure caption indicators, please let me know.

Adoption of visual color indicators such as the ones presented here would be a significant accessibility improvement, but it would require buy-in from both publishers and authors. The chances of success are unclear but would certainly be improved with advocacy.

Implementation

The color annotation command was defined as

% Black square
\usepackage{amsmath}

% Define color
\usepackage{xcolor}
\definecolor{tab:blue}{RGB}{31, 119, 180}

% Color underlines with breaks for descenders, based on:
% https://tex.stackexchange.com/a/75406
% https://tex.stackexchange.com/a/24771
% https://tex.stackexchange.com/a/321235
\usepackage{soul}
\usepackage[outline]{contour}
\newcommand \colorindicator[2]{%
  \begingroup%
  \setul{0.25ex}{0.4ex}%
  \contourlength{0.2ex}%
  \setulcolor{#1}%
  \ul{{\phantom{#2}}}\llap{\contour{white}{#2}} \textcolor{#1}{\tiny{$\blacksquare$}}%
  \endgroup%
}

and used with \colorindicator{tab:blue}{blue}. For HTML, this CSS

.color-underline {
  text-decoration-line: underline;
  text-decoration-style: solid;
  text-decoration-thickness: 0.2em;
  text-decoration-skip-ink: auto;
}
.blue {
  text-decoration-color: #1f77b4;
}
.blue-square::after {
  content: "\202f\25a0";
  position: relative;
  display: inline-block;
  color: #1f77b4;
}

was used with <span class="color-underline blue blue-square">blue</span> to produce blue. A production implementation would probably involve a symbol web font to improve and normalize the symbol appearance and possibly a better way to draw underlines.

Update (2020-10-31): see update on search for existing examples

1. Cadot, O., Y. Couder, A. Daerr, S. Douady, and A. Tsinober. “Energy injection in closed turbulent flows: Stirring through boundary layers versus inertial stirring.” Physical Review E 56, no. 1 (1997): 427. doi:10.1103/PhysRevE.56.427  ↩

2. Haque, Khorshed Banu, and J. G. Valatin. “An investigation of the separation energies of lighter nuclei.” Nuclear Physics A 95, no. 1 (1967): 97-114. doi:10.1016/0375-9474(67)90154-6  ↩

3. Aydin, C. “The spectral variations of CU Virginis (HD 124224).” Memorie della Societa Astronomica Italiana 39 (1968): 721. bibcode:1968MmSAI..39..721A  ↩

4. Blosser, H. G., and T. H. Handley. “Survey of (p, n) reactions at 12 MeV.” Physical Review 100, no. 5 (1955): 1340. doi:10.1103/PhysRev.100.1340  ↩

5. Evans, D. S., G. V. Raynor, and R. T. Weiner. “The lattice spacings of thorium-lanthanum alloys.” Journal of Nuclear Materials 2, no. 2 (1960): 121-128. doi:10.1016/0022-3115(60)90039-8  ↩

6. Vernov, S. N., E. V. Gorchakov, Yu I. Logachev, V. E. Nesterov, N. F. Pisarenko, I. A. Savenko, A. E. Chudakov, and P. I. Shavrin. “Investigations of radiation during flights of satellites, space vehicles and rockets.” Journal of the Physical Society of Japan Supplement 17 (1962): 162. bibcode:1962JPSJS..17B.162V  ↩

7. I found most of the above examples by performing full-text searches in NASA ADS for terms such as “black diamond” or “filled square” and looking through hundreds of results to find the few instances that included both the search terms and the symbols.  ↩

To maximize the information gleaned from the survey responses, the network was trained in four steps. The process started with a single network and ended with a conjoined network, as before, except the single network underwent three stages of training instead of one. First, the color set responses—the responses that were used in the previous analysis—were used to train the network for 50 epochs, to learn color representations. Next, the ordering responses were used with the data augmented with all possible cyclic shifts to train the network for an additional 50 epochs, to learn internal cycle orderings. Then, the non-augmented ordering responses were used to train the network for another 100 epochs, to learn the ideal starting color. Finally, the last layer of the network was replaced, as before, to make a conjoined network, and the new network was trained for a final 100 epochs, again with the non-augmented ordering responses.

As with the previous analysis, an ensemble of 100 network instantiations was trained, and the average and standard deviation of the scores were computed. The accuracy for the ordering was a bit worse than for the color sets, with an accuracy of 56% on the training data and an accuracy of 54% on the test data. Since the ideal ordering depends on the specific color set used, the highest ranked color set from the previous analysis was used in this evaluation. The error band from the trained ensemble for this color set was larger than the error band from the set ranking analysis. While the model could be evaluated for any color set, it is likely more accurate for color sets that were ranked highly in the previous analysis, since the Color Cycle Survey only asks the user about the preferred ordering of the user’s preferred color set, so data are not collected on poorly-liked color sets.

The trained network shows a clear preference for blue / purple as the first color instead of green / yellow; as many existing color cycles start with blue, this seems reasonable. The network also seems fairly confident in picking the third color, since it’s the same for the top fifteen orderings, but there’s more variation in the second color.