Matthew Petroff Sat, 31 Oct 2020 21:18:18 +0000 en-US hourly 1 Update on Figure Caption Color Indicators Sat, 31 Oct 2020 21:15:08 +0000 Continue reading ]]> Last year, I published a blog post on figure caption color indicators. The positive feedback I received on it from a number of individuals prompted me to revisit the subject. At the time, I did not have a good way of locating published examples of such caption indicators and was only able to locate a few published examples with shape indicators but none with color indicators. When thinking about revisiting the subject, I had the epiphany that although searching for such indicators in the published literature is next to impossible, searching in the LaTeX source markup for papers is not. As arXiv provides bulk access to the TeX source markup for its pre-prints, this provided a large corpus of manuscripts to search through. After finding examples in pre-prints, I was then able to see if the indicators survived the publication process and was thereby able to locate well over one hundred examples of color line or shape indicators in the figure captions of published academic papers.

I broke the process into four steps: acquiring the data, extracting LaTeX commands from caption environments, finding potential figure caption candidates, and verifying these candidates. As the arXiv source archive is well over 1 TB in size, it is provided in an AWS S3 bucket configured such that the requester pays for bandwidth, which would result in a bandwidth bill of >$100 if downloaded directly. As I was only interested in the TeX source and not the figures, which account for most of the total file size, and since AWS does not charge to transfer between S3 buckets and EC2 instances in the same region, I first ran a script on an EC2 instance to download from arXiv’s S3 bucket and extract and repackage just the TeX source files. This allowed me to greatly reduce the amount of data transfer required and allowed me to download the full TeX source file corpus for <$5. Next, I used the TexSoup Python package to process the TeX files and produce a list of LaTeX commands used in the caption environment. I then used a final script to search for papers that used command names that referenced colors or shapes to compile a list of likely paper candidates and produced HTML files for each year containing a link to the PDF for each candidate paper as well as the full TeX source for the identified caption, with the matching commands highlighted. Finally, I manually verified the papers using the HTML files that were produced. Except for trivial false positivies, which could be identified by looking at the included caption source, I manually looked at the PDF for each candidate paper, verified that it included a visual caption indicator, and classified the caption indicator if it had one. For papers that included indicators, I then attempted to locate the published version of record of the paper and did the same for it.

Through this process, my scripts located around ~5100 paper candidates from the beginning of arXiv in 1992 through the end of June 2020. I manually verified these candidates for papers submitted prior to the end of 2016; these accounted for ~2000 candidates, of which I verified ~1100 papers to have some sort of visual caption indicator. For ~700 of these, I was able to verify the presence of some form of visual caption indicator in the published version of record. Of these, ~60% included a black shape or line indicator, ~25% included a color shape or line indicator, and the remainder included colored text. The fraction of papers with color shape or line indicators was higher in the pre-prints, since it was not uncommon for the published version to include a black indicator when the pre-print included a colored indicator. I stopped at the end of 2016 since the verification process was quite time consuming, and I could only look at so many papers before giving up.

These findings show that the idea of using figure color caption indicators is by no means a new idea. However, it’s still quite rare in relative terms, since at most a couple thousand out of arXiv’s ~1.7 million pre-prints include such indicators. Most of the examples I found used a colored shape () or line () in parentheses, or both in cases where both a line and marker were used. My proposal to use a colored underline does still appear to have been a novel concept, but it proved quite complicated to implement, so using shapes or lines in parentheses is much more practical, since it is simpler and is evidentially compatible with many publishers’ workflows. Furthermore, the existing examples can be used as evidence when complaining about paper proofs, after the typesetter predictably removes the indicators, to show that the indicators are possible and that they can and should be included in the final published version of the paper.

One color indicator that I recommend against using is colored text, since it can be difficult to read and often violates WCAG contrast guidelines. Its use seems particularly common in the computer vision literature and, to a lesser degree, the machine learning literature. It is often used to highlight table entries, a purpose much better served by using italic, bold, or bold–italic text.

I have made the scripts used for this analysis, the paper candidates, and the final verified results available. The final verified results are also available separately for easy viewing. Note that the verified results are incomplete and may contain errors.

]]> 0
Pre-calculated line breaks for HTML / CSS Mon, 25 May 2020 16:12:05 +0000 Continue reading ]]> Although slowly improving, typography on the web pages is considerably lower quality than that of high-quality print / PDF typography, such as that produced by LaTeX or Adobe InDesign. In particular, line breaks and hyphenation need considerable improvement. While CSS originally never specified what sort of line breaking algorithm should be used, browsers all converged on greedy line breaking, which produces poor-quality typography but is fast, simple, and stable. CSS Text Module Level 4 standardizes the current behavior as the default with a text-wrap property while introducing a pretty option, which instructs the browser to use a higher quality line breaking algorithm. However, as of the time of writing, no browsers supported this property.

I recently came across a CSS library for emulating LaTeX’s default appearance.1 However, it doesn’t emulate the Knuth–Plass line breaking algorithm, which is one of the things that makes LaTeX look good. This got me wondering whether or not it’s possible to emulate this with plain HTML and CSS. A JavaScript library already exists to emulate this, but it adds extra complexity and is a bit slow. It turns out that it is possible to pre-calculate line breaks and hyphenation for specific column widths in a manner that can be encoded in HTML and CSS, as long as web fonts are used to standardize the text appearance across various browsers.

The key is to wrap all the potential line breaks (inserted via ::after pseudo-elements) and hyphens in <span> elements that are hidden by default with display: none;. Media queries are then used to selectively show the line breaks specific to a given column width. Since every line has an explicit line break, justification needs to be enabled using text-align-last: justify;, and word-spacing: -10px; is used to avoid additional automatic line breaks due to slight formatting differences between browsers. However, this presents a problem for the actual last line of each paragraph, since it is now also justified instead of left aligned. This is solved by wrapping each possible last line in a <span> element. Using media queries, the <span> element corresponding to the given column width is set to use display: flex;, which makes the content be left-aligned and take up the minimum space required, thereby undoing the justification; word-spacing: 0; is also set to undo the previous change to it and fix the word spacing. Unfortunately, the nested <span> elements are problematic, because there are no spaces between them; this is fixed by including a space in the HTML markup at the beginning of the <span> and setting white-space: pre; to force the space to appear.

I’ve prepared a demo page demonstrating this technique. It was constructed by calculating line breaks in Firefox 76 using the tex-linebreak bookmarklet and manually inserting the markup corresponding to the line breaks; some fixes were manually made because the library does not properly support em dashes. Line breaks were calculated for column widths between 250 px and 500 px at 50 px increments. The Knuth–Plass line breaks lead to a considerable improvement in the text appearance, particularly for narrower column widths. In addition to the improved line breaks, I also implemented protrusion of hyphens, periods, and commas into the right margin, a microtypography technique, which further improves the appearance. To (hopefully) avoid issues with screen readers, aria-hidden="true" is set on the added markup; user-select: none; is also set, to avoid issues with text copying.

While this technique works fine in Firefox and Chrome, it does not work in Safari, since Safari does not support text-align-last as of Safari 13.2 Despite it not working, the corresponding WebKit bug is marked as “resolved fixed”; it seems that support was actually added in 2014, but the support is behind the CSS3_TEXT compile-time flag, which is disabled by default. Thus, I devised an alternative method that used invisible 100% width elements to force line breaks without using explicit line breaks. This again worked in Firefox and Chrome, although it caused minor issues with text selection, but it again had significant issues in Safari. It appears that Safari does not properly handle justified text with negative word spacing; relaxing the word spacing, however, causes extra line breaks due to formatting differences, which breaks the technique. At this point, I gave up on supporting Safari and just set it to use the browser default line breaking by placing the technique’s CSS behind an @supports query for text-align-last: justify.

Automated creation of the markup would be necessary to make this technique more generally useful, but the demo page serves as a proof of concept. Ideally, browsers would implement an improved line breaking algorithm, which would make this technique obsolete.

  1. Also see corresponding Hacker News discussion.  

  2. Even Internet Explorer 6 supports this.  

]]> 0
A Case Study in Product Label Regressions Sun, 29 Mar 2020 22:02:24 +0000 Continue reading ]]> Sometime last year, the Shop & Shop and Giant (of Landover) grocery store chains began introducing redesigned packaging for their store brand products. The two chains share a parent company and share branding, so the labels only use the shared logo without a brand name. The old label designs heavily featured a white background, which made them easy to locate in the store.1 The new brand identity is less distinct, but whether it’s better or worse is a matter of taste. However, there are specific design decisions that were made on some of the labels that have fundamental issues.

In particular, I will focus on the labels for canned vegetables. As one would expect, both the old and new label designs feature the name of the vegetable along with a picture of a “serving suggestion.” Since many vegetables are similar in color, it is often easier to find one’s desired vegetable on the shelf by looking for the name, especially when a particular vegetable comes in multiple variants, such as green beans (whole, cut, diagonally cut, and French style). The old design featured a plain sans-serif font in a dark color on a solid white background, resulting in good contrast and readability. The new design, however, is a clear regression; it trades the consistent, easily readable font for a hodgepodge of different heavily-stylized display fonts on a busier background with lower contrast, which results in much worse readability. This loss of readability makes it take longer to locate a particular product on the shelf.

No salt added green beans cans: old design on left, new design on right

Many of the vegetables come in three variants, regular (“full salt”), low sodium, and no salt added. In the old designs, these were marked using text in a brightly-colored oval. Blue was used for no salt added, and red was used for low sodium; the regular variant did not include an oval. This design allowed one to quickly differentiate between the variants on the shelf. With the new design, these colored ovals were eliminated. The low sodium variant trades the black text on the regular variant for bright blue text and a distinctive blue bar above the text with a clearly readable “low sodium” label. This is an improvement over the old design as it makes the labeling more distinctive and easier to differentiate. Unfortunately, the same is not true for the no salt added variant, which, for some inexplicable reason, is labeled the same as the regular variant except for a small, blandly-colored circular badge in the corner. Instead, it should have been labeled with a distinctive color and a bar with a clearly readable “no salt added” label, similar to the low sodium variant, except using a different color.

Low sodium green beans cans: old design on left, new design on right

The font size was increased for the ingredients list, which is one of the only improvements in the new designs.

  1. It’s the closest I’ve seen to what’s suggested by xkcd: Brand Identity.  

]]> 0
Color Cycle Survey Update Fri, 31 Jan 2020 23:38:30 +0000 Continue reading ]]> Since my last update on the Color Cycle Survey, there have been no drastic changes, but responses have continued to trickle in. There are now ~13.7k total responses, with ~6k responses each for the six color and eight color components. This long-delayed—and somewhat brief—post serves as an update to my previously published six color analysis, while also extending it to eight colors.

➡ Click Here to Take Color Cycle Survey ⬅

I have only made minor changes to the previously detailed analysis procedures (see previous set ranking and order ranking posts for details), but there are now ~50% more responses, which has helped with training stability and has reduced uncertainty between different models in the network ensemble. The figure below shows the fifteen lowest ranked six-color color sets on the left and the fifteen highest ranked six-color color sets on the right.

Six-Color Ranked Color Sets Visualization

The accuracy for both the training and tests sets remained at 58%. The plot below shows the average six-color color set scores as a function of rank, with a 1-sigma error band.

Using the highest-ranked six-color set, the figure below shows the fifteen lowest ranked orderings on the left and the fifteen highest ranked orderings on the right.

Six-Color Ranked Color Cycles Visualization

Accuracy was similar to before, with an accuracy of 55% on the training set and an accuracy of 54% on the test set. The plot below shows the average ordering scores as a function or rank, with a 1-sigma error band.

Next, the same technique was extended to the eight-color color sets. The figure below shows the fifteen lowest ranked eight-color color sets on the left and the fifteen highest ranked eight-color color sets on the right.

Eight-Color Ranked Color Sets Visualization

The accuracy was 57% for both the training and test sets. The plot below shows the average eight-color color set scores as a function of rank, with a 1-sigma error band.

Using the highest-ranked eight-color set, the figure below shows the fifteen lowest ranked orderings on the left and the fifteen highest ranked orderings on the right.

Eight-Color Ranked Color Cycles Visualization

The accuracy was 55% on the training set and 53% on the test set. The plot below shows the average ordering scores as a function or rank, with a 1-sigma error band.

This is an incremental improvement over the previous results, as it just used extra data, while keeping the analysis procedure the same. The fact that accuracy was similar when the analysis was extended to eight-color color sets and color cycles is promising. I’d like to devise a method that combines both the six-color and eight-color color sets in the training process to maximize the use of the response data; I have a few ideas on how to do this but nothing concrete yet. I’ve also looked more into the idea of devising a color namability criterion by reanalyzing the xkcd Color Survey results. While my reanalysis has led to some interesting tidbits about color names, it didn’t really pan out as far as becoming a useful criterion for ranking the color sets at hand. I’ve been trying to clarify the licensing on the raw xkcd Color Survey responses database dump before writing up my findings, but so far, I have not received a reply from Randall Munroe (which is understandable). As always, more responses would be helpful. I had not originally intended for the survey to go on as long as it has, but as I’ve been busy with my normal (cosmology-related) research and as I’ve not received as many responses as I had hoped for, the survey remains open to responses. I plan on leaving it open until the analysis is close to final (at least a few more months), after which I’ll close the survey to responses and execute the final analysis runs.

]]> 0
Figure Caption Color Indicators Sat, 23 Nov 2019 18:36:42 +0000 Continue reading ]]> .fccip-color-underline { text-decoration-line: underline; text-decoration-style: solid; text-decoration-thickness: 0.2em; text-decoration-skip-ink: auto; } .fccip-red { text-decoration-color: #d62728; } .fccip-red-square::after { content: "\202f\25a0"; position: relative; display: inline-block; color: #d62728; } .fccip-blue { text-decoration-color: #1f77b4; } .fccip-blue-gray { text-decoration-color: #6a6a6a; } .fccip-blue-square::after { content: "\202f\25a0"; position: relative; display: inline-block; color: #1f77b4; } .fccip-blue-gray-square::after { content: "\202f\25a0"; position: relative; display: inline-block; color: #6a6a6a; } .fccip-blue-square-mono::after { content: "\25a0"; position: relative; display: inline-block; color: #1f77b4; margin-left: 0.2rem; top: -0.08rem; } .fccip-blue-diamond::after { content: "\25c6"; font-size: 0.85rem; position: relative; top: -0.05rem; display: inline-block; color: #1f77b4; } .fccip-orange { text-decoration-color: #ff7f0e; } .fccip-orange-gray { text-decoration-color: #878787; } .fccip-orange-square::after { content: "\202f\25a0"; position: relative; display: inline-block; color: #ff7f0e; } .fccip-orange-gray-square::after { content: "\202f\25a0"; position: relative; display: inline-block; color: #878787; } .fccip-orange-circle::after { content: "\25cf"; font-size: 1.2rem; position: relative; display: inline-block; color: #ff7f0e; } .fccip-gray { text-decoration-color: #ccc; } .fccip-gray-square::after { content: "\202f\25a0"; position: relative; display: inline-block; color: #ccc; } .fccip-dotted { text-decoration-style: dotted; } .fccip-caption { margin-left: 3em; margin-right: 3em; padding: 0.5em 0; border-top: 1px solid #aaa; border-bottom: 1px solid #aaa; }

Earlier this year, I became aware of a feature in GitHub-flavored Markdown that displays a colored square inline when HTML color codes are surrounded by backticks, e.g., #1f77b4. Although I only recently became aware of this feature, it dates back to at least 2017 and is similar to a feature that Slack has had since at least 2014. When I saw this inline color presentation, I immediately thought of its applicability to figure captions, particularly in academic papers; as a colorblind individual, matching colors referenced in figure captions to features in the figures themselves can be challenging at times due to difficulties with naming colors. Thus, I added similar annotations to figure captions in my recently submitted paper, Two-year Cosmology Large Angular Scale Surveyor (CLASS) Observations: A First Detection of Atmospheric Circular Polarization at Q Band:

Fig. 2. Frequency dependence of polarized atmospheric signal at zenith for the CLASS observing site, both for circular polarization (|V|, shown in blue) and linear polarization (\sqrt{Q^2+U^2}, shown in orange). The light gray bands indicate CLASS observing frequencies, with the lowest frequency band corresponding to the Q-band telescope.

Fig. 5. Example binned azimuth profiles are shown…angle cut. The profile in blue is from a zenith angle of 43.9° and a boresight rotation angle of −45°, the profile in orange is from a zenith angle of 46.7° and a boresight rotation angle of 0°, and the profile in red is from a zenith angle of 52.8° and a boresight rotation angle of +45°.

The first caption refers to a line plot, while the second caption refers to a scatter plot with best fit lines. These examples, as well as underlining examples elsewhere in this post, display best in a browser that supports changing the underline thickness via the text-decoration-thickness CSS property. At the time of writing, this includes Firefox 70+ and Safari 12.2+ but does not include any version of Chrome; however, browser underlining support is still subpar to the underline rendered by \LaTeX, so the reader is encouraged to view the figures in the paper.

While the primary purpose of these annotations is to improve accessibility for individuals with color vision deficiencies, they are also helpful when a paper is printed or displayed in grayscale. For example, it is much easier to distinguish blue and orange in grayscale with the annotations than without.

As this was an experiment, I included two different methods for visualizing the color, a thick colored underline under and a colored square following the color name. Since the colors are referring to solid lines in the plot, the underlines make sense because they match the plot features, e.g., a solid blue line. Likewise, a dotted underline might make sense for a dotted blue line, although it is more difficult to discern the color of the dotted line than the solid line. I am undecided as to whether or not including the colored square is a good idea. While it adds an additional visual cue, the main reason I included it was to increase the chances of at least one of the indicators making it past the editors and into the final published paper; as the paper is currently under review, it remains to be seen if either indicator survives the publication process.

For scatter plots, however, colored shapes make perfect sense. A scatter plot with red squares (), blue diamonds (), and orange circles () should include such shapes in the figure caption when the caption refers to the corresponding points. I am undecided as to whether or not the color names in such cases should be underlined, just as I am undecided as to whether or not line plots should included a colored square. Although I have not seen any color indicators, for either lines or scatter points, in the scientific literature, the use of shapes in figure captions is not a new practice. I have found examples dating from the mid-1950s through the early 2000s. The closest example I have found is in a 1997 paper1 that refers to a symbol with both its name and a graphical representation:

Fig. 5. Couette-Taylor experiments. Logarithmic…number. The black triangles (▲) are the results obtained with smooth cylinders, and the open ones (△) correspond to those obtained with the ribbed ones. The crosses (×) show for comparison…and Swinney [8].

Other examples include a 1967 paper2 (and a 1968 paper3) that uses graphical representations inline instead of symbol names:

Fig. 13. Additional…symmetries. Points marked with ■ are the excess…nuclei, points marked with □ the excess…N = Z. The points ▼ show the differences…larger Z-values. The points △ are the differences…for even-Z–odd-N nuclei.

and a 1955 paper4 that puts the figure legend inline in the figure caption:

Fig. 1. (p,n) cross sections in millibarns. ○—measured total…isotope; □—partial…isotope; ×—observed…estimate. Curves…of r0. The dotted bands indicate…energy.

There are other examples, e.g., this 1960 paper,5 that put the legend on separate lines at the end of the caption, but doing so isn’t really the same idea. There are also papers that treated line styles in the same manner as scatter plot symbols, such as this 1962 paper:6

Fig. 1. Counting rate…in pulses per cm2 sec. Maximum…is indicated by broken lines (– – –). The zone…has been shaded.

These examples should not be considered by any means exhaustive, since searching for this sort of thing is extremely difficult.7 In particular, while I don’t know of any prior publications that include color indicators, this does not mean that they do not exist. If anyone reading this is aware of any such examples, or of other interesting figure caption indicators, please let me know.

Adoption of visual color indicators such as the ones presented here would be a significant accessibility improvement, but it would require buy-in from both publishers and authors. The chances of success are unclear but would certainly be improved with advocacy.


The \LaTeX color annotation command was defined as

% Black square

% Define color
\definecolor{tab:blue}{RGB}{31, 119, 180}

% Color underlines with breaks for descenders, based on:
\newcommand \colorindicator[2]{%
  \ul{{\phantom{#2}}}\llap{\contour{white}{#2}} \textcolor{#1}{\tiny{$\blacksquare$}}%

and used with \colorindicator{tab:blue}{blue}. For HTML, this CSS

.color-underline {
  text-decoration-line: underline;
  text-decoration-style: solid;
  text-decoration-thickness: 0.2em;
  text-decoration-skip-ink: auto;
.blue {
  text-decoration-color: #1f77b4;
.blue-square::after {
  content: "\202f\25a0";
  position: relative;
  display: inline-block;
  color: #1f77b4;

was used with <span class="color-underline blue blue-square">blue</span> to produce blue. A production implementation would probably involve a symbol web font to improve and normalize the symbol appearance and possibly a better way to draw underlines.

Update (2020-10-31): see update on search for existing examples

  1. Cadot, O., Y. Couder, A. Daerr, S. Douady, and A. Tsinober. “Energy injection in closed turbulent flows: Stirring through boundary layers versus inertial stirring.” Physical Review E 56, no. 1 (1997): 427. doi:10.1103/PhysRevE.56.427  

  2. Haque, Khorshed Banu, and J. G. Valatin. “An investigation of the separation energies of lighter nuclei.” Nuclear Physics A 95, no. 1 (1967): 97-114. doi:10.1016/0375-9474(67)90154-6  

  3. Aydin, C. “The spectral variations of CU Virginis (HD 124224).” Memorie della Societa Astronomica Italiana 39 (1968): 721. bibcode:1968MmSAI..39..721A  

  4. Blosser, H. G., and T. H. Handley. “Survey of (p, n) reactions at 12 MeV.” Physical Review 100, no. 5 (1955): 1340. doi:10.1103/PhysRev.100.1340  

  5. Evans, D. S., G. V. Raynor, and R. T. Weiner. “The lattice spacings of thorium-lanthanum alloys.” Journal of Nuclear Materials 2, no. 2 (1960): 121-128. doi:10.1016/0022-3115(60)90039-8  

  6. Vernov, S. N., E. V. Gorchakov, Yu I. Logachev, V. E. Nesterov, N. F. Pisarenko, I. A. Savenko, A. E. Chudakov, and P. I. Shavrin. “Investigations of radiation during flights of satellites, space vehicles and rockets.” Journal of the Physical Society of Japan Supplement 17 (1962): 162. bibcode:1962JPSJS..17B.162V  

  7. I found most of the above examples by performing full-text searches in NASA ADS for terms such as “black diamond” or “filled square” and looking through hundreds of results to find the few instances that included both the search terms and the symbols.  

]]> 0