# Making ggplot2 Accessible

Paul Murrell, Debra Warren
The University of Auckland
A. Jonathon Godfrey
Massey University
August 2018

## BrailleR

A histogram

library(BrailleR)
"faithful$eruptions" is marked on the x-axis. Tick marks for the x-axis are at: 2, 3, 4, and 5 There are a total of 272 elements for this variable. Tick marks for the y-axis are at: 0, 20, 40, and 60 It has 8 equal-width bins, starting at 1.5 and ending at 5.5 . The mids and counts for the bins are: mid = 1.75 count = 55 mid = 2.25 count = 37 mid = 2.75 count = 5 mid = 3.25 count = 9 mid = 3.75 count = 34 mid = 4.25 count = 75 mid = 4.75 count = 54 mid = 5.25 count = 3  The VI() function from the 'BrailleR' package can generate a text description of the histogram. In combination with a screen reader, this provides some information about the histogram for blind or visually impaired R users. ## BrailleR 'BrailleR' uses information returned by hist() h[1:5]  $breaks
[1] 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5

$counts [1] 55 37 5 9 34 75 54 3$density
[1] 0.40441176 0.27205882 0.03676471 0.06617647 0.25000000
[6] 0.55147059 0.39705882 0.02205882

$mids [1] 1.75 2.25 2.75 3.25 3.75 4.25 4.75 5.25$xname
[1] "faithful$eruptions"  The hist() function returns an object containing information about the histogram as well as drawing the histogram. VI() uses that information to create a text description of the histogram. ## BrailleR Not all plotting functions return information p <- plot(eruptions ~ waiting, faithful)  The plot() functions does not return ANY information about the plot that it draws. ## BrailleR Not all plotting functions return information p  NULL  This makes it hard for VI() to do anything. ## BrailleR 'BrailleR' augments some plotting functions > library(BrailleR) Attaching package: ‘BrailleR’ The following objects are masked from ‘package:graphics’: boxplot, hist  The 'BrailleR' package has its own versions of some major plotting functions that augment the information that is returned ... ## BrailleR 'BrailleR' has its own plotting functions s <- with(faithful, ScatterPlot(waiting, eruptions))  ... and the 'BrailleR' package has some new plotting functions ... ## BrailleR 'BrailleR' has its own plotting functions str(s[1:2])  List of 2$ data     :'data.frame':	272 obs. of  2 variables:
..$x: num [1:272] 43 45 45 45 46 46 46 46 46 47 ... ..$ y: num [1:272] 1.98 1.87 1.92 2.2 1.78 ...
$ExtraArgs:List of 4 ..$ xlab: chr "waiting"
..$ylab: chr "eruptions" ..$ main: chr ""
..$sub : chr ""  ... that generate information about the plot. ## BrailleR 'BrailleR' has a lot of work to do methods(VI)   [1] VI.aov VI.boxplot VI.data.frame VI.default [5] VI.dotplot VI.ggplot VI.hist VI.histogram [9] VI.list VI.lm VI.matrix VI.summary.lm [13] VI.tsplot VI.TukeyHSD see '?methods' for accessing help and source code  The 'BrailleR' package has to do a lot of work for each individual plot that it wants to support ## ggplot2 A 'ggplot2' plot library(ggplot2) g <- ggplot(faithful) + geom_histogram(aes(x=eruptions), breaks=h$breaks)


'ggplot2' is a quite popular package for drawing plots in R

## ggplot2 in BrailleR

(OLD) 'BrailleR' text from a 'ggplot2' plot

VI(g)

This untitled chart;
has no subtitle;
and no caption.
with x-axis "eruptions" labeled from  to  and
y-axis "count" labeled from  to ;


But 'BrailleR' could not generate text descriptions for 'ggplot2' plots

## ggplot2 in BrailleR

'ggplot2' always generates a plot object

class(g)

[1] "gg"     "ggplot"

names(g)

[1] "data"        "layers"      "scales"      "mapping"
[5] "theme"       "coordinates" "facet"       "plot_env"
[9] "labels"


'ggplot2' always generates a "ggplot" object

## ggplot2 in BrailleR

'ggplot2' exports ggplot_build()

gb <- ggplot_build(g)
names(gb)

[1] "data"   "layout" "plot"

names(gb\$plot)

[1] "data"        "layers"      "scales"      "mapping"
[5] "theme"       "coordinates" "facet"       "plot_env"
[9] "labels"


## ggplot2 in BrailleR

(NEW) 'BrailleR' text from a 'ggplot2' plot

VI(g)

This is an untitled chart with no subtitle or caption.
It has x-axis 'eruptions' with labels 2, 3, 4 and 5.
It has y-axis 'count' with labels 0, 20, 40 and 60.
The chart is a bar chart containing 8 vertical bars.
Bar 1 is centered at 1.75, and spans from 0 to 55.
Bar 2 is centered at 2.25, and spans from 0 to 37.
Bar 3 is centered at 2.75, and spans from 0 to 5.
Bar 4 is centered at 3.25, and spans from 0 to 9.
Bar 5 is centered at 3.75, and spans from 0 to 34.
Bar 6 is centered at 4.25, and spans from 0 to 75.
Bar 7 is centered at 4.75, and spans from 0 to 54.
Bar 8 is centered at 5.25, and spans from 0 to 3.


So now 'BrailleR' can not generate text descriptions for 'ggplot2' plots!

## ggplot2 in BrailleR

(NEW) 'BrailleR' text from a 'ggplot2' plot

g2 <- ggplot(faithful[1:5,]) +
geom_point(aes(x=waiting, y=eruptions))
print(g2)


And this just works for a range of plots straight away

## ggplot2 in BrailleR

(NEW) 'BrailleR' text from a 'ggplot2' plot

VI(g2)

This is an untitled chart with no subtitle or caption.
It has x-axis 'waiting' with labels 60, 70 and 80.
It has y-axis 'eruptions' with labels 2, 3 and 4.
The chart is a set of 5 points.
The points are at:
(79, 3.6),
(54, 1.8),
(74, 3.33),
(62, 2.28) and
(85, 4.53)


And this just works for a range of plots straight away

## ggplot2 in BrailleR

Issues:

• How to decide what to "say" about a plot?

Deciding what to say and how to say it is hard; there is no "best" solution

## ggplot2 in BrailleR

Solutions:

• Base text off a "template" that can be modified
{{#title}}This chart has title '{{title}}'.{{/title}}
{{^title}}This chart is untitled.{{/title}}
{{#subtitle}}It has the subtitle '{{subtitle}}'.{{/subtitle}}
{{#caption}}It has caption '{{caption}}'.{{/caption}}


So we allow for flexibility; people can use their own template (based on the R package 'whisker' and the Mustache templating language; ...

## ggplot2 in BrailleR

Solutions:

• Rather than cat() text onto the screen, generate an object that contains the text
grep("axis", VI(g))

It has x-axis 'eruptions' with labels 2, 3, 4 and 5.
It has y-axis 'count' with labels 0, 20, 40 and 60.


... and people can do things with the object, like searching

## ggplot2 in BrailleR

Solutions:

• Rather than cat() text onto the screen, generate an object that contains the text
sort(VI(g2))

This is an untitled chart with no subtitle or caption.
It has x-axis 'waiting' with labels 60, 70 and 80.
It has y-axis 'eruptions' with labels 2, 3 and 4.
The chart is a set of 5 points.
The points are at:
(54, 1.8),
(62, 2.28),
(74, 3.33),
(79, 3.6) and
(85, 4.53)


... and people can do things with the object, like sorting

## ggplot2 in BrailleR

The overall design involves gathering information about a plot, generating text from the information (using a template), and then writing methods that work with the combination of information and text.

## Example

qp <- qplot(displ, hwy, data=mpg, facets=.~year,
geom=c("point", "smooth"))
print(qp)


With 'ggplot2', we describe a plot in terms of a small number of building blocks - we can create an infinite array of plots by combining the building blocks in different ways.

## Example

VI(qp)

This is an untitled chart w/ no subtitle or caption.
The chart is comprised of 2 panels arranged horizontally.
The panels represent different values of year.
Each panel has x-axis 'displ' w/ labels 2, 3, 4, 5, 6 and 7.
Each panel has y-axis 'hwy' w/ labels 20, 30 and 40.
Each panel has 2 layers.
Panel 1 represents data for year = 1999.
Layer 1 of panel 1 is a set of 117 points.
Layer 2 of panel 1 is a smooth curve w/ confidence intervals.
Panel 2 represents data for year = 2008.
Layer 1 of panel 2 is a set of 117 points.
Layer 2 of panel 2 is a smooth curve w/ confidence intervals.


This means that the support for 'ggplot2' in 'BrailleR' can just add support for a small set of building blocks and then we can generate accessible text for an infinite array of plots.

## Summary

'BrailleR' now has 'ggplot2' support

This vastly extends the range of R plots for which we can generate accessible text

We have also extended the flexibility of the generation of accessible text

We have also made it easy to search and sort accessible text

Slide says it all

## Acknowledgements

Debra Warren (Master's project)

Jonathon Godfrey ('BrailleR')

The 'whisker' R package

The Mustache templating language