% !TeX root = RJwrapper.tex
\title{Enhanced Vector Image Import: The grConvert and grImport2 packages}
\author{by Simon Potter and Paul Murrell}
% Ask if Toby wants to be author too?

\newcommand{\ps}{PostScript}

\maketitle

\abstract{
Two new packages, \pkg{grConvert} and \pkg{grImport2},
have been created to improve the support for importing 
external vector images and rendering those images as
part of an R graphics scene.  Where the old \pkg{grImport}
package can import \ps{} images, the new \pkg{grImport2}
package can import SVG images, which means that more complex
images can be imported. The \pkg{grConvert} package
provides functions for converting images to either 
\ps{} or SVG for importing into R.
}

\section{The problem}

For some time, the \pkg{grImport} package \citep{grimport} 
has allowed \ps{} files to be imported into R and drawn
as part of an R plot.  As long as the original image is a \ps{}
image and the image consists of only
paths and text, the \pkg{grImport} package performs quite well.
For example, Figure \ref{fig:grimport-success} shows a test image 
consisting of a coloured background rectangle, some text, and a circle.
The following code is all that is required to import that image
and draw it in R (the result is shown in Figure \ref{fig:grimport-success}).

<<>>=
library(grImport)
@ 
<<echo=FALSE>>=
testimage <- function(text, fill=FALSE, gradient=FALSE, raster=FALSE) {
    grid.rect(gp=gpar(col=NA, fill=hcl(20, 100, 60)))
    grid.text(text, gp=gpar(fontface="italic", col="white"))
    if (fill) {
        grid.circle(r=.3, name="c", 
                    gp=gpar(col="white", fill=hcl(200, 100, 60, .5)))
    } else {
        grid.circle(r=.3, name="c", gp=gpar(col="white"))
    }
    if (gradient) {
        grid.gradientFill("c", linearGradient(hcl(c(20, 200), 100, 60, .8),
                                              stops=c(.2, .8)))
    }
    if (raster) {
        # image is 1 inch wide == 72 pts => 72/8 = 9 pts for raster
        # (i.e., whole number of pts per pixel)
        grid.raster(matrix(c(0:1, 1:0), ncol=4, nrow=2), interp=FALSE,
                    x=.9, y=.8, width=1/9)
    }
}
@ 
<<echo=FALSE, results=hide>>=
postscript("testimage-simple.ps", width=1, height=.5, 
           paper="special", horizontal=FALSE)
testimage("simple")
dev.off()
# for inclusion
pdf("testimage-simple.pdf", width=1, height=.5)
testimage("simple")
dev.off()
@ 
<<>>=
PostScriptTrace("testimage-simple.ps", "testimage-simple.xml")
image <- readPicture("testimage-simple.xml")
@ 
<<eval=FALSE>>=
grid.picture(image)
@ 
<<echo=FALSE>>=
library(png)
img <- readPNG(system.file("img", "Rlogo.png", package="png"))
@  
<<echo=FALSE>>=
rpic <- function(FUN, lab=1) {
    grid.rect(gp=gpar(col=NA, fill="#DDDDDD"))
    # grid.raster(img, x=.1, y=.15, width=.1)
    switch(lab,
           grid.text("grImport", x=.1, rot=90, 
                     gp=gpar(col="grey60", fontface="italic", lineheight=1)),
           grid.text("grImport2", x=.1, rot=90, 
                     gp=gpar(fontface="italic", lineheight=1)),
           { grid.text("grImport2", x=.1, rot=90, 
                       gp=gpar(fontface="italic", lineheight=1))
             grid.text("+ gridSVG ", x=.18, rot=90, 
                       gp=gpar(col="grey60", cex=.7,
                           fontface="italic", lineheight=1))
           },
           { grid.text("grImport2", x=.1, rot=90, 
                       gp=gpar(fontface="italic", lineheight=1))
             grid.text("+ clipbbox ", x=.18, rot=90, 
                       gp=gpar(col="grey60", cex=.7,
                           fontface="italic", lineheight=1))
           }) 
    pushViewport(viewport(x=.2, width=.8, just="left"))
    FUN()
    popViewport()
}
@ 
<<echo=FALSE, results=hide>>=
pdf("testimage-simple-grimport.pdf", width=2, height=1)
rpic(function() grid.picture(image) )
dev.off()
@ 

\begin{figure}[htbp]
  \hspace*{\fill}
  % what it is supposed to look like
  \includegraphics[width=1.5in]{testimage-simple.pdf}
  \hspace*{\fill}
  \includegraphics[width=1.5in]{testimage-simple-grimport.pdf}
  \hspace*{\fill}
  \caption{\label{fig:grimport-success}On the left is a simple \ps{} test image
    consisting of only paths and text.  On the right is the result of 
    importing and rendering the test image using \pkg{grImport}. 
    Because the original image is simple, \pkg{grImport} does a good
    job of reproducing it in R.}
\end{figure}

However, there are two main weaknesses in the \pkg{grImport} package:
not all images that we might want to import 
are in a \ps{} format;  and \pkg{grImport} does not import
all possible \ps{} images.

An example of the latter weakness
is the fact that \pkg{grImport} does not import
any raster elements from the original \ps{} image.  To demonstrate this,
Figure \ref{fig:grimport-raster} shows a test image that includes a
small checkerboard raster element in the top-right corner, 
along with the
result of importing and rendering that image with \pkg{grImport}
(to show that the raster element does not make it).

<<echo=FALSE, results=hide>>=
postscript("testimage-raster.ps", width=1, height=.5, horizontal=FALSE)
testimage("raster", raster=TRUE)
dev.off()
# for inclusion
pdf("testimage-raster.pdf", width=1, height=.5)
testimage("raster", raster=TRUE)
dev.off()
@ 
<<echo=FALSE>>=
PostScriptTrace("testimage-raster.ps", "testimage-raster.xml")
image <- readPicture("testimage-raster.xml")
@ 
<<echo=FALSE, results=hide>>=
pdf("testimage-raster-grimport.pdf", width=2, height=1)
rpic(function() grid.picture(image) )
dev.off()
@ 

\begin{figure}[htbp]
  \hspace*{\fill}
  % what it is supposed to look like
  \includegraphics[width=1.5in]{testimage-raster.pdf}
  \hspace*{\fill}
  \includegraphics[width=1.5in]{testimage-raster-grimport.pdf}
  \hspace*{\fill}
  \caption{\label{fig:grimport-raster}On the left is a \ps{} test image
    that contains a raster element in the top-right corner.
    On the right is the result of 
    importing and rendering the test image using \pkg{grImport}. 
    The raster element is lost from the image during the import process.}
\end{figure}

When the original image is not in a \ps{}
format, it must be converted to \ps{} using third-party 
software, such as ImageMagick or Inkscape \citep{imagemagick, inkscape},
and this creates two further problems:  first, it places a burden on
the user to install and learn to use a conversion utility;
second, and worse, the conversion to \ps{} may result in degradation or
loss of features from the original image.

An example of the second problem is shown in 
Figure \ref{fig:grimport-semitrans}, where an original 
image is in an SVG format and it
contains a semitransparent fill.  Converting this image
to \ps{} with ImageMagick
leads to rasterisation of the image and we have already seen that
\pkg{grImport} cannot deal with raster input.

<<echo=FALSE, results=hide>>=
library(gridSVG)
library(grid)
gridsvg("testimage-semitrans.svg", width=1, height=.5)
testimage("semitrans", fill=TRUE, raster=TRUE)
dev.off()
@ 
<<echo=FALSE>>=
# for inclusion
system("convert -density 300x300 testimage-semitrans.svg testimage-semitrans.png")
# Deliberately exaggerate the rasterisation for dramatic effect
system("convert testimage-semitrans.svg testimage-semitrans-IM.ps")
# for inclusion
system("convert testimage-semitrans-IM.ps testimage-semitrans-IM.pdf")
@ 
<<echo=FALSE>>=
PostScriptTrace("testimage-semitrans-IM.ps", "testimage-semitrans-IM.xml")
image <- readPicture("testimage-semitrans-IM.xml")
@ 
<<echo=FALSE, results=hide>>=
pdf("testimage-semitrans-IM-grimport.pdf", width=2, height=1)
rpic(function() grid.picture(image) )
dev.off()
@ 

\begin{figure}[htbp]
  \hspace*{\fill}
  % what it is supposed to look like
  \includegraphics[width=1.5in]{testimage-semitrans.png}
  \hspace*{\fill}
  \includegraphics[width=1.5in]{testimage-semitrans-IM.pdf}
  \hspace*{\fill}
  \includegraphics[width=1.5in]{testimage-semitrans-IM-grimport.pdf}
  \hspace*{\fill}
  \caption{\label{fig:grimport-semitrans}On the left is a test image
    that is in an SVG format and that contains a semitransparent fill.
    In the middle is the result of converting the original image
    to a \ps{} format with ImageMagick, which results in rasterisation
    of the image.
    On the right is the result of importing and rendering the converted
    image with \pkg{grImport}, which does not work very well because
    \pkg{grImport} does not import raster elements.}
\end{figure}

In some cases, the conversion problem can be worked around simply
by using a different conversion tool.  For example, Inkscape does
a better job than ImageMagick at
retaining semitransparent fills during conversion
to \ps{}.  However, for more complex images (or images containing 
more sophisticated features), there may be no solution.
Figure \ref{fig:grimport-gradient} shows another test image
(requiring conversion because it is in an SVG format),
that contains a semitransparent \emph{gradient} fill.
The conversion to \ps{} results in either rasterisation, if we use 
ImageMagick, or simply loss of the fill information, if we use
Inkscape.

<<echo=FALSE, results=hide>>=
library(gridSVG)
library(grid)
gridsvg("testimage-gradient.svg", width=1, height=.5)
testimage("gradient", gradient=TRUE, raster=TRUE)
dev.off()
@ 
<<echo=FALSE>>=
# for inclusion
system("convert -density 300x300 testimage-gradient.svg testimage-gradient.png")
system("inkscape --export-ps=testimage-gradient-inkscape.ps --export-text-to-path testimage-gradient.svg")
# For inclusion
system("inkscape --export-pdf=testimage-gradient-inkscape.pdf --export-text-to-path testimage-gradient.svg")
@ 
<<echo=FALSE>>=
PostScriptTrace("testimage-gradient-inkscape.ps", "testimage-gradient-inkscape.xml")
image <- readPicture("testimage-gradient-inkscape.xml")
@ 
<<echo=FALSE, results=hide>>=
pdf("testimage-gradient-inkscape-grimport.pdf", width=2, height=1)
rpic(function() grid.picture(image) )
dev.off()
@ 

\begin{figure}[htbp]
  \hspace*{\fill}
  % what it is supposed to look like
  \includegraphics[width=1.5in]{testimage-gradient.png}
  \hspace*{\fill}
  \includegraphics[width=1.5in]{testimage-gradient-inkscape.pdf}
  \hspace*{\fill}
  \includegraphics[width=1.5in]{testimage-gradient-inkscape-grimport.pdf}
  \hspace*{\fill}
  \caption{\label{fig:grimport-gradient}On the left is a test image
    that is in an SVG format and that contains a semitransparent 
    gradient fill.
    In the middle is the result of converting the original image
    to a \ps{} format with Inkscape, which results in loss of the 
    gradient fill.
    On the right is the result of importing and rendering the converted
    image with \pkg{grImport}, which (apart from dropping the raster
    element) works fine, but the result is
    imperfect because of loss of information during the conversion to \ps{}.}
\end{figure}

This article describes two new packages that are designed to address
the limitations of the \pkg{grImport} package.  
The \pkg{grConvert} package aims to improve support
for converting original images to a format that can be imported
into R and the \pkg{grImport2} package aims to improve support for
importing more complex images into R.

\section{The \pkg{grConvert} package}

The \pkg{grConvert} package provides a simple function, called
\code{convertPicture()}, to convert between different graphics formats.
The original image format can be \ps{}, PDF, or SVG
and any of those can be converted to either \ps{} or SVG.

Converting an image to \ps{} is useful for importing an image
with the \pkg{grImport} package.
For example, if the simple test image from Figure \ref{fig:grimport-success}
was originally an SVG image, it
could be converted to \ps{} with the following code.

<<>>=
library(grConvert)
@ 
<<eval=FALSE>>=
convertPicture("testimage-simple.svg", "testimage-simple.ps")
@ 

Using \pkg{grConvert} to convert
an image to SVG is useful for importing an image with the new
\pkg{grImport2} package, as we will see in the next section.

The main purpose of the \pkg{grConvert} package is to provide more
convenience for the task of converting an image from its
original format to a format that can be imported into R.
However, the \pkg{grConvert} 
package depends on the Spectre library 
for reading and writing
\ps{} files \citep{spectre}, 
the Poppler library for reading PDF files
\citep{poppler}, and the
rsvg library for reading and writing SVG files \citep{rsvg}, so these must
be available for the package to install.  This means that, in practice,
the package is only available for Linux systems at the time of writing.

One reason for making \pkg{grConvert} a separate package was to
emphasise the separation between the two steps of preparing an image
for import, which is a one-off operation, and importing the prepared
image into R.  These steps can be carried out by different people on
different platforms if necessary.

\section{The \pkg{grImport2} package}

The \pkg{grImport2} package provides two main functions:
\code{readPicture()} for reading an external image into R
and \code{grid.picture()} for drawing the image.

In contrast to the \code{readPicture()}
function from the \pkg{grImport} package,
which is used to import \ps{} files, 
the \code{readPicture()} function 
in \pkg{grImport2} imports SVG files.  

The \pkg{grImport2} package does not understand all of SVG, 
but rather a subset defined to be SVG code that is produced by 
the Cairo graphics library \citep{cairo}.  This subset of SVG
is used because it is much easier to develop code to import a subset
rather than all of the possible complexity of the SVG language.
Furthermore, this
Cairo SVG format is exactly what the \code{grConvert} package produces
when it converts an image to SVG.
This means that the 
typical process for importing an image into R using \pkg{grImport2}
involves a call to the \code{convertPicture()} function
from the \pkg{grConvert} package, to produce a Cairo SVG version
(even if the original image format was SVG),
then a call to \code{readPicture()} to import the image, and finally
 a call to 
\code{grid.picture()} to render the imported image.  

As an example, the following code starts with the simple test image
from Figure \ref{fig:grimport-success} in an SVG format, converts
it to Cairo SVG, and then imports and renders it with the \pkg{grImport2}
package (see Figure \ref{fig:grimport2-simple}).

<<>>=
library(grImport2)
@ 
<<echo=FALSE>>=
library(gridSVG)
gridsvg("testimage-simple.svg", width=1, height=.5)
testimage("simple")
dev.off()
# for inclusion
system("convert -density 300x300 testimage-simple.svg testimage-simple.png")
@ 
<<>>=
convertPicture("testimage-simple.svg", "testimage-simple-cairo.svg")
image <- readPicture("testimage-simple-cairo.svg")
@ 
<<>>=
grid.picture(image)
@ 
<<echo=FALSE, results=hide>>=
pdf("testimage-simple-grimport2.pdf", width=2, height=1)
rpic(function() { grid.picture(image) }, lab=2)
dev.off()
@ 
\begin{figure}[htbp]
  \hspace*{\fill}
  % what it is supposed to look like
  \includegraphics[width=1.5in]{testimage-simple.png}
  \hspace*{\fill}
  \includegraphics[width=1.5in]{testimage-simple-grimport2.pdf}
  \hspace*{\fill}
  \caption{\label{fig:grimport2-simple}On the left is a simple SVG test image
    consisting of only paths and text.  On the right is the result of
    converting the image to Cairo SVG using \pkg{grConvert} then 
    importing and rendering the image using \pkg{grImport2}.}
\end{figure}

The previous example demonstrates that the \pkg{grImport2} package
can be used in a very similar manner to the \pkg{grImport} package and
it should be just as good at handling images consisting of just paths and text.
The next example begins to demonstrate where the \pkg{grImport2} package
performs better than the \pkg{grImport} package.  

In the following code,
we start with an SVG version of the test image from 
Figure \ref{fig:grimport-raster}, which contains a raster element,
convert it to Cairo SVG and import and render it with \pkg{grImport2}.
The result is shown in Figure \ref{fig:grimport2-raster},
which illustrates that \pkg{grImport2} is able to import all features
of the original image, in this case including the raster element
in the test image.

<<echo=FALSE>>=
library(gridSVG)
gridsvg("testimage-raster.svg", width=1, height=.5)
testimage("raster", raster=TRUE)
dev.off()
# for inclusion
system("convert -density 300x300 testimage-raster.svg testimage-raster.png")
@ 
<<>>=
convertPicture("testimage-raster.svg", "testimage-raster-cairo.svg")
image <- readPicture("testimage-raster-cairo.svg")
@ 
<<>>=
grid.picture(image)
@ 
<<echo=FALSE, results=hide>>=
pdf("testimage-raster-grimport2.pdf", width=2, height=1)
rpic(function() { grid.picture(image) }, lab=2)
dev.off()
@ 
\begin{figure}[htbp]
  \hspace*{\fill}
  % what it is supposed to look like
  \includegraphics[width=1.5in]{testimage-raster.png}
  \hspace*{\fill}
  \includegraphics[width=1.5in]{testimage-raster-grimport2.pdf}
  \hspace*{\fill}
  \caption{\label{fig:grimport2-raster}On the left is an SVG test image
    that contains a raster element.  On the right is the result of
    converting the image to Cairo SVG using \pkg{grConvert} then 
    importing and rendering the image using \pkg{grImport2}.
    The important point is that the raster element is retained during
    the import process.}
\end{figure}

The next example shows that \pkg{grImport2} can also correctly
import the test image that contains semitransparency 
(see Figure \ref{fig:grimport2-semitrans}).

<<echo=FALSE>>=
library(gridSVG)
gridsvg("testimage-semitrans.svg", width=1, height=.5)
testimage("semitrans", fill=TRUE, raster=TRUE)
dev.off()
# for inclusion
system("convert -density 300x300 testimage-semitrans.svg testimage-semitrans.png")
@ 
<<>>=
convertPicture("testimage-semitrans.svg", "testimage-semitrans-cairo.svg")
image <- readPicture("testimage-semitrans-cairo.svg")
@ 
<<>>=
grid.picture(image)
@ 
<<echo=FALSE, results=hide>>=
pdf("testimage-semitrans-grimport2.pdf", width=2, height=1)
rpic(function() { grid.picture(image) }, lab=2)
dev.off()
@ 
\begin{figure}[htbp]
  \hspace*{\fill}
  % what it is supposed to look like
  \includegraphics[width=1.5in]{testimage-semitrans.png}
  \hspace*{\fill}
  \includegraphics[width=1.5in]{testimage-semitrans-grimport2.pdf}
  \hspace*{\fill}
  \caption{\label{fig:grimport2-semitrans}On the left is an SVG test image
    that contains a semitransparent fill.  On the right is the result of
    converting the image to Cairo SVG using \pkg{grConvert} then 
    importing and rendering the image using \pkg{grImport2}.
    The important point is that the semitransparent fill
    is correctly reproduced.}
\end{figure}

The final test image, which contains a semitransparent 
gradient (see Figure \ref{fig:grimport-gradient}), 
demonstrates a slightly more advanced use
of the \code{grid.picture()} function.  This image can be 
imported by \pkg{grImport2}, but rendering is hampered by the fact that
the R graphics engine does not support gradient fills.
The \pkg{grImport2} package provides support for complex image features like 
these by integrating with the \pkg{gridSVG} package, which
does have support for gradient fills (among other things).
The following code demonstrates the use of the \code{ext} argument
to the \code{grid.picture()} function that is required to produce a
correct rendering of this test image (see Figure \ref{fig:grimport2-gradient}).
Also notice that the rendering on a standard R graphics device
does not include the gradient fill, but the exported
SVG image (rendered with \pkg{gridSVG}) does include the gradient fill.

<<echo=FALSE>>=
library(gridSVG)
gridsvg("testimage-gradient.svg", width=1, height=.5)
testimage("gradient", gradient=TRUE, raster=TRUE)
dev.off()
# for inclusion
system("convert -density 300x300 testimage-gradient.svg testimage-gradient.png")
@ 
<<>>=
convertPicture("testimage-gradient.svg", "testimage-gradient-cairo.svg")
image <- readPicture("testimage-gradient-cairo.svg")
@ 
<<>>=
grid.picture(image, ext="gridSVG")
@ 
<<>>=
library(gridSVG)
@ 
<<evla=FALSE>>=
grid.export("testimage-gradient-grimport2-gridsvg.svg")
@ 
<<echo=FALSE, results=hide>>=
pdf("testimage-gradient-grimport2.pdf", width=2, height=1)
rpic(function() { grid.picture(image) }, lab=2)
dev.off()
gridsvg("testimage-gradient-grimport2-gridsvg.svg", width=2, height=1)
rpic(function() { grid.picture(image, ext="gridSVG") }, lab=3)
dev.off()
# for inclusion
system("convert -density 300x300 testimage-gradient-grimport2-gridsvg.svg testimage-gradient-grimport2-gridsvg.png")
@ 
\begin{figure}[htbp]
  \hspace*{\fill}
  % what it is supposed to look like
  \includegraphics[width=1.5in]{testimage-gradient.png}
  \hspace*{\fill}
  \includegraphics[width=1.5in]{testimage-gradient-grimport2.pdf}
  \hspace*{\fill}
  \includegraphics[width=1.5in]{testimage-gradient-grimport2-gridsvg.png}
  \hspace*{\fill}
  \caption{\label{fig:grimport2-gradient}On the left is an SVG test image
    that contains a semitransparent gradient fill.  
    In the middle is the result of 
    converting the image to Cairo SVG using \pkg{grConvert} then 
    importing and rendering the image using \pkg{grImport2} on
    a standard R graphics device;  although the gradient fill is imported,
    the standard graphics device  cannot render it.
    On the right is the result of rendering the imported image as an
    SVG file using 
    \pkg{gridSVG}, which is capable of rendering the gradient fill.}
\end{figure}

\section{An image test suite}

The original motivation for the new packages arose from a problem 
posed by Toby Dylan Hocking, which involved importing the entire set
of flags of the U.S. 
states.  These images are available from Wikipedia in an SVG 
format.\footnote{\url{http://en.wikipedia.org/wiki/State_flags}}
We will use several of the flags 
of the U.S. states
to demonstrate some further examples of the improved performance of 
\pkg{grImport2} over \pkg{grImport}.

First up is the state flag of Oklahoma, just
to show that, if an image consists of only paths and text, even
if there are lots of complicated paths, both
the old \pkg{grImport} package and the new \pkg{grImport2}
package produce a good result (see Figure \ref{fig:oklahoma}).

% Only needs doing once so eval=FALSE
<<eval=FALSE, echo=FALSE, results=hide>>=
# For inclusion
system("convert -density 300x300 Flag_of_Oklahoma.svg Flag_of_Oklahoma.png")
@ 
<<>>=
convertPicture("Flag_of_Oklahoma.svg", "Flag_of_Oklahoma.ps")
PostScriptTrace("Flag_of_Oklahoma.ps", "Flag_of_Oklahoma.xml")
oklahomaPS <- grImport::readPicture("Flag_of_Oklahoma.xml")
@
<<eval=FALSE>>=
grImport::grid.picture(oklahomaPS)
@ 
<<oklahoma-grimport, echo=FALSE, fig=TRUE, include=FALSE, width=2, height=1>>=
rpic(function() { grImport::grid.picture(oklahomaPS) })
@ 
<<>>=
convertPicture("Flag_of_Oklahoma.svg", "Flag_of_Oklahoma_cairo.svg")
oklahoma <- readPicture("Flag_of_Oklahoma_cairo.svg")
@ 
<<eval=FALSE>>=
grid.picture(oklahoma)
@ 
% Don't know why I have to create PDF myself here
<<oklahoma-grimport2, echo=FALSE, results=hide>>=
pdf("oklahoma-grimport2.pdf", width=2, height=1)
rpic(function() { grid.picture(oklahoma) }, lab=2)
dev.off()
@ 

\begin{figure}[htbp]
  \hspace*{\fill}
  \includegraphics[width=1.5in]{Flag_of_Oklahoma.png}
  \hspace*{\fill}
  \includegraphics[width=1.5in]{potter-murrell-oklahoma-grimport}
  \hspace*{\fill}
  \includegraphics[width=1.5in]{oklahoma-grimport2}
  \caption{\label{fig:oklahoma}At left, the state flag of Oklahoma as an SVG image
  (from Wikipedia).  In the middle, the flag after importing and rendering
  with \pkg{grImport}.  At right, the flag after importing and rendering
  with \pkg{grImport2}.  Although the image is complex, it only contains
  paths and text, so both packages reproduce the image faithfully.}
\end{figure}

The next example, the state flag of Kansas,
provides a more dramatic demonstration of 
the ability to import gradient fills.  On the state seal at the
centre of the flag, the sky, the sea, and the land
all contain linear gradient fills.
As with the test image that contained a gradient fill
(Figure \ref{fig:grimport-gradient}), a correct \emph{rendering} 
of the Kansas flag requires
the involvement of the \pkg{gridSVG} package (see 
Figure \ref{fig:kansas}).

% Only needs doing once so eval=FALSE
<<eval=FALSE, echo=FALSE, results=hide>>=
# For inclusion
system("convert -density 100x100 Flag_of_Kansas.svg Flag_of_Kansas.png")
@ 
<<kansas-grimport2-gridsvg>>=
convertPicture("Flag_of_Kansas.svg", "Flag_of_Kansas_cairo.svg")
kansas <- readPicture("Flag_of_Kansas_cairo.svg")
grid.picture(kansas, ext="gridSVG")
@ 
<<eval=FALSE>>=
grid.export("kansas-grimport2-gridsvg.svg")
@ 
<<echo=FALSE, result=hide>>=
gridsvg("kansas-grimport2-gridsvg.svg", width=2, height=1)
rpic(function() { grid.picture(kansas, ext="gridSVG") }, lab=3)
dev.off()
# For inclusion
system("convert -density 300x300 kansas-grimport2-gridsvg.svg kansas-grimport2-gridsvg.png")
@ 
<<kansas-grimport2, echo=FALSE, fig=TRUE, include=FALSE, width=2, height=1>>=
rpic(function() { grid.picture(kansas) }, lab=2)
@ 

\begin{figure}[htbp]
  \hspace*{\fill}
  % import and draw with gridSVG
  \includegraphics[width=1.5in]{Flag_of_Kansas.png}
  \hspace*{\fill}
  \includegraphics[width=1.5in]{potter-murrell-kansas-grimport2}
  \hspace*{\fill}
  \includegraphics[width=1.5in]{kansas-grimport2-gridsvg.png}
  \hspace*{\fill}
  \caption{\label{fig:kansas}At left, the state flag of Kansas as an SVG image
  (from Wikipedia).  In the middle, the flag after importing and rendering
  with \pkg{grImport2} on a standard R graphics device; the standard
  device cannot render the gradient fills, which results in black regions
  on the flag.
  At right, the flag after importing with \pkg{grImport2} and rendering
  with \pkg{gridSVG}. }
\end{figure}

The state flag of Hawaii contains a feature that we have not previously
encountered:  \dfn{clipping}.
The Union Jack at the top-left of the flag is drawn as a collection of
simple shapes and lines, with clipping used to limit the visible output
to just the top-left corner of the flag.  This presents a problem for
the \pkg{grImport} package because that package ignores clipping 
information (see Figure \ref{fig:hawaii}).  

The \pkg{grImport2} package imports the clipping information for an image,
but, similar to gradient fills, the rendering of clipping information is
hampered by the limitations of the R graphics engine, which can only
clip to rectangular regions.  The \code{ext} argument to
the \code{grid.picture()} function
can be given the value \code{"clipbbox"} to force
all clipping information in an image to be converted to bounding boxes
(i.e., rectangular clipping regions) for rendering.  
In the case of the state flag of Hawaii,
this works well 
because the clipping regions in the original image are
already rectangles (see Figure \ref{fig:hawaii}).

% Only needs doing once so eval=FALSE
<<eval=FALSE, echo=FALSE, results=hide>>=
# For inclusion
system("convert -density 300x300 Flag_of_Hawaii.svg Flag_of_Hawaii.png")
@ 
<<hawaii-grimport, echo=FALSE, fig=TRUE, include=FALSE, width=2, height=1>>=
convertPicture("Flag_of_Hawaii.svg", "Flag_of_Hawaii.ps")
PostScriptTrace("Flag_of_Hawaii.ps", "Flag_of_Hawaii.xml")
hawaiiPS <- grImport::readPicture("Flag_of_Hawaii.xml")
rpic(function() { grImport::grid.picture(hawaiiPS) })
@ 
% Don't know why I have to create PDF myself here
<<hawaii-grimport2, echo=FALSE, results=hide>>=
convertPicture("Flag_of_Hawaii.svg", "Flag_of_Hawaii_cairo.svg")
hawaii <- readPicture("Flag_of_Hawaii_cairo.svg")
pdf("hawaii-grimport2.pdf", width=2, height=1)
rpic(function() { grid.picture(hawaii, ext="clipbbox") }, lab=4)
dev.off()
@ 

\begin{figure}[htbp]
  \hspace*{\fill}
  % import and draw with gridSVG
  \includegraphics[width=1.5in]{Flag_of_Hawaii.png}
  \hspace*{\fill}
  \includegraphics[width=1.5in]{potter-murrell-hawaii-grimport}
  \hspace*{\fill}
  \includegraphics[width=1.5in]{hawaii-grimport2}
  \hspace*{\fill}
  \caption{\label{fig:hawaii}At left, the state flag of Hawaii as an SVG image
  (from Wikipedia).  In the middle, the flag after importing and rendering
  with \pkg{grImport}, which does not import clipping information.  
  At right, the flag after importing and rendering
  with \pkg{grImport2}.}
\end{figure}

The state flag of Ohio presents the more complex clipping case, where 
the clipping regions are not rectangular (see Figure \ref{fig:ohio}).  
In this situation, there is no hope of producing the correct result
on a standard R graphics device, but, as for gradient fills,
we can use \code{ext="gridSVG"} and export to SVG
to obtain the correct rendering (see Figure \ref{fig:ohio}).

% Only needs doing once so eval=FALSE
<<eval=FALSE, echo=FALSE, results=hide>>=
# For inclusion
system("convert -density 300x300 Flag_of_Ohio.svg Flag_of_Ohio.png")
@ 
<<ohio-grimport2-gridsvg>>=
convertPicture("Flag_of_Ohio.svg", "Flag_of_Ohio_cairo.svg")
ohio <- readPicture("Flag_of_Ohio_cairo.svg")
grid.picture(ohio, ext="gridSVG")
@ 
<<>>=
grid.export("Flag_of_Ohio_gridsvg.svg")
@ 
<<echo=FALSE, result=hide>>=
gridsvg("ohio-grimport2-gridsvg.svg", width=2, height=1)
rpic(function() { grid.picture(ohio, ext="gridSVG") }, lab=3)
dev.off()
# For inclusion
system("convert -density 300x300 ohio-grimport2-gridsvg.svg ohio-grimport2-gridsvg.png")
@ 
<<ohio-grimport2, echo=FALSE, fig=TRUE, include=FALSE, width=2, height=1>>=
rpic(function() { grid.picture(ohio) }, lab=2)
@ 

\begin{figure}[htbp]
  \hspace*{\fill}
  % import and draw with gridSVG
  \includegraphics[width=1.5in]{Flag_of_Ohio.png}
  \hspace*{\fill}
  \includegraphics[width=1.5in]{potter-murrell-ohio-grimport2}
  \hspace*{\fill}
  \includegraphics[width=1.5in]{ohio-grimport2-gridsvg.png}
  \hspace*{\fill}
  \caption{\label{fig:ohio}At left, the state flag of Ohio as an SVG image
  (from Wikipedia).  In the middle, the flag after importing and rendering
  with \pkg{grImport2} on a standard R graphics device;
  the non-rectangular clipping information in this flag cannot be
  rendered correctly on a standard graphics device.  
  At right, the flag after importing with \pkg{grImport2} and rendering
  with \pkg{gridSVG}.}
\end{figure}

In addition to gradient fills and clipping paths, 
\pkg{grImport2} can
also import images that make use of masks and pattern fills,
although correct rendering of these features will again require
the help of the \pkg{gridSVG} package.

\section{A plot example}

The main point of importing images into R is not simply to
replicate the image by itself, but to incorporate the image
as part of an R graphics scene that contains other drawing.
This section presents an example of the latter, incorporating
an imported image within a \pkg{lattice} plot \citep{pkg:lattice}.

The external image is a balloon icon (an SVG image) by Yuko Iwai, 
from thenounproject.com 
collection\footnote{\url{http://thenounproject.com/term/balloon/3663/}}
(see Figure \ref{fig:balloon}).
We will use this image within a plot to show
the popularity of the top 5 male baby names (in New Zealand
in 2000\footnote{\url{http://www.stats.govt.nz/~/media/Statistics/browse-categories/population/births/tables/babies-names.xls}}).

<<echo=FALSE>>=
babynames <- read.csv("babies-names.csv", skip=7, nrow=5)
names <- factor(babynames$X.6, levels=babynames$X.6)
counts <- babynames$X.7
@ 

<<balloon, fig=TRUE, include=FALSE>>=
convertPicture("noun_project_3663.svg", "balloon.svg")
balloon <- readPicture("balloon.svg")
grid.picture(balloon)
@ 

\begin{figure}[htbp]
  \hspace*{\fill}
  \includegraphics[width=1.5in]{balloon}
  \hspace*{\fill}
  \caption{\label{fig:balloon}A balloon icon by Yuko Iwai, 
    from thenounproject.com collection.}
\end{figure}

The following code draws a \pkg{lattice} plot of the number of male babies
with the given name (for the top 5 names),
with a custom panel function defined in the 
\code{xyplot()} call to add the ballon icon as a plotting symbol.
This code demonstrates another function in the \pkg{grImport2} package,
\code{grid.symbols()}, which is useful for drawing multiple copies
of an imported image at once.

<<>>=
library(lattice)
library(grid)
@ 

<<latticeballoon, eval=FALSE>>=
xyplot(counts ~ names, pch=16,
       type="h", lty="dotted", lwd=3, col="grey",
       ylab="", ylim=c(300, 750),
       xlab="Most Popular Male Baby Names (NZ 2000)",
       scales=list(x=list(draw=FALSE)),
       panel=function(x, y, ...) {
           panel.xyplot(x, y, ...)
           grid.symbols(balloon, x, y, size=unit(3.5, "cm"))
       })
@ 
<<echo=FALSE, results=hide>>=
pdf("latticeballoon.pdf")
print(
<<latticeballoon>>
)
dev.off()
@ 

Figure \ref{fig:latticeballoon} shows a slightly more complex result
than that produced by the code above,
with a few extra embellishments added.  In particular, it 
demonstrates that the imported image can be modified by altering the
graphical parameter settings, such as colour, when drawing the image
(full code for this figure is available from the second author's web site).

<<echo=FALSE, results=hide>>=
library(grid)
cols <- hcl(seq(80, 260, length=5), 80, 60, .7)
cols2 <- hcl(seq(80, 260, length=5), 80, 40, .7)
makecol <- function() {
    colindex <- 0
    function() {
        colindex <<- colindex + 1
	cols[colindex]
    }
}
getcol <- makecol()
adjustFill <- function(gp) {
    if (!is.null(gp$fill))
        gp$fill <- getcol()
    gp
}
pdf("names.pdf", useDingbats=FALSE)
print(xyplot(counts ~ names, type="h", lty="dotted", lwd=3, col="grey",
             ylab="", ylim=c(300, 750),
             xlab="Most Popular Male Baby Names (NZ 2000)",
             scales=list(x=list(draw=FALSE)),
             par.settings=list(axis.line=list(lwd=3, col="grey60"),
                 axis.text=list(col="grey60")),
             panel=function(x, y, ...) {
                 panel.xyplot(x, y, ...)
                 grid.symbols(balloon, x, y,
                              size=unit(3.5, "cm"),
                              gpFUN=adjustFill)
                 grid.circle(x, y, r=unit(1, "mm"),
                             default="native",
                             gp=gpar(col=NA, fill=cols2))
                 grid.text(names, unit(x, "native"),
                           unit(y, "native") + unit(1, "lines"),
                           default="native", 
                           gp=gpar(col="white"))
             }))
dev.off()
@ 

\begin{figure}[htbp]
  \hspace*{\fill}
  \includegraphics[width=4.5in]{names}
  \hspace*{\fill}
  \caption{\label{fig:latticeballoon}The balloon icon from Figure 
    \ref{fig:balloon} used as a plotting symbol in a \pkg{lattice} plot.}
\end{figure}

\section{Summary}

The following list summarises the main scenarios that arise when we
want to import an external image and use it within an R graphic:

\begin{compactitem}
\item
  If the original image is in \ps{} format and consists only of text and paths
(e.g., no raster elements and no clipping),
the \pkg{grImport} package should work well.
\item
If the original image is not in \ps{} format, but consists only of text
and paths, then it should convert well to \ps{} for import via \pkg{grImport}.
\item
If the original image is not in \ps{} format and/or includes complex
content such as raster elements, gradient fills, or clipping paths,
it should be possible to convert the image to SVG via \pkg{grConvert}
and then import with \pkg{grImport2}.
\item
Rendering of some complex images will only be possible in an SVG format
with the support of the \pkg{gridSVG} package.
\end{compactitem}

\section{Discussion}

The main body of this article 
has demonstrated some of the limitations of the old
\pkg{grImport} package and some of the benefits of the new 
\pkg{grImport2} package.  This section expands the discussion to
include some weaknesses in the new \pkg{grImport2} package
and 
some redeeming features of the old \pkg{grImport} package.

First of all, despite being able to import a wider range of 
images, there are still some images that \pkg{grImport2} cannot
import and render correctly.  Figure \ref{fig:arizona} provides
an example from the state flag of Arizona.
In this case, the problem is that the yellow "rays" are drawn
with a very thick dashed  line.  The \pkg{grImport2} package
cannot import this image correctly because the dash pattern
cannot be properly represented in R graphics.

% Only needs doing once so eval=FALSE
<<eval=FALSE, echo=FALSE, results=hide>>=
# For inclusion
system("convert -density 300x300 Flag_of_Arizona.svg Flag_of_Arizona.png")
@ 
<<arizona-grimport2-gridsvg>>=
convertPicture("Flag_of_Arizona.svg", "Flag_of_Arizona_cairo.svg")
arizona <- readPicture("Flag_of_Arizona_cairo.svg")
grid.picture(arizona)
@ 
<<arizona-grimport2, echo=FALSE, fig=TRUE, include=FALSE, width=2, height=1>>=
rpic(function() { grid.picture(arizona) }, lab=2)
@ 

\begin{figure}[htbp]
  \hspace*{\fill}
  % import and draw with gridSVG
  \includegraphics[width=1.5in]{Flag_of_Arizona.png}
  \hspace*{\fill}
  \includegraphics[width=1.5in]{potter-murrell-arizona-grimport2}
  \hspace*{\fill}
  \caption{\label{fig:arizona}At left, the state flag of Arizona as an SVG image
  (from Wikipedia).  At right, the flag after importing and rendering
  with \pkg{grImport2} on a standard R graphics device.}
\end{figure}

Another limitation of the \pkg{grImport2} package is that it cannot 
import text content from the original image as text.  This is because 
the Cairo SVG subset does not support text;  it represents all text 
as paths that draw the individual characters.  This means that
the R version of an imported image will not contain any text;  for
example, it will not be possible to search for text values within a PDF file
produced by R from an imported image.

This leads us to one of the advantages of the old \pkg{grImport} package
because that package is capable of importing text from an external image and
rendering it as text.  Another advantage of \pkg{grImport} (compared to 
\pkg{grImport2}) is the relative simplicity of the data structure 
that is used to represent an imported image.  The practical result
of this difference is that it is a simple matter to extract a subset from
the components an imported image with \pkg{grImport}, if we only
want to render a small part of an imported image rather than the entire image.
This subsetting of an imported image is much harder to do with
\pkg{grImport2}.  The \pkg{grImport2} package also currently has no
support for rendering an imported image as part of a scene based on the
standard \pkg{graphics} package.

A final reason for retaining the old \pkg{grImport} package 
is the fact that the conversion of an image from a format other
than SVG to Cairo SVG 
can introduce problems, such as loss of features from
the original image.  This means that, if the original image is 
in a \ps{} format, the best result may be obtained 
by importing and rendering with \pkg{grImport} rather than
converting the image to SVG with \pkg{grConvert} and importing
and rendering with
\pkg{grImport2}.

% Something for JSS article?
% (non-group) Animation may not work on grobs that have makeContent() method
% because the forced grob may not have attributes that the original grob
% had (so animation may have no effect)

\section{Links to other information}

The \pkg{grConvert} and \pkg{grImport2} packages are currently 
available from 
R-Forge.\footnote{\url{https://r-forge.r-project.org/projects/grimport/}}
A more detailed discussion of \pkg{grConvert} and \pkg{grImport2}
packages is provided in \citet{techreport} and
the complete set of state flags and how well they are rendered by
\pkg{grImport} and \pkg{grImport2} is available from the
following link:\\
\url{https://dl.dropboxusercontent.com/u/54315147/import/state-table.html}.

\section{Acknowledgements}

Simon Potter's 
work on the \pkg{grConvert} and \pkg{grImport2} packages was funded by
a Google Summer of Code scholarship.  The Summer of Code
project was originally proposed by Toby Dylan Hocking.

\bibliography{potter-murrell}

\address{Simon Potter\\
  The University of Auckland\\
  Auckland\\
  New Zealand}
\email{simon@sjp.co.nz}

\address{Paul Murrell\\
  The University of Auckland\\
  Auckland\\
  New Zealand}
\email{paul@stat.auckland.ac.nz}