This web page provides a demonstration of a data set that is published on the web in the form of multiple URL links, with a strong pattern to the URLs. This example effectively doubles up as an example of a Web API, which is where data are deliberately made available via a URL pattern.
The data set is available as a set of CSV files:
20130901103549-NAV-NZ2.csv
20130901103549-NAV-ATA.csv
20130901103549-NAV-POR.csv
20130901103549-NAV-AUS.csv
20130901103549-NAV-PRO.csv
20130901103549-NAV-CN1.csv
20130901103549-NAV-RBO.csv
20130901103549-NAV-CO1.csv
20130901103549-NAV-REL.csv
20130901103549-NAV-CO2.csv
20130901103549-NAV-SUI.csv
20130901103549-NAV-FRA.csv
20130901103549-NAV-SWE.csv
20130901103549-NAV-GER.csv
20130901103549-NAV-U1.csv
20130901103549-NAV-HL1.csv
20130901103549-NAV-US1.csv
20130901103549-NAV-HL2.csv
20130901103549-NAV-US2.csv
20130901103549-NAV-MIS.csv
20130901103549-NAV-VOL.csv
20130901103549-NAV-NZ1.csv
20130901103549-NAV-XM1.csv
In this case, all we need to do is generate an appropriate set of URLs with string processing tools, as shown below.
countries <- c("NZ1", "AUS", "US1") urls <- sprintf("https://www.stat.auckland.ac.nz/~paul/stats769/web-scraping/AC34/20130901103549-NAV-%s.csv", countries) urls
## [1] "https://www.stat.auckland.ac.nz/~paul/stats769/web-scraping/AC34/20130901103549-NAV-NZ1.csv" ## [2] "https://www.stat.auckland.ac.nz/~paul/stats769/web-scraping/AC34/20130901103549-NAV-AUS.csv" ## [3] "https://www.stat.auckland.ac.nz/~paul/stats769/web-scraping/AC34/20130901103549-NAV-US1.csv"
races <- do.call(rbind, lapply(urls, read.csv)) head(races)
## Boat Date Secs LocalTime Zone Lat Lon Hdg Heel ## 1 NZ1 01:09:2013 38150.0 10:35:49.997 -7 37.80866 -122.4493 114.0 7.9 ## 2 NZ1 01:09:2013 38150.2 10:35:50.197 -7 37.80865 -122.4493 113.6 8.2 ## 3 NZ1 01:09:2013 38150.4 10:35:50.397 -7 37.80864 -122.4493 113.1 8.2 ## 4 NZ1 01:09:2013 38150.6 10:35:50.597 -7 37.80863 -122.4493 112.7 8.3 ## 5 NZ1 01:09:2013 38150.8 10:35:50.797 -7 37.80862 -122.4493 112.3 8.4 ## 6 NZ1 01:09:2013 38151.0 10:35:50.997 -7 37.80861 -122.4492 111.6 8.6 ## Pitch COG SOG CourseWindDirection CourseWindSpeed yHdg ySpeed yTWS ## 1 0.4 116.3 22.17 255.0 17.9 0 0 0 ## 2 0.4 115.8 22.19 255.0 17.9 0 0 0 ## 3 0.5 115.1 22.02 254.9 17.9 0 0 0 ## 4 0.5 114.9 21.90 254.9 17.9 0 0 0 ## 5 0.7 114.9 21.84 254.9 17.9 0 0 0 ## 6 0.9 114.9 21.95 254.9 17.9 0 0 0 ## yTWD yAWS yAWA yTWA ySOG yCOG yRudder ## 1 0 0 0 0 0 0 0 ## 2 0 0 0 0 0 0 0 ## 3 0 0 0 0 0 0 0 ## 4 0 0 0 0 0 0 0 ## 5 0 0 0 0 0 0 0 ## 6 0 0 0 0 0 0 0