Type:	Package
Title:	Read-Write API Client Library for Wikidata
Version:	2.3.3
Date:	2021-11-16
Description:	Read from, interogate, and write to Wikidata https://www.wikidata.org - the multilingual, interdisciplinary, semantic knowledgebase. Includes functions to: read from wikidata (single items, properties, or properties); query wikidata (retrieving all items that match a set of criterial via Wikidata SPARQL query service); write to Wikidata (adding new items or statements via QuickStatements); and handle and manipulate Wikidata objects (as lists and tibbles). Uses the Wikidata and Quickstatements APIs.
BugReports:	https://github.com/TS404/WikidataR/issues
URL:	https://github.com/TS404/WikidataR
License:	MIT + file LICENSE
Imports:	httr, jsonlite, WikipediR, WikidataQueryServiceR, tibble, dplyr, stringr, Hmisc, progress, pbapply, stats, readr, crayon, utils
Suggests:	markdown, testthat, tidyverse, knitr, pageviews
RoxygenNote:	7.1.1
Encoding:	UTF-8
Depends:	R (≥ 3.5.0)
NeedsCompilation:	no
Packaged:	2021-11-16 11:48:41 UTC; thoma
Author:	Thomas Shafee [aut, cre], Os Keyes [aut], Serena Signorelli [aut], Alex Lum [ctb], Christian Graul [ctb], Mikhail Popov [ctb]
Maintainer:	Thomas Shafee <T.Shafee@latrobe.edu.au>
Repository:	CRAN
Date/Publication:	2021-11-16 13:00:06 UTC

API client library for Wikidata

Description

This package serves as an API client for reading and writing to and from Wikidata, (including via the QuickStatements format), as well as for reading from Wikipedia.

See Also

get_random for selecting a random item or property, get_item for a /specific/ item or property, or find_item for using search functionality to pull out item or property IDs where the descriptions or aliases match a particular search term.

Global variables for Wikidata properties

Description

A dataset of Wikidata global variables.

Format

A list of tibbles documenting key property constraints from wikidata

SID.valid

valid reference source properties

PID.datatype

required data type for each property

PID.constraint

expected regex match for each property

lang.abbrev

language abbreviations

lang.abbrev.wiki

language abbreviations for current wikis

abbrev.wiki

Wikimedia abbreviations for current wikis

...

Convert an input to a property PID

Description

Convert an input string to the most likely property PID

Usage

as_pid(x)

Arguments

Value

if the inputted string is a valid PID, return the string. If the inputted string matches a property label, return its PID. If the inputted string matches multiple labels of multiple properties, return the PID of the first hit.

Examples

# if input string is a valid PID
as_pid("P50")
# if input string matches multiple item labels
as_pid("author")
# if input string matches a single unique label
as_pid("Scopus author ID")

Convert an input to a item QID

Description

Convert an input string to the most likely item QID

Usage

as_qid(x)

Arguments

Value

if the inputted string is a valid QID, return the string. If the inputted string matches an item label, return its QID. If the inputted string matches multiple labels of multiple items, return the QID of the first hit.

Examples

# if input string is a valid QID
as_qid("Q42")
# if input string matches multiple item labels
as_qid("Douglas Adams")
# if input string matches a single unique label
as_qid("Douglas Adams and the question of arterial blood pressure in mammals")

Add quotations marks

Description

Add escaped quotation marks around strings that need them ready for submision to an API

Usage

as_quot(x, format = "tibble")

Arguments

Value

tibble of items inside of escaped quotation marks unless they are already in escaped quotation marks, is a QID, (in which chase it is returned unchanged)

Examples

as_quot("text")

Convert an input to a source property SID

Description

Convert an input string to the most likely source SID (equivalent to PID)

Usage

as_sid(x)

Arguments

Value

if the inputted string is a valid SID, return the string. If the inputted string matches a property label, return its SID If the inputted string matches multiple labels of multiple properties, return the SID of the first hit.

Examples

# if input string is a valid SID
as_pid("S854")
# if input string matches multiple item labels
as_pid("URL")
# if input string matches a single unique label
as_pid("Reference URL")

Generic input checker

Description

Utility function to handle namespaces. Used by get_item and get_property

Usage

check_input(input, substitution)

Arguments

Value

boolian indicating whether the checked string contains a match for the substitution string

"CREATE" rows

Description

Add in empty lines for QuickStatements CREATE rows that mint new QIDs. This is a slightly messy quirk of the QuickStatements format that mints new QIDs via a line containing only "CREATE", so this function is a way to approximate that bevaviour in a tibble

Usage

createrows(items, vector)

Arguments

Value

if the vector is NULL, return NULL. Otherwise, if the "CREATE" keyword appears in the items vector, insert blank strings at those positions in the vector.

"CREATE" rows from tidy format

Description

Add in QuickStatements CREATE rows that mint new QIDs from tidy input data. New items are created by any item starting that starts with the text "CREATE" followed by any unique ID.

Usage

createrows.tidy(QS.tib)

Arguments

Value

a tibble, with items that start with "CREATE" followed by any unique text causing the addition of a "Create" line above, being replaced with "LAST" in the Quickstatemnts format to create new QIDs.

Disambiguate QIDs

Description

Interactive function that presents alternative possible QID matches for a list of text strings and provides options for choosing between alternatives, rejecting all presented alternatives, or creating new items. Useful in cases where a list of text strings may have either missing wikidata items or multiple alternative potential matches that need to be manually disambuguated. Can also used on lists of lists (see examples). For long lists of items, the process can be stopped partway through and the returned vector will indicate where the process was stopped.

Usage

disambiguate_QIDs(
  list,
  variablename = "variables",
  variableinfo = NULL,
  filter_property = NULL,
  filter_variable = NULL,
  filter_firsthit = FALSE,
  limit = 10
)

Arguments

Value

a vector of:

QID

Selected QID (for when an appropriate Wikidata match exists)

CREATE

Mark that a new Wikidata item should be created (for when no appropriate Wikidata match yet exists)

NA

Mark that no Wikidata item is needed

STOP

Mark that the process was halted at this point (so that output can be used as input to the function later)

Examples

## Not run: 
#Disambiguating possible QID matches for these music genres
#Results should be:
# "Q22731" as the first match
# "Q147538" as the first match
# "Q3947" as the second alternative match
disambiguate_QIDs(list=c("Rock","Pop","House"),
                 variablename="music genre")

#Disambiguating possible QID matches for these three words, but not the music genres
#This will take longer as the filtering step is slower
#Results should be:
# "Q22731" (the material) as the first match
# "Q147538" (the soft drink) as the second alternative match
# "Q3947" (the building) as the first match
disambiguate_QIDs(list=c("Rock","Pop","House"),
                 filter_property="instance of",
                 filter_variable="music genre",
                 filter_firsthit=TRUE,
                 variablename="concept, not the music genre")

#Disambiguating possible QID matches for the multiple expertise of
#these three people as list of lists
disambiguate_QIDs(list=list(alice=list("physics","chemistry","maths"),
                           barry=list("history"),
                           clair=list("law","genetics","ethics")),
                 variablename="expertise")

## End(Not run)

Extract Claims from Returned Item Data

Description

extract claim information from data returned using get_item.

Usage

extract_claims(items, claims)

Arguments

Value

a list containing one sub-list for each entry in items, and (below that) the found data for each claim. In the event a claim cannot be found for an item, an NA will be returned instead.

Examples

# Get item data
adams_data <- get_item("42")
# Get claim data
claims <- extract_claims(adams_data, "P31")

Extract a paragraph of text

Description

Return the nth paragraph of a section of text Useful for extracting information from wikipedia or other wikimarkup text

Usage

extract_para(text, para = 1, templ = NULL)

Arguments

Value

the nth paragraph of the input text.

Filter QIDs

Description

For a QID or vector of QIDs, remove ones that match a particular statement (e.g. remove all that are instances of academic publications or books).

Usage

filter_qids(
  ids,
  property = "P31",
  filter = c("Q737498", "Q5633421", "Q7725634", "Q13442814", "Q18918145"),
  message = NULL
)

Arguments

Value

a vector of QIDs that do not match the property filter

Examples

## Not run: 
# Filter three items called "Earth Science" to show only those that aren't
# books, journals or journal articles
filter_qids(ids = c("Q96695546","Q8008","Q58966429"),
            property = "P31",
            filter = c("Q737498","Q5633421","Q7725634","Q13442814","Q18918145"))

## End(Not run)

Search for Wikidata items or properties that match a search term

Description

find_item and find_property allow you to retrieve a set of Wikidata items or properties where the aliase or descriptions match a particular search term. As with other WikidataR code, custom print methods are available; use str to manipulate and see the underlying structure of the data.

Usage

find_item(search_term, language = "en", limit = 10, ...)

find_property(search_term, language = "en", limit = 10)

Arguments

See Also

get_random for selecting a random item or property, or get_item for selecting a specific item or property.

Examples

#Check for entries relating to Douglas Adams in some way
adams_items <- find_item("Douglas Adams")

#Check for properties involving the peerage
peerage_props <- find_property("peerage")

Get an example SPARQL query from Wikidata

Description

Gets the specified example(s) from [SPARQL query service examples page](https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/queries/examples) using [Wikidata's MediaWiki API](https://www.wikidata.org/w/api.php).

Usage

get_example(example_name)

Arguments

Details

If you are planning on extracting multiple examples, please provide all the names as a single vector for efficiency.

Value

The SPARQL query as a character vector.

See Also

[query_wikidata]

Examples

## Not run: 
sparql_query <- extract_example(c("Cats", "Horses"))
query_wikidata(sparql_query)
# returns a named list with two data frames
# one called "Cats" and one called "Horses"
sparql_query <- extract_example("Largest cities with female mayor")
cat(sparql_query)
query_wikidata(sparql_query)

## End(Not run)

Get geographic entities based on a bounding box

Description

get_geo_box retrieves all geographic entities in Wikidata that fall between a bounding box between two existing items with geographic attributes (usually cities).

Usage

get_geo_box(
  first_city_code,
  first_corner,
  second_city_code,
  second_corner,
  language = "en",
  ...
)

Arguments

Value

a data.frame of 5 columns:

• item the Wikidata identifier of each object associated with entity.

• name the name of the item, if available, in the requested language. If it is not available, NA will be returned instead.

• latitude the latitude of item

• longitude the longitude of item

• entity the entity the item is associated with (necessary for multi-entity queries).

See Also

get_geo_entity for using an unrestricted search or simple radius, rather than a bounding box.

Examples

# Simple bounding box
bruges_box <- get_geo_box("Q12988", "NorthEast", "Q184287", "SouthWest")

# Custom language
bruges_box_fr <- get_geo_box("Q12988", "NorthEast", "Q184287", "SouthWest",
                                        language = "fr")

Retrieve geographic information from Wikidata

Description

get_geo_entity retrieves the item ID, latitude and longitude of any object with geographic data associated with another object with geographic data (example: all the locations around/near/associated with a city).

Usage

get_geo_entity(entity, language = "en", radius = NULL, limit = 100, ...)

Arguments

Value

a data.frame of 5 columns:

• item the Wikidata identifier of each object associated with entity.

• name the name of the item, if available, in the requested language. If it is not available, NA will be returned instead.

• latitude the latitude of item

• longitude the longitude of item

• entity the entity the item is associated with (necessary for multi-entity queries).

See Also

get_geo_box for using a bounding box rather than an unrestricted search or simple radius.

Examples

# All entities
sf_locations <- get_geo_entity("Q62")

# Entities with French, rather than English, names
sf_locations <- get_geo_entity("Q62", language = "fr")

# Entities within 1km
sf_close_locations <- get_geo_entity("Q62", radius = 1)

# Multiple entities
multi_entity <- get_geo_entity(entity = c("Q62", "Q64"))

Retrieve specific Wikidata items or properties

Description

get_item and get_property allow you to retrieve the data associated with individual Wikidata items and properties, respectively. As with other WikidataR code, custom print methods are available; use str to manipulate and see the underlying structure of the data.

Usage

get_item(id, ...)

get_property(id, ...)

Arguments

See Also

get_random for selecting a random item or property, or find_item for using search functionality to pull out item or property IDs where the descriptions or aliases match a particular search term.

Examples

#Retrieve a specific item
adams_metadata <- get_item("42")

#Retrieve a specific property
object_is_child <- get_property("P40")

Get names of properties

Description

For a claim or set of claims, return the names of the properties

Usage

get_names_from_properties(properties)

Arguments

Value

tibble of labels for each property for a set of claims

Retrieve randomly-selected Wikidata items or properties

Description

get_random_item and get_random_property allow you to retrieve the data associated with randomly-selected Wikidata items and properties, respectively. As with other WikidataR code, custom print methods are available; use str to manipulate and see the underlying structure of the data.

Usage

get_random_item(limit = 1, ...)

get_random_property(limit = 1, ...)

Arguments

See Also

get_item for selecting a specific item or property, or find_item for using search functionality to pull out item or property IDs where the descriptions or aliases match a particular search term.

Examples

## Not run: 
#Random item
random_item <- get_random_item()

#Random property
random_property <- get_random_property()

## End(Not run)

identifier from identifier

Description

convert unique identifiers to other unique identifiers

Usage

identifier_from_identifier(
  property = "ORCID iD",
  return = "IMDb ID",
  value = "0000-0002-7865-7235"
)

Arguments

Value

vector of identifiers corresponding to identifiers submitted

Examples

identifier_from_identifier('ORCID iD','IMDb ID',c('0000-0002-7865-7235','0000-0003-1079-5604'))

Format short form person names

Description

Converting names into first initial and surname, or just initials

Usage

initials(x, format = "FLast")

Arguments

Value

the inputted name strings with first names shortened based on the selected format.

List properties of a Wikidata item

Description

for a downloaded wikidata item, list the properties of all statements

Usage

list_properties(item, names = FALSE)

Arguments

Value

a list containing one sub-list for each entry in items, and (below that) the found data for each claim. In the event a claim cannot be found for an item, an NA will be returned instead.

Examples

# Get item data
adams_data <- get_item("42")
# Get claim data
claims <- extract_claims(adams_data, "P31")

Print method for find_item

Description

print found items.

Usage

## S3 method for class 'find_item'
print(x, ...)

Arguments

Print method for find_property

Description

print found properties.

Usage

## S3 method for class 'find_property'
print(x, ...)

Arguments

Print method for Wikidata objects

Description

print found objects generally.

Usage

## S3 method for class 'wikidata'
print(x, ...)

Arguments

See Also

get_item, get_random_item, get_property or get_random_property

QID from DOI

Description

simple converter from DOIs to QIDs (for items in wikidata)

Usage

qid_from_DOI(DOI = "10.15347/WJM/2019.001")

Arguments

Value

vector of QIDs corresponding to DOIs submitted

QID from ORCID

Description

simple converter from ORCIDs to QIDs (for items in wikidata)

Usage

qid_from_ORCID(ORCID = "0000-0002-2298-7593")

Arguments

Value

vector of QIDs corresponding to ORCIDs submitted

QID from identifier

Description

convert unique identifiers to QIDs (for items in wikidata).

Usage

qid_from_identifier(
  property = "DOI",
  value = c("10.15347/WJM/2019.001", "10.15347/WJM/2020.002")
)

Arguments

Value

vector of QIDs corresponding to identifiers submitted

Examples

qid_from_identifier('ISBN-13','978-0-262-53817-6')

QID from label name

Description

simple converter from label names to QIDs (for items in wikidata). Essentially a simplification of find_item

Usage

qid_from_name(name = "Thomas Shafee", limit = 100, format = "vector")

Arguments

Value

vector of QIDs corresponding to names submitted. Note: some names may return multiple QIDs.

Send one or more SPARQL queries to WDQS

Description

Makes a POST request to Wikidata Query Service SPARQL endpoint.

Usage

query_wikidata(sparql_query, format = "tibble", ...)

Arguments

Value

A 'tibble' or 'vector'. Note: QID values will be returned as QIDs, rather than URLs.

Query limits

There is a hard query deadline configured which is set to 60 seconds. There are also following limits: - One client (user agent + IP) is allowed 60 seconds of processing time each 60 seconds - One client is allowed 30 error queries per minute See [query limits section](https://www.mediawiki.org/wiki/Wikidata_Query_Service/User_Manual#Query_limits) in the WDQS user manual for more information.

Examples

# R's versions and release dates:
sparql_query <- 'SELECT DISTINCT
  ?softwareVersion ?publicationDate
  WHERE {
    BIND(wd:Q206904 AS ?R)
    ?R p:P348 [
      ps:P348 ?softwareVersion;
      pq:P577 ?publicationDate
    ] .
}'
query_wikidata(sparql_query)

## Not run: 
# "smart" format converts all datetime columns to POSIXct
query_wikidata(sparql_query, format = "smart")

## End(Not run)

Convert an input to a item QID

Description

Convert an input string to the most likely item QID

Usage

searcher(search_term, language, limit, type, ...)

Arguments

Value

If the inputted string matches an item label, return its QID. If the inputted string matches multiple labels of multiple items, return the QID of the first hit. If the inputted string is already a QID, return the string.

Examples

# if input string is a valid QID
as_qid("Q42")
# if input string matches multiple item labels
as_qid("Douglas Adams")
# if input string matches a single unique label
as_qid("Douglas Adams and the question of arterial blood pressure in mammals")

Download full Wikidata items matching a sparql query

Description

Utility wrapper for wikidata spargle endpoint to download items. Used by get_geo_entity and get_geo_box

Usage

sparql_query(query, ...)

Arguments

Value

a download of the full wikidata objects formatted as a nested json list

Remove special characters

Description

Special characters can otherwise mess up wikidata read-writes

Usage

unspecial(x)

Arguments

Value

the inputted strings with special characters replaced with closest match plan characters.

Extract an identifier from a wikidata URL

Description

Convert a URL ending in an identifier (returned by SPARQL queries) to just the plain identifier (QID or PID).

Convert a URL ending in an identifier (returned by SPARQL queries) to just the plan identifier (QID or PID).

Usage

url_to_id(x)

url_to_id(x)

Arguments

Value

if the URL ends in a QID or PID, return that PID or QID, else return the original string

QID or PID

Examples

url_to_id("http://www.wikidata.org/entity/42")
url_to_id("http://www.wikidata.org/Q42")

Download a Wikidata item

Description

Utility wrapper for wikidata API to download item. Used by get_item and get_property

Usage

wd_query(title, ...)

Arguments

Value

a download of the full wikidata object (item or property) formatted as a nested json list

Download random Wikidata items

Description

Utility wrapper for wikidata API to download random items. Used by random_item

Usage

wd_rand_query(ns, limit, ...)

Arguments

Value

a download of the full wikidata objects (items or properties) formatted as nested json lists

Write statements to Wikidata

Description

Upload data to wikidata, including creating items, adding statements to existing items (via the quickstatements format and API).

Usage

write_wikidata(
  items,
  properties = NULL,
  values = NULL,
  qual.properties = NULL,
  qual.values = NULL,
  src.properties = NULL,
  src.values = NULL,
  remove = FALSE,
  format = "tibble",
  api.username = NULL,
  api.token = NULL,
  api.format = "v1",
  api.batchname = NULL,
  api.submit = TRUE
)

Arguments

Value

data formatted to upload to wikidata (via quickstatemsnts), optionally also directly uploded to wikidata (see format parameter).

Examples

# Add a statement to the "Wikidata sandbox" item (Q4115189)
# to say that it is an "instance of" (P31) of Q1 (the universe).
# The instruction will submit directly to wikidata via the API
# (if you include your wikimedia username and token)

write_wikidata(items        = "Wikidata Sandbox",
               properties   = "instance of",
               values       = "Q1",
               format       = "api",
               api.username = "myusername", 
               api.token    = , #REDACTED#
               )
#note: