
Translate texts
qlm_translate.RdUsage
qlm_translate(
x,
model,
to = "en",
from = NULL,
instructions = NULL,
context = NULL,
glossary = NULL,
protect = NULL,
params = NULL,
chunk_size = NULL,
adapter = NULL,
api_key = NULL,
base_url = NULL,
max_active = NULL,
rpm = 60,
on_error = c("continue", "return", "stop"),
...
)Arguments
- x
Character vector of texts, optionally named, or a quanteda corpus. A transcript from
qlm_transcribe()or an earlier translation can also be translated, and its history is kept. Names must be unique and non-empty. Missing texts are recorded as failures without a request.- model
The provider and model, as
"provider/model"; required. See "Providers" for the translation services; any otherellmer::chat()model translates throughqlm_code().- to
Target language, as a BCP 47 language tag such as
"en","en-GB","pt-BR"or"zh-Hant": one tag for all texts, or one for each text, as when translating translations back into their source languages. See "Language codes".- from
Source language, as a BCP 47 language tag: one tag for all texts, or one for each text, such as the
ISOdocument variable of a multilingual corpus.NULL, orNAfor a text, leaves the provider to detect that text's language. Set it whenever you know it: see "Unsupported languages".- instructions
Character vector of translation directives, such as a register to use or how to treat names.
- context
One string of background about the texts, to inform the translation but not to be translated.
- glossary
Named character vector of source terms and their required translations, such as
c(Bundestag = "Bundestag").- protect
Character vector of regular expressions matching text to keep exactly as it is, such as names or codes. Each is matched against the whole text, so a match may span paragraphs and
^means the start of the text. Matches must not overlap or be empty; every one is checked in the translation.- params
Generation settings from
ellmer::params(), such astemperatureormax_tokens.- chunk_size
The most source characters in one request, or
NULLfor the provider's default. Characters only estimate tokens: a response cut off at the output limit is detected and sent again in smaller parts.- adapter
NULLto choose how requests are made frommodel, or"generic"to send a recognised translation model the ordinary chat request instead.- api_key
An API key, or
NULLto use the provider's environment variable or saved key (see "API keys"). Keys are never recorded.- base_url
Optional API base URL, recorded with any credentials in it removed.
- max_active
Maximum concurrent requests, or
NULLfor the provider's default.- rpm
Maximum requests per minute.
- on_error
Failure policy:
"continue"attempts every text,"return"stops sending after a failure, and"stop"raises it.- ...
Named arguments specific to the provider; see "Providers". Any other name is an error.
Value
A qlm_translation: a character vector of translations named by
document, with NA for each translation that failed or was not
submitted. Its "provenance" attribute is a data frame with one row per
text: the source text and its hash, the output hash, requested and
reported models, languages, adapter, settings, status, error, timestamp,
any earlier preprocessing history, and attempts: for each text, a data
frame with one row per request attempt, giving the segment, any error,
the reductions made, and usage (tokens and cost, or DeepL's billed
characters).
Subsetting with [, renaming, and c() keep the provenance aligned.
Assigning plain text records an edit; assigning another
qlm_translation, such as a retry of failed texts, carries its
provenance. as.character() drops the provenance, and inputs() returns
the source texts.
For a corpus input, the document variables and metadata are kept, and
quanteda::corpus(x) (see corpus.qlm_translation()) returns a corpus of
the translated texts with them restored.
Details
Translates texts and returns them as a named character vector carrying
per-document translation provenance, as qlm_transcribe() does for
transcripts. The result can be passed to qlm_code(); qlm_trail() then
records translation as part of the coding instrument.
Providers
to, from and protect work the same for every
provider. The other arguments depend on it; where a provider cannot honour
one, using it is an error rather than being ignored.
| DeepL | Qwen-MT | Google Translation LLM | Chat models | |
model | "deepl/" and quality_optimized, latency_optimized or prefer_quality_optimized | "dashscope/" and qwen-mt-plus, qwen-mt-flash or qwen-mt-lite | "google-translate/translation-llm" | any ellmer::chat() model |
instructions | at most 10, of at most 300 characters each | yes | error | yes |
context | yes | yes | error | yes |
glossary | error: not yet implemented | yes | error | yes |
params | error | temperature, top_p, top_k, seed, max_tokens | error | any |
... | formality: "default", "more", "less", "prefer_more" or "prefer_less" | none | project_id (default: GOOGLE_CLOUD_PROJECT); region (default "us-central1") | passed to ellmer::chat(), e.g. api_args |
| One request | up to 50 whole documents of one language pair | adjacent paragraphs of one document | one document | adjacent paragraphs of up to 50 documents of one language pair |
chunk_size default | 16,000 | 4,000 | 30,000, Google's limit | 8,000, or 2 * params$max_tokens if smaller |
max_active default | 2 | 2 | 2 | 10 |
| API key | DEEPL_API_KEY, or keyring "deepl" | DASHSCOPE_API_KEY | GOOGLE_TRANSLATE_API_KEY, or keyring "google-translate" | the provider's, through ellmer |
| Also recorded | detected language, model type used, billed characters | model, tokens | detected language, model | tokens, cost |
Chat models translate several short documents in one request, so that the
instructions sent with every request are not repeated for each text. A
document that fits in one request is not divided between two, and the
model is told that paragraphs of different documents are unrelated.
With on_error = "return" or "stop", each request contains text from
one document and requests are scheduled in input order. Tokens and
cost are apportioned by source characters in the recorded attempts.
DeepL model types express a quality or speed goal, not a fixed model
version. Google's region selects the model's location, but the Basic API
uses a global endpoint, so it does not guarantee where data is processed.
For chat models the answering model version is not yet recorded
(#199). Arguments
passed to a chat model through ... are recorded by name only, since they
can carry credentials.
Language codes
to and from take BCP 47 language tags: an ISO 639
language code ("fr"), optionally followed by an ISO 15924 script
("zh-Hant") and an ISO 3166 region or UN M.49 area ("pt-BR",
"es-419"). A plain language code is usually what you want; add a region
only for a regional variant of the target. W3C's
Choosing a language tag
explains the parts, and the
Language Subtag Lookup finds the tag
for a language. Case does not matter, and an underscore may be used for a
hyphen.
A three-letter ISO 639-3 code, such as the ISO document variable of
quanteda.corpora::data_corpus_udhr holds, is replaced by its two-letter
ISO 639-1 code where there is one (fra becomes fr, deu becomes de,
and cmn, Mandarin, becomes zh). Any other is passed on as it is, so
that a provider that does not support the language says so instead of
guessing.
Each provider spells some languages differently, so the tag is converted
to the provider's own code, and the provenance records both for each text
(settings$provider_languages). For Chinese, an explicit script decides
between Simplified and Traditional (zh-Hans-HK is Simplified); a region
only supplies the usual script when none is given (zh-TW is
Traditional).
DeepL takes regional variants only as targets, and only
en-GB,en-US,fr-CA,fr-FR,de-DE,de-CH,pt-BR,pt-PT,es-ES,es-419,zh-Hansandzh-Hant; any other is an error before any request. A source is reduced to its language.zh-CNandzh-TWbecomeZH-HANSandZH-HANT,nobecomesNB, andfilbecomesTL.Google Translation LLM:
zh-CNandzh-TWbecomezh-Hansandzh-Hant, andtlbecomesfil.Qwen-MT takes language codes only, with
zh_twfor Traditional Chinese; a regional variant such asen-GBis requested in its instructions instead.Chat models receive the tag as given.
Unsupported languages
Language detection fails silently. Given text in a language it does not
support, every provider tested, DeepL, Google, Qwen-MT and a chat model,
returned a fluent but invented English text, recorded as "ok"; DeepL and
Google reported a wrong detected language (Swahili for Kimbundu). With
from set, DeepL, Google and Qwen-MT instead refuse an unsupported
language, and the text is NA with the provider's reason. Chat models do
not refuse, and may invent a translation even when told the source
language.
So set from whenever the source language is known, and when it is
not, check the recorded detected_language against what you expect. For a
corpus in several languages, pass one tag for each text, for example
from = quanteda::docvars(corpus, "ISO"); use NA for a text whose
language you do not know. DeepL packs the texts of each language pair into
requests of their own. Results that look fluent are not evidence of a
correct translation.
API keys
DeepL: see access and authentication. A key ending in
:fxuses the Free API host.Qwen-MT: see Model Studio API keys. The key must match the endpoint's region.
Google: see Cloud Translation setup. Enable billing and the Cloud Translation API for the project, and create an API key allowed to call it.
Chat models: see the ellmer provider documentation.
Where the table names a keyring entry, a key saved once with, for example,
keyring::key_set("deepl") is used when the environment variable is unset;
this needs the optional keyring package and may prompt to unlock the
system store. An explicit api_key overrides both.
Paragraphs, retries and failures
Paragraphs are separated by blank lines. The separators, and whitespace at
the start and end of each paragraph, are kept aside and restored exactly.
DeepL, Qwen-MT and chat models are told where each paragraph begins, and a
response that loses, merges or reorders paragraphs is sent again in
smaller parts. If a chat request containing several documents fails,
it is split by document and retried unless the provider reports a
configuration error such as invalid credentials. Google receives each
document whole and handles paragraph formatting itself, so exact
separators are not guaranteed there. Text longer than chunk_size is
split at sentence ends, else at spaces, else between characters.
A response that is malformed or changes a protected term is retried once,
and one cut off at the output limit is sent again in smaller parts. A
document with any part that still fails is NA, never a partial
translation, and its error says why. Every attempt, with its usage, is
recorded in the provenance (attempts). The provenance records the
settings requested, not a guarantee that the provider applied them. Save
the result with saveRDS() to keep the exact translations used for
coding.
Examples
if (FALSE) { # \dontrun{
# Reads DEEPL_API_KEY, or supply api_key = keyring::key_get("deepl")
translated <- qlm_translate(c(speech = "Bonjour tout le monde."),
model = "deepl/quality_optimized", to = "en-GB")
coded <- qlm_code(translated, data_codebook_sentiment,
model = "openai/gpt-4o-mini")
qlm_trail(coded, path = "translation_trail")
texts <- c(fr = "Bonjour tout le monde.", de = "Guten Morgen.")
# Qwen-MT: reads DASHSCOPE_API_KEY for the Singapore endpoint
translated_qwen <- qlm_translate(texts,
model = "dashscope/qwen-mt-plus", to = "en-GB",
context = "Greetings in everyday conversation.",
params = ellmer::params(temperature = 0.1))
translated_qwen
# Google: reads GOOGLE_TRANSLATE_API_KEY, falling back to the optional
# keyring entry saved with keyring::key_set("google-translate")
translated_google <- qlm_translate(texts,
model = "google-translate/translation-llm", to = "en-GB",
project_id = "your-google-cloud-project-id")
# Alternatively, set GOOGLE_CLOUD_PROJECT and omit project_id.
attr(translated_google, "provenance")
# DeepL with a formal register
qlm_translate(texts, model = "deepl/quality_optimized", formality = "more")
# A multilingual corpus, with each text's language as ISO 639-3 codes
qlm_translate(texts, model = "deepl/quality_optimized", from = c("fra", "deu"))
# Convert to a quanteda corpus, if quanteda is installed
quanteda::corpus(translated_google)
} # }