Skip to contents

[Experimental]

Usage

qlm_translate(
  x,
  model,
  to = "en",
  from = NULL,
  instructions = NULL,
  context = NULL,
  glossary = NULL,
  protect = NULL,
  params = NULL,
  chunk_size = NULL,
  adapter = NULL,
  api_key = NULL,
  base_url = NULL,
  max_active = NULL,
  rpm = 60,
  on_error = c("continue", "return", "stop"),
  ...
)

Arguments

x

Character vector of texts, optionally named, or a quanteda corpus. A transcript from qlm_transcribe() or an earlier translation can also be translated, and its history is kept. Names must be unique and non-empty. Missing texts are recorded as failures without a request.

model

The provider and model, as "provider/model"; required. See "Providers" for the translation services; any other ellmer::chat() model translates through qlm_code().

to

Target language, as a BCP 47 language tag such as "en", "en-GB", "pt-BR" or "zh-Hant": one tag for all texts, or one for each text, as when translating translations back into their source languages. See "Language codes".

from

Source language, as a BCP 47 language tag: one tag for all texts, or one for each text, such as the ISO document variable of a multilingual corpus. NULL, or NA for a text, leaves the provider to detect that text's language. Set it whenever you know it: see "Unsupported languages".

instructions

Character vector of translation directives, such as a register to use or how to treat names.

context

One string of background about the texts, to inform the translation but not to be translated.

glossary

Named character vector of source terms and their required translations, such as c(Bundestag = "Bundestag").

protect

Character vector of regular expressions matching text to keep exactly as it is, such as names or codes. Each is matched against the whole text, so a match may span paragraphs and ^ means the start of the text. Matches must not overlap or be empty; every one is checked in the translation.

params

Generation settings from ellmer::params(), such as temperature or max_tokens.

chunk_size

The most source characters in one request, or NULL for the provider's default. Characters only estimate tokens: a response cut off at the output limit is detected and sent again in smaller parts.

adapter

NULL to choose how requests are made from model, or "generic" to send a recognised translation model the ordinary chat request instead.

api_key

An API key, or NULL to use the provider's environment variable or saved key (see "API keys"). Keys are never recorded.

base_url

Optional API base URL, recorded with any credentials in it removed.

max_active

Maximum concurrent requests, or NULL for the provider's default.

rpm

Maximum requests per minute.

on_error

Failure policy: "continue" attempts every text, "return" stops sending after a failure, and "stop" raises it.

...

Named arguments specific to the provider; see "Providers". Any other name is an error.

Value

A qlm_translation: a character vector of translations named by document, with NA for each translation that failed or was not submitted. Its "provenance" attribute is a data frame with one row per text: the source text and its hash, the output hash, requested and reported models, languages, adapter, settings, status, error, timestamp, any earlier preprocessing history, and attempts: for each text, a data frame with one row per request attempt, giving the segment, any error, the reductions made, and usage (tokens and cost, or DeepL's billed characters).

Subsetting with [, renaming, and c() keep the provenance aligned. Assigning plain text records an edit; assigning another qlm_translation, such as a retry of failed texts, carries its provenance. as.character() drops the provenance, and inputs() returns the source texts.

For a corpus input, the document variables and metadata are kept, and quanteda::corpus(x) (see corpus.qlm_translation()) returns a corpus of the translated texts with them restored.

Details

Translates texts and returns them as a named character vector carrying per-document translation provenance, as qlm_transcribe() does for transcripts. The result can be passed to qlm_code(); qlm_trail() then records translation as part of the coding instrument.

Providers

to, from and protect work the same for every provider. The other arguments depend on it; where a provider cannot honour one, using it is an error rather than being ignored.

DeepLQwen-MTGoogle Translation LLMChat models
model"deepl/" and quality_optimized, latency_optimized or prefer_quality_optimized"dashscope/" and qwen-mt-plus, qwen-mt-flash or qwen-mt-lite"google-translate/translation-llm"any ellmer::chat() model
instructionsat most 10, of at most 300 characters eachyeserroryes
contextyesyeserroryes
glossaryerror: not yet implementedyeserroryes
paramserrortemperature, top_p, top_k, seed, max_tokenserrorany
...formality: "default", "more", "less", "prefer_more" or "prefer_less"noneproject_id (default: GOOGLE_CLOUD_PROJECT); region (default "us-central1")passed to ellmer::chat(), e.g. api_args
One requestup to 50 whole documents of one language pairadjacent paragraphs of one documentone documentadjacent paragraphs of up to 50 documents of one language pair
chunk_size default16,0004,00030,000, Google's limit8,000, or 2 * params$max_tokens if smaller
max_active default22210
API keyDEEPL_API_KEY, or keyring "deepl"DASHSCOPE_API_KEYGOOGLE_TRANSLATE_API_KEY, or keyring "google-translate"the provider's, through ellmer
Also recordeddetected language, model type used, billed charactersmodel, tokensdetected language, modeltokens, cost

Chat models translate several short documents in one request, so that the instructions sent with every request are not repeated for each text. A document that fits in one request is not divided between two, and the model is told that paragraphs of different documents are unrelated. With on_error = "return" or "stop", each request contains text from one document and requests are scheduled in input order. Tokens and cost are apportioned by source characters in the recorded attempts.

DeepL model types express a quality or speed goal, not a fixed model version. Google's region selects the model's location, but the Basic API uses a global endpoint, so it does not guarantee where data is processed. For chat models the answering model version is not yet recorded (#199). Arguments passed to a chat model through ... are recorded by name only, since they can carry credentials.

Language codes

to and from take BCP 47 language tags: an ISO 639 language code ("fr"), optionally followed by an ISO 15924 script ("zh-Hant") and an ISO 3166 region or UN M.49 area ("pt-BR", "es-419"). A plain language code is usually what you want; add a region only for a regional variant of the target. W3C's Choosing a language tag explains the parts, and the Language Subtag Lookup finds the tag for a language. Case does not matter, and an underscore may be used for a hyphen.

A three-letter ISO 639-3 code, such as the ISO document variable of quanteda.corpora::data_corpus_udhr holds, is replaced by its two-letter ISO 639-1 code where there is one (fra becomes fr, deu becomes de, and cmn, Mandarin, becomes zh). Any other is passed on as it is, so that a provider that does not support the language says so instead of guessing.

Each provider spells some languages differently, so the tag is converted to the provider's own code, and the provenance records both for each text (settings$provider_languages). For Chinese, an explicit script decides between Simplified and Traditional (zh-Hans-HK is Simplified); a region only supplies the usual script when none is given (zh-TW is Traditional).

  • DeepL takes regional variants only as targets, and only en-GB, en-US, fr-CA, fr-FR, de-DE, de-CH, pt-BR, pt-PT, es-ES, es-419, zh-Hans and zh-Hant; any other is an error before any request. A source is reduced to its language. zh-CN and zh-TW become ZH-HANS and ZH-HANT, no becomes NB, and fil becomes TL.

  • Google Translation LLM: zh-CN and zh-TW become zh-Hans and zh-Hant, and tl becomes fil.

  • Qwen-MT takes language codes only, with zh_tw for Traditional Chinese; a regional variant such as en-GB is requested in its instructions instead.

  • Chat models receive the tag as given.

Unsupported languages

Language detection fails silently. Given text in a language it does not support, every provider tested, DeepL, Google, Qwen-MT and a chat model, returned a fluent but invented English text, recorded as "ok"; DeepL and Google reported a wrong detected language (Swahili for Kimbundu). With from set, DeepL, Google and Qwen-MT instead refuse an unsupported language, and the text is NA with the provider's reason. Chat models do not refuse, and may invent a translation even when told the source language.

So set from whenever the source language is known, and when it is not, check the recorded detected_language against what you expect. For a corpus in several languages, pass one tag for each text, for example from = quanteda::docvars(corpus, "ISO"); use NA for a text whose language you do not know. DeepL packs the texts of each language pair into requests of their own. Results that look fluent are not evidence of a correct translation.

API keys

Where the table names a keyring entry, a key saved once with, for example, keyring::key_set("deepl") is used when the environment variable is unset; this needs the optional keyring package and may prompt to unlock the system store. An explicit api_key overrides both.

Paragraphs, retries and failures

Paragraphs are separated by blank lines. The separators, and whitespace at the start and end of each paragraph, are kept aside and restored exactly. DeepL, Qwen-MT and chat models are told where each paragraph begins, and a response that loses, merges or reorders paragraphs is sent again in smaller parts. If a chat request containing several documents fails, it is split by document and retried unless the provider reports a configuration error such as invalid credentials. Google receives each document whole and handles paragraph formatting itself, so exact separators are not guaranteed there. Text longer than chunk_size is split at sentence ends, else at spaces, else between characters.

A response that is malformed or changes a protected term is retried once, and one cut off at the output limit is sent again in smaller parts. A document with any part that still fails is NA, never a partial translation, and its error says why. Every attempt, with its usage, is recorded in the provenance (attempts). The provenance records the settings requested, not a guarantee that the provider applied them. Save the result with saveRDS() to keep the exact translations used for coding.

Examples

if (FALSE) { # \dontrun{
# Reads DEEPL_API_KEY, or supply api_key = keyring::key_get("deepl")
translated <- qlm_translate(c(speech = "Bonjour tout le monde."),
  model = "deepl/quality_optimized", to = "en-GB")
coded <- qlm_code(translated, data_codebook_sentiment,
  model = "openai/gpt-4o-mini")
qlm_trail(coded, path = "translation_trail")

texts <- c(fr = "Bonjour tout le monde.", de = "Guten Morgen.")

# Qwen-MT: reads DASHSCOPE_API_KEY for the Singapore endpoint
translated_qwen <- qlm_translate(texts,
  model = "dashscope/qwen-mt-plus", to = "en-GB",
  context = "Greetings in everyday conversation.",
  params = ellmer::params(temperature = 0.1))
translated_qwen

# Google: reads GOOGLE_TRANSLATE_API_KEY, falling back to the optional
# keyring entry saved with keyring::key_set("google-translate")
translated_google <- qlm_translate(texts,
  model = "google-translate/translation-llm", to = "en-GB",
  project_id = "your-google-cloud-project-id")
# Alternatively, set GOOGLE_CLOUD_PROJECT and omit project_id.
attr(translated_google, "provenance")

# DeepL with a formal register
qlm_translate(texts, model = "deepl/quality_optimized", formality = "more")

# A multilingual corpus, with each text's language as ISO 639-3 codes
qlm_translate(texts, model = "deepl/quality_optimized", from = c("fra", "deu"))

# Convert to a quanteda corpus, if quanteda is installed
quanteda::corpus(translated_google)
} # }