Example: Ideology detection

The annotate() function with a predefined task_ideology() allows you to perform ideological scaling (0-10) on texts regarding a specified ideological dimension. In this example, we will demonstrate how to use the task_ideology() for ideology detection on a sample corpus of innaugural speeches from U.S. presidents. We will use the dimension “inclusive–exclusive” as an example. To refine the task, we will also provide a definition of the dimension (optional).

Loading packages and data

# We will use the quanteda package 
# for loading a sample corpus of innaugural speeches
# If you have not yet installed the quanteda package, you can do so by:
# install.packages("quanteda")
library(quanteda)

## Package version: 4.3.1
## Unicode version: 14.0
## ICU version: 71.1

## Parallel computing: disabled

## See https://quanteda.io for tutorials and examples.

library(quallmer)

## Loading required package: ellmer

# For educational purposes, 
# we will use a subset of the inaugural speeches corpus
# The three most recent speeches in the corpus
data_corpus_inaugural <- quanteda::data_corpus_inaugural[57:60]

Using `annotate()` for ideological scaling of texts

# Define ideological dimension
dimension <- "inclusive–exclusive"
# Provide definition for the dimension
definition <- "Inclusive language emphasizes equal rights, diversity, pluralism, 
and protection of minorities, whereas exclusive language emphasizes exclusion 
of groups, national homogeneity, and restricting rights."
# Apply predefined ideology task with task_ideology() in the annotate() function
result <- annotate(data_corpus_inaugural, task = task_ideology(dimension, definition),
                   model_name = "openai/gpt-4o",
                   params = list(temperature = 0))

## [working] (0 + 0) -> 3 -> 1 | ■■■■■■■■■                         25%

## [working] (0 + 0) -> 2 -> 2 | ■■■■■■■■■■■■■■■■                  50%

## [working] (0 + 0) -> 0 -> 4 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  100%

id	score	explanation
2013-Obama	1	The text emphasizes inclusivity, highlighting themes of equality, diversity, and collective action. It references historical struggles for civil rights and equality, and calls for unity across differences. Phrases like ‘all men are created equal,’ ‘diversity and openness,’ and ‘our journey is not complete until our gay brothers and sisters are treated like anyone else under the law’ strongly support an inclusive ideology.
2017-Trump	7	The text emphasizes national homogeneity and prioritizes American interests with phrases like “America first” and “buy American and hire American.” It highlights protectionism and a focus on American workers and borders, suggesting an exclusive stance. However, it also includes some inclusive elements, such as unity and equality among Americans, regardless of race, which slightly moderates the exclusivity.
2021-Biden	0	The text emphasizes unity, inclusion, and addressing systemic issues like racial justice and political extremism. It calls for seeing each other as neighbors, treating each other with dignity, and uniting against common challenges. These elements strongly align with inclusive language.
2025-Trump	8	The text emphasizes national sovereignty, exclusion of ‘criminal aliens,’ and a focus on American exceptionalism, which aligns with exclusive language. Phrases like ‘put America first,’ ‘reclaim our sovereignty,’ and ‘halt illegal entry’ suggest a restrictive stance on immigration and national identity. The mention of ‘only two genders’ and ending ‘socially engineered race and gender’ policies further indicates exclusivity.

Adjusting the ideology scaling task

You can customize the ideological scaling task by defining your own task with task() (for a more detailed explanation, see our “Defining custom tasks” tutorial). For example, you might like to change the scale from 0-10 to -5 to +5.

custom_ideology <- task(
    name = "Ideological scaling",
    system_prompt = paste0(
      "You are an expert political scientist performing ideological text scaling.",
      "Task:",
      "- Read each short text carefully.",
      "- Place the text on a -5 to +5 scale for the following ideological dimension: ",
      dimension, 
      definition
    ),
    type_def = ellmer::type_object(
      score       = ellmer::type_integer("Ideological position on the specified dimension (0–10, where -5 = first pole, +5 = second pole)"),
      explanation = ellmer::type_string("Brief justification for the assigned score, referring to specific elements in the text")
    ),
    input_type = "text"
  )
# Apply the custom task
custom_result <- annotate(data_corpus_inaugural, task = custom_ideology,
                          model_name = "openai/gpt-4o",
                          params = list(temperature = 0))

## [working] (0 + 0) -> 3 -> 1 | ■■■■■■■■■                         25%

## [working] (0 + 0) -> 0 -> 4 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  100%

id	score	explanation
2013-Obama	5	The text strongly emphasizes inclusivity, highlighting themes of equality, diversity, and collective action. It references historical struggles for civil rights and equality, and advocates for the protection and empowerment of marginalized groups, such as immigrants, women, and the LGBTQ+ community. The language consistently supports pluralism and the idea that America’s strength lies in its diversity and commitment to equal rights for all citizens.
2017-Trump	-2	The text emphasizes national homogeneity and prioritizes American interests with phrases like “America first” and “protect our borders.” It also highlights exclusionary economic policies such as “buy American and hire American.” While there are inclusive elements, such as unity and shared national identity, the overall tone leans towards exclusivity.
2021-Biden	5	The text emphasizes unity, inclusion, and the protection of democracy, highlighting the importance of diversity, racial justice, and overcoming division. It calls for collective action and respect for all Americans, regardless of differences, and explicitly rejects exclusionary ideologies like white supremacy. The language is inclusive, focusing on healing, unity, and equal rights.
2025-Trump	-3	The speech emphasizes national sovereignty, protection of borders, and exclusionary policies such as halting illegal entry and returning ‘criminal aliens.’ It also includes language about national pride and reclaiming American greatness, which aligns with exclusive rhetoric. However, there are mentions of unity and support for diverse communities, which slightly moderates the exclusivity.

In this example, we demonstrated how to use the task_ideology() for scaling texts regarding their ideological position on a specified dimension. We also showed how to customize the task using the task() function for more tailored annotation needs, e.g., changing the scale from 0-10 to -5 to +5. Now you can apply these techniques to your own text data for ideological analysis!

Loading packages and data

Using annotate() for ideological scaling of texts

Adjusting the ideology scaling task

Using `annotate()` for ideological scaling of texts