Example: Salience of topics

The annotate() function with a predefined task_salience() can be used to identify and rank the salience of topics discussed in texts. In this example, we will demonstrate how to apply this task to a sample corpus of innaugural speeches from US presidents.

Loading packages and data

# We will use the quanteda package 
# for loading a sample corpus of innaugural speeches
# If you have not yet installed the quanteda package, you can do so by:
# install.packages("quanteda")
library(quanteda)

## Package version: 4.3.1
## Unicode version: 15.1
## ICU version: 74.2

## Parallel computing: disabled

## See https://quanteda.io for tutorials and examples.

library(quallmer)

## Loading required package: ellmer

# For educational purposes, 
# we will use a subset of the inaugural speeches corpus
# The three most recent speeches in the corpus
data_corpus_inaugural <- quanteda::data_corpus_inaugural[57:60]

Using annotate() for salience of ANY topics discussed in texts

# Apply predefined salience task with task_salience() in the annotate() function
result <- annotate(data_corpus_inaugural, task = task_salience(),
                   model_name = "openai/gpt-4o",
                   params = list(temperature = 0))

## [working] (0 + 0) -> 3 -> 1 | ■■■■■■■■■                         25%

## [working] (0 + 0) -> 1 -> 3 | ■■■■■■■■■■■■■■■■■■■■■■■           75%

## [working] (0 + 0) -> 0 -> 4 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  100%

id	topics	explanation
2013-Obama	American Values and Ideals , Equality and Civil Rights , Collective Action and Unity , Economic Prosperity and Middle Class, Role of Government and Policy Reform	American Values and Ideals: The speech frequently references foundational American principles such as liberty, democracy, and the Constitution, emphasizing their enduring importance. Equality and Civil Rights: There is a strong focus on equality, referencing historical movements and the ongoing journey towards equal rights for all citizens, including women and LGBTQ+ individuals. Collective Action and Unity: The text emphasizes the need for unity and collective action to address modern challenges, highlighting the importance of working together as a nation. Economic Prosperity and Middle Class: The speech discusses economic recovery, the importance of a strong middle class, and the need for economic policies that support broad prosperity. Role of Government and Policy Reform: There is discussion on the need for government reform, including tax code changes and education, to meet contemporary needs while staying true to founding principles.
2017-Trump	Transfer of Power to the People , America First Policy , National Unity and Patriotism , Economic Rebuilding and Job Creation, Critique of Political Establishment	Transfer of Power to the People: The speech emphasizes returning power from Washington to the citizens, highlighting this as a central theme. America First Policy: The repeated focus on prioritizing American interests in trade, immigration, and foreign policy underscores its importance. National Unity and Patriotism: The call for unity and patriotism is a recurring theme, with references to shared values and protection. Economic Rebuilding and Job Creation: There is significant emphasis on rebuilding infrastructure and creating jobs, reflecting its salience. Critique of Political Establishment: The speech critiques the political establishment for failing the people, making it a notable topic.
2021-Biden	Unity , Democracy , Challenges facing America , American values and ideals , Historical context and legacy	Unity: The speech emphasizes unity as a central theme, repeatedly calling for Americans to come together to overcome challenges. The word ‘unity’ is mentioned multiple times, highlighting its importance. Democracy: Democracy is a key focus, with references to its fragility, the recent test it faced, and its ultimate triumph. The speech celebrates the peaceful transfer of power as a democratic victory. Challenges facing America: The text outlines several pressing issues, including the pandemic, racial justice, political extremism, and climate change, emphasizing the need to address these challenges collectively. American values and ideals: The speech frequently references core American values such as liberty, dignity, and truth, framing them as guiding principles for the nation’s future. Historical context and legacy: The speech situates current events within a broader historical narrative, referencing past struggles and achievements to inspire hope and action for the future.
2025-Trump	American Renewal and Greatness , Government Reform and Efficiency , National Security and Immigration, Economic Policies and Energy , Unity and National Pride	American Renewal and Greatness: The speech emphasizes a new era for America, focusing on national success, prosperity, and reclaiming America’s position globally. Phrases like “golden age of America” and “make America great again” highlight this theme. Government Reform and Efficiency: There is significant focus on reforming government structures, ending corruption, and restoring trust. The mention of a new Department of Government Efficiency and stopping government censorship underscores this. National Security and Immigration: The speech discusses securing borders, ending illegal immigration, and designating cartels as terrorist organizations. This is a major focus, with specific policies like the Remain in Mexico policy and military involvement at the border. Economic Policies and Energy: Economic revitalization through energy independence, ending the Green New Deal, and revamping trade policies are highlighted. The speech mentions drilling for oil and establishing the External Revenue Service. Unity and National Pride: The speech frequently references national unity, pride, and the American spirit. It mentions diverse support across demographics and aims to unify the nation under common goals.

Using annotate() for salience of a SPECIFIED LIST of topics discussed in texts

# Define a list of topics to focus on
topics <- c("economy", "health", "education", "environment", "foreign policy")
# Apply predefined salience task with task_salience() in the annotate() function
result <- annotate(data_corpus_inaugural, task = task_salience(topics),
                   model_name = "openai/gpt-4o",
                   params = list(temperature = 0))

## [working] (0 + 0) -> 3 -> 1 | ■■■■■■■■■                         25%

## [working] (0 + 0) -> 0 -> 4 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  100%

id	topics	explanation
2013-Obama	economy , environment , foreign policy, education , health	The text prominently discusses the economy, emphasizing economic recovery, the importance of a rising middle class, and the need for infrastructure and fair markets. The environment is also a key topic, with a focus on climate change and sustainable energy. Foreign policy is addressed through discussions of alliances, peace, and global engagement. Education is mentioned in the context of training workers and reforming schools. Health is touched upon with references to healthcare costs and social security programs.
2017-Trump	economy , foreign policy, education	The text primarily focuses on the economy, discussing jobs, factories, and infrastructure, which are mentioned frequently and with significant emphasis. Foreign policy is also prominent, with references to trade, alliances, and military protection. Education is mentioned in the context of improving schools and addressing issues in the education system. These topics are highlighted through repeated references and detailed discussion.
2021-Biden	health , foreign policy, environment , economy , education	Health: The text frequently mentions the pandemic, the virus, and the impact on lives, making health the most salient topic. Foreign Policy: References to America’s role in the world, repairing alliances, and engaging globally highlight foreign policy. Environment: The mention of a ‘climate in crisis’ indicates a focus on environmental issues. Economy: Discussion of job losses, businesses closing, and rebuilding the middle class points to economic concerns. Education: Briefly mentioned in the context of teaching children in safe schools.
2025-Trump	foreign policy, economy , health , environment , education	Foreign Policy: The text frequently discusses international relations, border security, and actions against foreign entities, emphasizing the importance of America’s global standing and security. Economy: Economic issues are highlighted through mentions of inflation, energy policies, manufacturing, and trade reforms, indicating a focus on economic revitalization. Health: There are references to public health systems and a commitment to ending disease epidemics, showing a concern for health-related issues. Environment: Environmental policies are mentioned, particularly in the context of energy production and the rejection of the Green New Deal. Education: The text briefly touches on education, criticizing current systems and suggesting changes, but it is less emphasized compared to other topics.

Adjusting the task_salience() so it also returns the stance for each topic

# Customizing the task to include the stance for each topic
custom_task <- task(
  name = "Salience and stance of topics",
  system_prompt = paste(
    "You are an expert analysing the content of texts.",
    "",
    "Task:",
    "- Read the text carefully.",
    "- Identify and rank the salience of the following topics: economy, health, education, environment, foreign policy.",
    "- For each topic mentioned, assign a stance as one of the following:",
    "  pro, neutral, or contra.",
    "- Append the stance directly after each topic name in the form 'topic: stance'.",
    "- Return all topic:stance entries in descending order of salience.",
    "- Separate entries with commas when presenting them in a list.",
    "",
    "Do not infer information that is not in the text.",
    "Base all evaluations solely on the language and arguments in the document.",
    "",
    "Output:",
    "- `topic_stance`: a ranked list of topic labels with stance labels appended (e.g., 'economy: pro', 'health: contra').",
    "- `explanation`: a brief justification explaining why the topics were ordered and how stance was determined.",
    sep = "\n"
  ),
  type_def = ellmer::type_object(
    topic_stance = ellmer::type_array(
      ellmer::type_string("Topic and stance label combined (e.g., 'economy: pro'), ranked by salience.")
    ),
    explanation = ellmer::type_string(
      "Brief justification for the salience ordering and stance classification."
    )
  ),
  input_type = "text"
)

# Apply the customized task in the annotate() function
custom_result <- annotate(data_corpus_inaugural, task = custom_task,
                   model_name = "openai/gpt-4o",
                   params = list(temperature = 0))

## [working] (0 + 0) -> 3 -> 1 | ■■■■■■■■■                         25%

## [working] (0 + 0) -> 0 -> 4 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  100%

id	topic_stance	explanation
2013-Obama	economy: pro , environment: pro , foreign policy: pro, health: pro , education: pro	The text emphasizes the importance of a strong economy, highlighting the need for infrastructure, fair competition, and a thriving middle class, indicating a pro stance on the economy. The environment is addressed with a commitment to tackling climate change and leading in sustainable energy, showing a pro stance. Foreign policy is discussed with a focus on peace, alliances, and supporting democracy, indicating a pro stance. Health is mentioned in the context of reducing healthcare costs and supporting programs like Medicare, showing a pro stance. Education is highlighted as essential for future success, with a call for reform and investment, indicating a pro stance.
2017-Trump	economy: pro , foreign policy: pro , education: contra , environment: neutral, health: neutral	The text emphasizes economic revitalization, focusing on job creation, infrastructure, and protectionist policies, indicating a pro stance on the economy. Foreign policy is addressed with a focus on ‘America first’ and strengthening military and alliances, showing a pro stance. Education is mentioned negatively, describing it as failing despite funding, indicating a contra stance. The environment and health are not directly addressed, leading to a neutral stance for both.
2021-Biden	democracy: pro , health: pro , unity: pro , foreign policy: pro, economy: pro , environment: pro	The text primarily focuses on the theme of democracy, emphasizing its triumph and fragility, making ‘democracy: pro’ the most salient topic. Health is addressed through the discussion of the pandemic and its impact, leading to ‘health: pro’. Unity is a central theme, repeatedly emphasized as essential for overcoming challenges, resulting in ‘unity: pro’. Foreign policy is mentioned in terms of repairing alliances and engaging with the world, hence ‘foreign policy: pro’. Economic issues are highlighted through references to job losses and rebuilding the middle class, leading to ‘economy: pro’. The environment is mentioned in the context of a ‘cry for survival from the planet’, resulting in ‘environment: pro’. Each stance is determined by the positive and proactive language used to address these issues.
2025-Trump	foreign policy: pro, economy: pro , environment: contra, health: contra , education: contra	The speech emphasizes foreign policy with a strong stance on border security, military strength, and international respect, making it the most salient topic. The economy is also prominent, with plans to reduce inflation, increase manufacturing, and utilize energy resources, indicating a pro stance. Environmental policies are mentioned negatively, with a focus on ending the Green New Deal, showing a contra stance. Health is addressed in terms of public health system failures and COVID vaccine mandates, suggesting a contra stance. Education is criticized for teaching negative views of the country, indicating a contra stance.

In this example, we demonstrated how to use the task_salience() for identifying and ranking topics discussed in texts, both with and without a predefined list of topics. Additionally, we showed how to customize the task to include stance classification for each topic. This showcases the flexibility of the annotate() function and the task framework in quallmer for various text analysis tasks.