Skip to contents

Example: Stance detection

task_stance.Rmd

The annotate() function with a predefined task_stance() task allows you to perform stance detection on texts regarding a specific topic. This position taking analysis classifies texts as Pro, Neutral, or Contra towards the given topic, along with a brief explanation. In this example, we will analyze a set of inaugural speeches to determine their stance on “Climate Change”.

Loading packages and data 

# We will use the quanteda package 
# for loading a sample corpus of innaugural speeches
# If you have not yet installed the quanteda package, you can do so by:
# install.packages("quanteda")
library(quanteda)
## Package version: 4.3.1
## Unicode version: 15.1
## ICU version: 74.2
## Parallel computing: disabled
## See https://quanteda.io for tutorials and examples.
## Loading required package: ellmer
# For educational purposes, 
# we will use a subset of the inaugural speeches corpus
# The three most recent speeches in the corpus
data_corpus_inaugural <- quanteda::data_corpus_inaugural[57:60]

Using annotate() for stance detection of texts 

# Define topic of interest
topic <- "Climate Change"
# Apply predefined stance task with task_stance() in the annotate() function
result <- annotate(data_corpus_inaugural, task = task_stance(topic),
                   model_name = "openai/gpt-4o",
                   params = list(temperature = 0))
## [working] (0 + 0) -> 3 -> 1 | ■■■■■■■■■                         25%
## [working] (0 + 0) -> 0 -> 4 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  100%
id stance explanation
2013-Obama Pro The text explicitly acknowledges the threat of climate change and emphasizes the need for collective action to address it. It highlights the importance of transitioning to sustainable energy sources and leading in technology to preserve the environment for future generations.
2017-Trump Neutral The text is an inaugural speech focused on national pride, economic revitalization, and political change. It does not mention climate change or environmental issues, so it cannot be classified as Pro or Contra regarding climate change.
2021-Biden Pro The text acknowledges climate change as a crisis by referring to “a cry for survival comes from the planet itself” and “a climate in crisis.” This indicates a recognition of climate change as a significant issue that needs to be addressed, aligning with a pro stance on climate change action.
2025-Trump Contra The text expresses a stance against climate change initiatives by declaring an end to the Green New Deal and revoking the electric vehicle mandate. It emphasizes increased drilling and fossil fuel use, which are contrary to climate change mitigation efforts.

Adjusting the stance detection task

You can customize the stance detection task by defining your own task with task() (for a more detailed explanation, see our “Defining custom tasks” tutorial). For example, you might want to include an additional field for confidence level.

custom_stance <- task(
  name = "Custom stance detection",
  system_prompt = paste0(
    "You are an expert annotator. Read each short text carefully and determine its stance towards ",
    topic,
    ". Classify the stance as Pro, Neutral, or Contra, provide a brief explanation for your classification, and indicate your confidence level from 0 to 1."
  ),
  type_def = ellmer::type_object(
    stance = ellmer::type_string("Stance towards the topic: Pro, Neutral, or Contra"),
    explanation = ellmer::type_string("Brief explanation of the classification"),
    confidence = ellmer::type_number("Confidence level from 0 to 1")
  ),
  input_type = "text"
)
# Apply the custom stance task
custom_result <- annotate(data_corpus_inaugural, task = custom_stance,
                          model_name = "openai/gpt-4o",
                          params = list(temperature = 0))
id stance explanation confidence
2013-Obama Pro The text explicitly acknowledges the threat of climate change and emphasizes the need for collective action to address it. It mentions the importance of transitioning to sustainable energy sources and leading in technology to combat climate change, indicating a proactive stance. 0.95
2017-Trump Neutral The text is an inaugural speech focusing on national pride, economic revitalization, and political change. It does not explicitly mention climate change or environmental policies, making it neutral on the topic. 0.90
2021-Biden Pro The text acknowledges climate change as a crisis, referring to it as a ‘climate in crisis’ and emphasizing the need to address it as part of the broader challenges facing the nation. This indicates a stance that recognizes the reality and urgency of climate change. 0.95
2025-Trump Contra The text explicitly mentions ending the Green New Deal and revoking the electric vehicle mandate, which are measures associated with combating climate change. This indicates a stance against climate change initiatives. 0.95

Or, you might want the LLM to extract specific arguments supporting the stance.

argument_stance <- task(
  name = "Argument-based stance detection",
  system_prompt = paste0(
    "You are an expert annotator. Read each short text carefully and determine its stance towards ",
    topic,
    ". Classify the stance as Pro, Neutral, or Contra, provide a brief explanation for your classification, and list up to three key arguments supporting the stance."
  ),
  type_def = ellmer::type_object(
    stance = ellmer::type_string("Stance towards the topic: Pro, Neutral, or Contra"),
    explanation = ellmer::type_string("Brief explanation of the classification"),
    arguments = ellmer::type_string("Key arguments supporting the stance")
  ),
  input_type = "text"
)
# Apply the argument-based stance task
argument_result <- annotate(data_corpus_inaugural, task = argument_stance,
                            model_name = "openai/gpt-4o",
                            params = list(temperature = 0))
## [working] (0 + 0) -> 3 -> 1 | ■■■■■■■■■                         25%
## [working] (0 + 0) -> 0 -> 4 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  100%
id stance explanation arguments
2013-Obama Pro The text explicitly acknowledges the threat of climate change and emphasizes the need for action to address it, aligning with a pro-climate change stance.
  1. Acknowledgment of the threat: The text states that failing to address climate change would betray future generations.
  2. Scientific consensus: It mentions the overwhelming judgment of science on climate change.
  3. Economic and environmental leadership: The text argues for leading the transition to sustainable energy to maintain economic vitality and preserve natural resources.
2017-Trump Neutral The text does not explicitly mention climate change or environmental issues. It focuses on national pride, economic growth, and political change without addressing climate-related topics.
  1. Emphasis on economic growth and job creation without reference to environmental impact.
  • Focus on national interests and infrastructure development without mentioning sustainability.
  • Absence of any direct reference to climate change or environmental policies.
    • 2021-Biden Pro The text acknowledges climate change as a critical issue by referring to it as a ‘climate in crisis’ and emphasizes the need for bold action to address it.
    1. The speech identifies climate change as one of the major challenges facing the nation, alongside other significant issues like the pandemic and systemic racism.
  • It calls for unity and boldness in addressing these challenges, implying a proactive stance on climate action.
  • The mention of ‘a cry for survival comes from the planet itself’ highlights the urgency and importance of addressing climate change.
    • 2025-Trump Contra The text expresses a stance against climate change initiatives, particularly by ending the Green New Deal and revoking the electric vehicle mandate.
    1. Declares a national energy emergency to increase drilling and use of fossil fuels.
  • Ends the Green New Deal and revokes the electric vehicle mandate.
  • Emphasizes using the country’s oil and gas reserves to boost the economy.

    • In this example, we demonstrated how to use the stance() task for stance detection on texts regarding “Climate Change”. We also showed how to customize the task to include additional fields such as confidence level and key arguments supporting the stance. Now it is your turn to explore stance detection with your own texts and topics of interest!