Skip to contents

Example: Stance detection

task_stance.Rmd

The annotate() function with a predefined task_stance() task allows you to perform stance detection on texts regarding a specific topic. This position taking analysis classifies texts as Pro, Neutral, or Contra towards the given topic, along with a brief explanation. In this example, we will analyze a set of inaugural speeches to determine their stance on “Climate Change”.

Loading packages and data 

# We will use the quanteda package 
# for loading a sample corpus of innaugural speeches
# If you have not yet installed the quanteda package, you can do so by:
# install.packages("quanteda")
library(quanteda)
## Package version: 4.3.1
## Unicode version: 14.0
## ICU version: 71.1
## Parallel computing: disabled
## See https://quanteda.io for tutorials and examples.
## Loading required package: ellmer
# For educational purposes, 
# we will use a subset of the inaugural speeches corpus
# The three most recent speeches in the corpus
data_corpus_inaugural <- quanteda::data_corpus_inaugural[57:60]

Using annotate() for stance detection of texts 

# Define topic of interest
topic <- "Climate Change"
# Apply predefined stance task with task_stance() in the annotate() function
result <- annotate(data_corpus_inaugural, task = task_stance(topic),
                   model_name = "openai/gpt-4o",
                   params = list(temperature = 0))
## [working] (0 + 0) -> 3 -> 1 | ■■■■■■■■■                         25%
## [working] (0 + 0) -> 2 -> 2 | ■■■■■■■■■■■■■■■■                  50%
## [working] (0 + 0) -> 0 -> 4 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  100%
id stance explanation
2013-Obama Pro The text explicitly acknowledges the threat of climate change and emphasizes the need for collective action to address it. It highlights the importance of transitioning to sustainable energy sources and leading in technology to combat climate change, aligning with a pro-climate action stance.
2017-Trump Neutral The text does not explicitly mention climate change or environmental issues. It focuses on national pride, economic growth, and political change without addressing climate-related topics.
2021-Biden Pro The text acknowledges climate change as a crisis, referring to it as a ‘cry for survival from the planet itself’ and a ‘climate in crisis.’ This indicates a recognition of the issue and a stance in favor of addressing it.
2025-Trump Contra The text expresses a stance against climate change initiatives by stating the intention to end the Green New Deal and revoke the electric vehicle mandate. It emphasizes increasing fossil fuel production and use, which contradicts efforts to combat climate change.

Adjusting the stance detection task

You can customize the stance detection task by defining your own task with task() (for a more detailed explanation, see our “Defining custom tasks” tutorial). For example, you might want to include an additional field for confidence level.

custom_stance <- task(
  name = "Custom stance detection",
  system_prompt = paste0(
    "You are an expert annotator. Read each short text carefully and determine its stance towards ",
    topic,
    ". Classify the stance as Pro, Neutral, or Contra, provide a brief explanation for your classification, and indicate your confidence level from 0 to 1."
  ),
  type_def = ellmer::type_object(
    stance = ellmer::type_string("Stance towards the topic: Pro, Neutral, or Contra"),
    explanation = ellmer::type_string("Brief explanation of the classification"),
    confidence = ellmer::type_number("Confidence level from 0 to 1")
  ),
  input_type = "text"
)
# Apply the custom stance task
custom_result <- annotate(data_corpus_inaugural, task = custom_stance,
                          model_name = "openai/gpt-4o",
                          params = list(temperature = 0))
## [working] (0 + 0) -> 3 -> 1 | ■■■■■■■■■                         25%
## [working] (0 + 0) -> 0 -> 4 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  100%
id stance explanation confidence
2013-Obama Pro The text explicitly acknowledges the threat of climate change and emphasizes the need for collective action to address it. It mentions the importance of transitioning to sustainable energy sources and leading in technology to combat climate change. 0.95
2017-Trump Neutral The text is a political speech focused on national pride, economic revitalization, and unity. It does not explicitly mention climate change or environmental policies, making it neutral on the topic. 0.90
2021-Biden Pro The text acknowledges climate change as a crisis, referring to it as a ‘climate in crisis’ and includes it among the significant challenges facing the nation. This indicates a recognition of climate change as a serious issue that needs to be addressed. 0.95
2025-Trump Contra The text expresses a stance against climate change initiatives by stating intentions to end the Green New Deal and revoke the electric vehicle mandate. It emphasizes increased drilling and fossil fuel use, which contradicts climate change mitigation efforts. 0.95

Or, you might want the LLM to extract specific arguments supporting the stance.

argument_stance <- task(
  name = "Argument-based stance detection",
  system_prompt = paste0(
    "You are an expert annotator. Read each short text carefully and determine its stance towards ",
    topic,
    ". Classify the stance as Pro, Neutral, or Contra, provide a brief explanation for your classification, and list up to three key arguments supporting the stance."
  ),
  type_def = ellmer::type_object(
    stance = ellmer::type_string("Stance towards the topic: Pro, Neutral, or Contra"),
    explanation = ellmer::type_string("Brief explanation of the classification"),
    arguments = ellmer::type_string("Key arguments supporting the stance")
  ),
  input_type = "text"
)
# Apply the argument-based stance task
argument_result <- annotate(data_corpus_inaugural, task = argument_stance,
                            model_name = "openai/gpt-4o",
                            params = list(temperature = 0))
## [working] (0 + 0) -> 3 -> 1 | ■■■■■■■■■                         25%
## [working] (0 + 0) -> 0 -> 4 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  100%
id stance explanation arguments
2013-Obama Pro The text explicitly acknowledges the threat of climate change and emphasizes the need for collective action to address it. It supports transitioning to sustainable energy and recognizes the scientific consensus on climate change.
  1. Acknowledges the threat of climate change and the need to respond to it to protect future generations.
  2. Emphasizes the importance of leading the transition to sustainable energy to maintain economic vitality.
  3. Recognizes the scientific consensus on climate change and the impact of environmental disasters.
2017-Trump Neutral The text is an inaugural speech focused on national issues such as economic revitalization, infrastructure, and national pride. It does not explicitly address climate change or environmental policies.
  1. Focus on economic revitalization and job creation.
  • Emphasis on national pride and unity.
  • Commitment to infrastructure development.
    • 2021-Biden Pro The text acknowledges climate change as a critical issue by referring to it as a ‘climate in crisis’ and emphasizes the need for bold action to address it.
    1. The speech identifies climate change as one of the major challenges facing the nation, alongside other significant issues like the pandemic and systemic racism.
  • It calls for boldness and action to resolve the ‘cascading crises of our era,’ which includes climate change.
  • The speaker emphasizes the responsibility to pass along a better world for future generations, implying the need for environmental stewardship.
    • 2025-Trump Contra The text expresses a stance against climate change initiatives, specifically mentioning the end of the Green New Deal and revoking the electric vehicle mandate.
    1. Declares a national energy emergency to increase drilling and use of oil and gas.
  • Ends the Green New Deal and revokes the electric vehicle mandate.
  • Emphasizes using America’s oil and gas reserves to become a manufacturing nation again.

    • In this example, we demonstrated how to use the stance() task for stance detection on texts regarding “Climate Change”. We also showed how to customize the task to include additional fields such as confidence level and key arguments supporting the stance. Now it is your turn to explore stance detection with your own texts and topics of interest!