Example: Stance detection

The annotate() function with a predefined stance() task allows you to perform stance detection on texts regarding a specific topic. This position taking analysis classifies texts as Pro, Neutral, or Contra towards the given topic, along with a brief explanation. In this example, we will analyze a set of inaugural speeches to determine their stance on “Climate Change”.

Loading packages and data

# We will use the quanteda package 
# for loading a sample corpus of innaugural speeches
# If you have not yet installed the quanteda package, you can do so by:
# install.packages("quanteda")
library(quanteda)

## Package version: 4.3.1
## Unicode version: 15.1
## ICU version: 74.2

## Parallel computing: disabled

## See https://quanteda.io for tutorials and examples.

library(quallmer)

## Loading required package: ellmer

# For educational purposes, 
# we will use a subset of the inaugural speeches corpus
# The three most recent speeches in the corpus
data_corpus_inaugural <- quanteda::data_corpus_inaugural[57:60]

Using `annotate()` for stance detection of texts

# Define topic of interest
topic <- "Climate Change"
# Apply predefined stance task with stance() in the annotate() function
result <- annotate(data_corpus_inaugural, task = stance(topic), 
                   chat_fn = chat_openai, model = "gpt-4o",
                   api_args = list(temperature = 0, seed = 42))

## Running task 'Stance detection' using model: gpt-4o

## [working] (0 + 0) -> 3 -> 1 | ■■■■■■■■■                         25%

## [working] (0 + 0) -> 0 -> 4 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  100%

id	stance	explanation
2013-Obama	Pro	The text explicitly acknowledges the threat of climate change and emphasizes the need for collective action to address it. It mentions the importance of transitioning to sustainable energy sources and leading in technology that will power new jobs and industries. The text also highlights the devastating impacts of climate change, such as fires, droughts, and storms, and frames addressing climate change as a responsibility to future generations.
2017-Trump	Neutral	The text is an inaugural speech that focuses on themes of national pride, economic revitalization, and political change. It does not specifically address climate change or environmental issues, so it cannot be classified as Pro or Contra regarding climate change.
2021-Biden	Pro	The text acknowledges climate change as a crisis, referring to it as ‘a cry for survival comes from the planet itself’ and ‘a climate in crisis.’ This indicates a recognition of the urgency and seriousness of climate change, aligning with a pro stance towards addressing it.
2025-Trump	Contra	The text expresses a stance against climate change initiatives, specifically mentioning the end of the Green New Deal and the revocation of the electric vehicle mandate. It emphasizes increasing fossil fuel production and use, such as drilling for oil and gas, which are contrary to efforts to combat climate change.

Adjusting the stance detection task

You can customize the stance detection task by defining your own task with define_task() (for a more detailed explanation, see our “Defining custom tasks” tutorial). For example, you might want to include an additional field for confidence level.

custom_stance <- define_task(
  name = "Custom stance detection",
  system_prompt = paste0(
    "You are an expert annotator. Read each short text carefully and determine its stance towards ",
    topic,
    ". Classify the stance as Pro, Neutral, or Contra, provide a brief explanation for your classification, and indicate your confidence level from 0 to 1."
  ),
  type_def = ellmer::type_object(
    stance = ellmer::type_string("Stance towards the topic: Pro, Neutral, or Contra"),
    explanation = ellmer::type_string("Brief explanation of the classification"),
    confidence = ellmer::type_number("Confidence level from 0 to 1")
  ),
  input_type = "text"
)
# Apply the custom stance task
custom_result <- annotate(data_corpus_inaugural, task = custom_stance, 
                          chat_fn = chat_openai, model = "gpt-4o",
                          api_args = list(temperature = 0, seed = 42))

## Running task 'Custom stance detection' using model: gpt-4o

## [working] (0 + 0) -> 3 -> 1 | ■■■■■■■■■                         25%

## [working] (0 + 0) -> 0 -> 4 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  100%

id	stance	explanation	confidence
2013-Obama	Pro	The text explicitly states that the country will respond to the threat of climate change, acknowledging the overwhelming judgment of science and the devastating impacts of environmental issues like fires, droughts, and storms. It emphasizes the need for sustainable energy and leadership in this transition, indicating a proactive stance towards addressing climate change.	1.00
2017-Trump	Neutral	The text is an inaugural speech that focuses on themes of national pride, economic revitalization, and political change. It does not specifically address climate change or environmental issues, so it cannot be classified as Pro or Contra regarding climate change.	0.90
2021-Biden	Pro	The text acknowledges climate change as a crisis by referring to ‘a cry for survival comes from the planet itself’ and ‘a climate in crisis.’ This indicates a recognition of climate change as a significant issue that needs to be addressed, aligning with a pro stance towards acknowledging and potentially taking action on climate change.	0.95
2025-Trump	Contra	The text expresses a stance against climate change initiatives, specifically mentioning the end of the Green New Deal and the revocation of the electric vehicle mandate. It emphasizes increased drilling and use of fossil fuels, which are contrary to efforts to combat climate change.	0.95

Or, you might want the LLM to extract specific arguments supporting the stance.

argument_stance <- define_task(
  name = "Argument-based stance detection",
  system_prompt = paste0(
    "You are an expert annotator. Read each short text carefully and determine its stance towards ",
    topic,
    ". Classify the stance as Pro, Neutral, or Contra, provide a brief explanation for your classification, and list up to three key arguments supporting the stance."
  ),
  type_def = ellmer::type_object(
    stance = ellmer::type_string("Stance towards the topic: Pro, Neutral, or Contra"),
    explanation = ellmer::type_string("Brief explanation of the classification"),
    arguments = ellmer::type_string("Key arguments supporting the stance")
  ),
  input_type = "text"
)
# Apply the argument-based stance task
argument_result <- annotate(data_corpus_inaugural, task = argument_stance, 
                            chat_fn = chat_openai, model = "gpt-4o",
                            api_args = list(temperature = 0, seed = 42))

## Running task 'Argument-based stance detection' using model: gpt-4o

## [working] (0 + 0) -> 3 -> 1 | ■■■■■■■■■                         25%

## [working] (0 + 0) -> 0 -> 4 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  100%

id	stance	explanation	arguments
2013-Obama	Pro	The text explicitly acknowledges the threat of climate change and emphasizes the need for collective action to address it. It highlights the importance of transitioning to sustainable energy sources and leading in technology that will power new jobs and industries.	The text states that failing to respond to climate change would betray future generations, indicating a moral obligation to act. It acknowledges the overwhelming judgment of science on climate change and the devastating impacts of natural disasters, reinforcing the urgency of the issue. The text emphasizes the economic benefits of leading the transition to sustainable energy, suggesting that it will maintain economic vitality and create new jobs and industries.
2017-Trump	Neutral	The text is an inaugural speech that focuses on themes of national pride, economic revitalization, and political change. It does not explicitly address climate change or environmental policies, making it neutral on the topic.	The speech emphasizes economic growth and job creation, particularly through rebuilding infrastructure and promoting American industry. 2. It focuses on national interests and protectionism, with no mention of environmental concerns or climate change. 3. The speech is centered on political and social unity, without addressing specific environmental issues.
2021-Biden	Pro	The text expresses a clear stance in favor of addressing climate change, referring to it as a ‘climate in crisis’ and emphasizing the need for bold action to tackle this and other challenges. The speaker acknowledges the urgency of the issue and includes it among the critical responsibilities facing the nation.	The text refers to a ‘cry for survival’ from the planet, indicating the urgent need to address climate change. It describes the climate as being in crisis, suggesting a recognition of the severity of the issue. The speaker calls for boldness and action to resolve the cascading crises of the era, which includes climate change.
2025-Trump	Contra	The speech explicitly opposes climate change initiatives by declaring an end to the Green New Deal and revoking the electric vehicle mandate. It emphasizes increased fossil fuel production and use, which contradicts efforts to combat climate change.	Declaration of a national energy emergency to increase fossil fuel production (‘drill, baby, drill’). Ending the Green New Deal and revoking the electric vehicle mandate. Emphasis on using the U.S.’s oil and gas reserves to become a manufacturing nation again.

In this example, we demonstrated how to use the stance() task for stance detection on texts regarding “Climate Change”. We also showed how to customize the task to include additional fields such as confidence level and key arguments supporting the stance. Now it is your turn to explore stance detection with your own texts and topics of interest!

Loading packages and data

Using annotate() for stance detection of texts

Adjusting the stance detection task

Using `annotate()` for stance detection of texts