Example: Sentiment analysis

The annotate() function with a predefined sentiment() task allows you to rate the sentiment of texts using an LLM. The predefined sentiment() object structures the response with a numeric sentiment score from -1 (very negative) to 1 (very positive) and a brief explanation.

Loading packages and data

library(quallmer)

## Loading required package: ellmer

#Example texts
texts <- c(
"This is wonderful!",
"I really dislike this approach.",
"The results are somewhat disappointing.",
"Absolutely fantastic work!"
)

Using annotate() for predefined sentiment analysis of texts

# Apply predefined sentiment task with sentiment() in the annotate() function
result <- annotate(texts, task = sentiment(), 
                   chat_fn = chat_openai, model = "gpt-4o",
                   api_args = list(temperature = 0, seed = 42))

## Running task 'Sentiment analysis' using model: gpt-4o

## [working] (0 + 0) -> 3 -> 1 | ■■■■■■■■■                         25%

## [working] (0 + 0) -> 0 -> 4 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  100%

id	score	explanation
1	1.0	The word ‘wonderful’ conveys a very positive sentiment, indicating delight or admiration.
2	-0.8	The word ‘dislike’ indicates a strong negative sentiment towards the approach.
3	-0.5	The word ‘disappointing’ indicates a negative sentiment, but ‘somewhat’ softens the negativity, suggesting a moderate level of disappointment.
4	1.0	The phrase ‘Absolutely fantastic work!’ is highly positive, using strong positive adjectives like ‘fantastic’ and ‘absolutely’ to express admiration and satisfaction.

Adjusting the sentiment task

You can customize the sentiment analysis task by defining your own task with define_task() (for a more detailed explanation, see our “Defining custom tasks” tutorial).

For example, you might want to include an additional field for confidence level.

custom_sentiment <- define_task(
  name = "Custom sentiment analysis",
  system_prompt = "You are an expert annotator. Rate the sentiment of each text from -1 (very negative) to 1 (very positive), briefly explain why, and provide a confidence level from 0 to 1.",
  type_def = ellmer::type_object(
    score = ellmer::type_number("Sentiment score between -1 (very negative) and 1 (very positive)"),
    explanation = ellmer::type_string("Brief explanation of the rating"),
    confidence = ellmer::type_number("Confidence level from 0 to 1")
  ),
  input_type = "text"
)
# Apply the custom sentiment task
custom_result <- annotate(texts, task = custom_sentiment, 
                          chat_fn = chat_openai, model = "gpt-4o",
                          api_args = list(temperature = 0, seed = 42))

## Running task 'Custom sentiment analysis' using model: gpt-4o

id	score	explanation	confidence
1	1.0	The word ‘wonderful’ is strongly positive, indicating a very positive sentiment.	1.0
2	-0.8	The word ‘dislike’ indicates a strong negative sentiment towards the approach.	0.9
3	-0.5	The word ‘disappointing’ indicates a negative sentiment, but the use of ‘somewhat’ suggests that the disappointment is not very strong.	0.9
4	1.0	The phrase ‘Absolutely fantastic work!’ is highly positive, expressing strong approval and admiration.	1.0

Or, you might want to change the scoring scale to a 5-point Likert scale.

likert_sentiment <- define_task(
  name = "Likert scale sentiment analysis",
  system_prompt = "You are an expert annotator. Rate the sentiment of each text on a scale from 1 (very negative) to 5 (very positive) and briefly explain why.",
  type_def = ellmer::type_object(
    score = ellmer::type_number("Sentiment score between 1 (very negative) and 5 (very positive)"),
    explanation = ellmer::type_string("Brief explanation of the rating")
  ),
  input_type = "text"
)
# Apply the Likert scale sentiment task
likert_result <- annotate(texts, task = likert_sentiment, 
                          chat_fn = chat_openai, model = "gpt-4o",
                          api_args = list(temperature = 0, seed = 42))

## Running task 'Likert scale sentiment analysis' using model: gpt-4o

## [working] (0 + 0) -> 3 -> 1 | ■■■■■■■■■                         25%

## [working] (0 + 0) -> 0 -> 4 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  100%

id	score	explanation
1	5	The word ‘wonderful’ conveys a strong positive sentiment, indicating that the speaker is very pleased or impressed.
2	2	The word ‘dislike’ indicates a negative sentiment, but it is not extremely negative, hence a score of 2.
3	2	The word ‘disappointing’ indicates a negative sentiment, but the use of ‘somewhat’ suggests that the disappointment is not very strong, leading to a slightly negative overall sentiment.
4	5	The phrase ‘Absolutely fantastic work!’ is highly positive, using strong positive adjectives like ‘fantastic’ and the intensifier ‘absolutely’ to express a high level of satisfaction and praise.

In this way, you can easily adapt the sentiment analysis task to fit your specific research needs!