Sentiment analysis
Worked example. Last reviewed: 2026-07-07.
This page applies the four steps from Before You Start to sentiment analysis: deciding whether a text is positive, negative, or neutral. It uses AfriSenti and NaijaSenti as the examples because they are the largest community-built sentiment datasets for African languages, annotated by native speakers. If your language is not mentioned here, go back to Step 1 and search.
Step 1: What a search turns up
Search Hugging Face, Lanfrica, or the ACL Anthology for sentiment in almost any major African language and you will quickly reach two datasets:
- AfriSenti (Muhammad et al., 2023): more than 110,000 tweets in 14 languages. Amharic, Algerian Arabic, Hausa, Igbo, Kinyarwanda, Moroccan Arabic, Mozambican Portuguese, Nigerian Pidgin, Oromo, Swahili, Tigrinya, Twi, Xitsonga, and Yoruba. It was also used in the AfriSenti-SemEval 2023 shared task, so many published systems report results on it.
- NaijaSenti (Muhammad et al., 2022): the earlier Nigerian dataset that AfriSenti grew from. Hausa, Igbo, Nigerian Pidgin, and Yoruba, with around 30,000 tweets per language (about 14,000 for Nigerian Pidgin). Its paper also released sentiment lexicons.
AfriSenti is on the Hugging Face Hub, and the GitHub repository has the data, the evaluation script, and individual annotator labels for several languages. NaijaSenti is on the Hub as HausaNLP/NaijaSenti-Twitter.
You will also find copies of the same data uploaded by other accounts, and small sentiment datasets built by translating English data or labelling tweets automatically with emojis or keywords. Copies are tempting because they load easily. Translated and auto-labelled data is tempting because it is cheap. Step 2 is where you check the licence of the copy you actually download, and whether auto-generated labels reflect how people in your language express feeling.
Step 2: Judging AfriSenti against the checklist
- Licence. Check the copy you use. The main Hugging Face card says CC BY 4.0, but other copies on the Hub are tagged CC BY-NC-SA (non-commercial, share-alike). If you are building a product, confirm the licence with the authors first.
- Access. Openly downloadable from Hugging Face and GitHub.
- Fit. Twitter/X text only. It is short, informal, and full of abbreviations and hashtags. Customer service transcripts, health surveys, formal writing, or long comments look very different. Always build an evaluation set from your own target domain.
- Culture. What reads as negative in one culture can read as neutral or even polite in another. Indirect criticism is common in many African-language contexts and is often labelled neutral by annotators from outside the culture. Read the paper's annotation section before writing your own guidelines.
- Code-switching. Normal, not an edge case. Tweets in every covered language mix in English, French, Arabic, or a regional lingua franca. Your data will too.
- Script. Amharic and Tigrinya are in Ge'ez script. Algerian and Moroccan Arabic tweets may appear in Arabic or Latin script. Check that your text processing handles both.
- Labels. Three classes: positive, negative, neutral. Neutral is the hardest class. Sarcasm, mixed feelings, indirect expression, and plain facts all end up there. Report results per class, not only an overall score (see core principles).
- Label scope. Polarity is not emotion, stance, polarization, or harm. If you need emotion labels, see emotion analysis. If you need content moderation, see hate speech analysis. If you need to detect hostility between groups, look at polarization data such as POLAR. Do not reuse AfriSenti labels for these tasks.
- Splits. Fixed train, development, and test splits for every language, so results are comparable across papers. The test sets have been public since 2023.
- Guidelines. Partial. The repository has a guidelines file for Algerian Arabic only. The paper describes the annotation process for each language.
- Quality. Annotated by native speakers. Individual annotator labels are released for several languages, so you can measure agreement yourself.
- Privacy. The tweets were anonymised: user mentions were replaced with
@userand URLs were removed. If you collect new social media data, do the same, and check the platform's terms before you redistribute it. - Transfer. Sentiment depends on culture-specific words and idioms. It transfers less well between languages than structural tasks such as NER. Measure transfer before you rely on it.
Step 3: Reuse, extend, or build
Find the row that matches your situation, then follow it across. The first question is whether your language is covered by AfriSenti or NaijaSenti.
| What you found | Your choice | What to do |
|---|---|---|
| Your language is covered by AfriSenti (14 languages) or NaijaSenti (4 Nigerian languages), and Twitter/X text is close to your domain | Reuse AfriSenti or NaijaSenti | Do not build a new corpus. Check the licence, then spend your effort on a careful human review of results on your own data. |
| Your language is covered, but your domain is different (formal writing, market speech, health, customer service) | Extend it | First build a 500–2,000 example evaluation set from your own domain. Only if AfriSenti does not work well enough, add a small 2,000–5,000 example in-domain training set, using the same three labels. |
| Your language is not covered, but it is related to a covered language | Borrow, then build small | Try cross-language transfer from the closest covered language, then build a small evaluation set in your own language. Sentiment transfers weakly, so measure before you rely on it. |
| Your language is not covered and not related to a covered language | Build from scratch | Start with the long-tail language onboarding chapter, then Data Collection, Annotation Design, and Data Quality. Read the AfriSenti paper before writing your guidelines, and start with a pilot of 2,000–5,000 examples. |
Step 4: Learning from the people who built it
AfriSenti was built by native-speaker researchers and annotators in many African countries, working in a participatory way.
Read first:
- The AfriSenti paper (Muhammad et al., 2023): the 14-language dataset, especially the section on the challenges of curating each language.
- The NaijaSenti paper (Muhammad et al., 2022): how tweets in four Nigerian languages were collected, filtered, and labelled.
- The participatory research paper (Nekoto et al., 2020): the Masakhane way of working that AfriSenti follows.
Questions to ask the authors:
- How did you decide what counts as neutral, and where did annotators disagree most?
- How did you find enough tweets in languages with little Twitter activity?
- What would you change about the guidelines if you started again?
Further reading
- Quality at a glance (Kreutzer et al., 2022): an audit of web-crawled low-resource data. Read it before you trust scraped or automatically labelled sentiment data.
Once you have your data, training a model is well covered elsewhere. The Hugging Face sequence classification tutorial is a good start.
