On 6 March 1957, Ghana became the first sub-Saharan African country to gain independence from colonial rule. Today, 69 years later, if you try to build a search engine that indexes content in Twi, Ewe, or Dagbani, you'll find that the tools to do it properly don't exist.
I know this because I tried to build one.
How I got here
Two years ago, I started building a local search engine, which was nothing fancy at first. Over time, I added full-text search, vector hybrid retrieval, auto-partitioned indexes, and cross-language binary serialisation, until I relied on it across several production systems, where it worked well.
Recently, I started turning it into something bigger, which is a distributed search engine with parallel query execution and production-grade features. Since the project is already open-source, I'm building it in the open. This is where the problems started, because when you build for a wider audience, the language support has to be exactly right.
The engine handled English, French, German, Spanish, and Portuguese cleanly. When I tried Twi, however, the pieces that the engine needed were missing.
So what broke?
A search engine uses a stemming algorithm. A stemmer takes words such as 'running', 'runs', and 'ran' and maps them all back to 'run'. Without a stemmer, the engine treats each of those words as a different word, so a search for 'run' would miss every document that contains 'running'.
Martin Porter published the first widely used stemming algorithm in 1980. Forty-six years later, his Snowball framework has stemmers for many languages, thanks to Porter and other contributors.
Languages with a Snowball stemmer
Snowball has stemmers for English, French, German, Spanish, Italian, Dutch, Russian, Portuguese, Arabic, Swedish, Danish, Norwegian, Finnish, Hungarian, Romanian, and more languages, which means that many European, Scandinavian, Slavic, and Semitic languages have a stemmer in it. Twi, Ewe, Dagbani, Yoruba, and Igbo have no stemmer in Snowball, in NLTK, or in spaCy.
The same gap exists for stopword lists. Stopwords are the common words ('the', 'is', 'and') that search engines filter out so that they can match on the words that matter. NLTK has stopword lists for 21 languages: Arabic, Azerbaijani, Danish, Dutch, English, Finnish, French, German, Greek, Hungarian, Indonesian, Italian, Kazakh, Nepali, Norwegian, Portuguese, Romanian, Russian, Spanish, Swedish, and Turkish. None of those 21 is an African language.
Researchers started the African Stopwords Project in 2023 to fix this. So far, their list has about 60 stopwords for Yoruba and none for Twi, Ewe, or Dagbani. And none of this work is in the toolkits that developers use when they build software.
The layer beneath translation is still missing
Plenty of people are working on these languages. The Masakhane community publishes datasets for 20 African languages, including Twi and Ewe. NLP Ghana and Algorine make the Khaya app for translation and speech recognition in Twi, Ewe, Ga, Yoruba, and Dagbani. In a 2024 study in the Springer journal Discover Computing, Sackitey and colleagues introduce a Twi-English parallel corpus for machine translation. Researchers at the University of Ghana are the authors of speech datasets for Akan, Ewe, Dagbani, Dagaare, and Ikposo.
That work matters; however, all of those teams are working on translation, speech recognition, and entity recognition, so the layer beneath them is still unbuilt. That layer is the stemmers, the , and the stopword lists that a search engine depends on.
Each African language needs its own stopword list
In a 2013 study, Asubiaro builds a stopword list for Yoruba with entropy-based methods and finds that only 69.1% of Yoruba stopwords overlap with English ones. The languages are too different, so each language has to have its own stopword list, which people who speak it must build from scratch.
That's my problem right now. I can get partway there, because anyone who speaks a language can curate a stopword list from their own knowledge of it, which is how I deliver partial support. Stemmers are harder, because a stopword list is only a list, whereas to write a stemmer, which is an algorithm, you need documented rules that are detailed enough for you to turn into code. Those rules are missing, although the stemmer is the piece with the biggest effect on search quality.
How these languages differ from European ones
The differences are structural, so a European stemming approach fails on these languages.
Akan has serial verb constructions, where several verbs combine into a single expression. Balmer and Grant coined the term 'serial verb' in 1929 when they wrote the first grammar of Fante Akan. Linguists have almost a century of research on this phenomenon, although no algorithm exists yet with which a search engine could break these constructions apart.
Speakers of Dagbani mark noun class with suffixes, where speakers of a Bantu language would use prefixes. From those suffixes, a listener can tell whether a word is singular or plural, which semantic category it belongs to, and whether it works as a noun or an adjective. In a 2023 paper in Topics in Linguistics, the authors show that these suffixes change the meaning of a word, so a search engine can lose that meaning when its stemmer only chops off the ending.
Yoruba is tonal, so the word 'oko' without tone marks could mean 'farm', 'husband', 'hoe', 'vehicle', or 'spear', which are five different words that look identical on screen. Most software strips the diacritical marks that distinguish them, so when a Yoruba speaker types the word for 'husband', the search engine will return results for all five.
Ewe is tonal too, although the way in which its prefixes and suffixes interact with tone is still undocumented at the depth that a programmer would need to write an algorithm.
What has to exist before software can support these languages
The first requirement is proper morphological documentation. How do suffixes chain together? How does tone change meaning? How do verbs serialise? That documentation, at the depth that a programmer needs to turn it into code, doesn't exist yet. Without it, no one can build the stemmers, tokenisers, or morphological analysers that search engines and NLP pipelines depend on.
Who decides when our languages go digital
This is the part that I keep coming back to.
In November 2024, Google announced Twi support for Voice Search, Gboard, and Google Translate. They built it through their AI Research Centre in Accra as part of a rollout of 15 African languages, which I'm glad to see.
But think about what that means. A company in Mountain View, California, decides when Twi becomes a language that software supports, which means that it sets the timeline, the priorities, and the order in which it adds languages.
Akan has over nine million speakers, Ewe has 3.8 million, and Dagbani has 1.2 million. Millions of people speak these languages today in homes, markets, schools, and churches; however, when those languages appear in software depends on when a Silicon Valley company gets around to them.
Kwame Nkrumah organised people, built coalitions, and demanded independence from the British. We should bring that same spirit to technology.
Western contributions matter, but the Masakhane community shows what Pan-African researchers can produce when they work together. When people who don't speak your language build the tools for it, they can miss what a speaker knows without thinking, since a Dagbani speaker feels how noun suffixes shift across forms and a Yoruba speaker hears the tonal difference between 'Ọkọ́' the farming tool and 'Ọkọ' the family member. That kind of knowledge comes from growing up with the language, so nobody could buy it with a research grant alone.
What we lose
Developers build search engines, autocomplete, recommendation systems, and NLP pipelines on the premise that your language has a stemmer, a tokeniser, and a stopword list. If it has none of those, that software can't process your language.
A Ghanaian student who searches for learning materials in Twi gets worse results than a French student who searches in French, because the tools to find the Twi content that exists are still missing.
Every 6 March, we celebrate independence with flags, parades, and speeches about the sacrifices through which we won self-governance. However, we have yet to win digital self-governance, in which software fully supports our languages because we build the infrastructure ourselves.
I need help
I'm writing this because I need help.
Update, August 2026
The search engine now has 107 language modules, and twenty of them are African: Amharic, Bambara, Dagbani, Ewe, Ga, Hausa, Igbo, Kinyarwanda, Kirundi, Lingala, Malagasy, Oromo, Shona, Swahili, Tigrinya, Twi (Akan), Wolof, Xhosa, Yoruba, and Zulu. That is the tokenisation and stopword layer, which I curate one language at a time in the way that I describe in this post. Swahili is the only one of the twenty with a stemmer. The other nineteen still have none, because the morphological documentation that I ask for in this post still doesn't exist. In the language support guide, I list what every module does.
If you work in computational linguistics, NLP, or language technology, and you speak an African language, then you have knowledge that I need for this work. The translation and speech recognition efforts are a strong foundation, so now we need to go a layer deeper. We need morphological rules for stemming, tokenisers that handle tonal systems and suffix-based noun classes, and stopword lists that native speakers build and linguists validate. All of those then have to be inside the toolkits that developers install every day.
The work starts with the basics. How do speakers of your language form plurals? How do verb tenses work? What suffixes exist, and how do they combine? Those are the building blocks that everything else depends on.
I'm building this search engine in the open, and plenty of language work on it is still to do. I want to build African language support in from day one. But the linguistic work, which is more than one person can do, has to come first.
So here's my ask: let's talk. If you're a linguist, a developer, a researcher, or anyone who cares about this, reach out. Nineteen of the twenty languages in that list still have no stemmer, and for each one we need two people: someone who speaks it, and someone who can write its morphology down at the depth that a programmer can code from. Let's build on the work that Masakhane, NLP Ghana, and the African Stopwords Project are already doing. Pick one language, map its morphology, and build one stemmer, then move to the next.
Ghana gained political independence 69 years ago today, while our languages still lack digital independence. Let's change that, and let's do it ourselves.