• 5 min read

Finding root words in a language no NLP library supports

Table of Contents

Building Aklish uncovered a familiar problem.

One feature of the app was a dictionary-based spell checker. It worked well until I started testing inflected words.

Take nagsueat.

The dictionary contains sueat, not nagsueat. A simple dictionary lookup treats it as an unknown word even though it’s just an inflected form of the same root. The same issue kept appearing because Aklanon, like many Philippine languages, changes words through prefixes, suffixes, infixes, and reduplication.

I had already solved this problem once while building TagLID by creating TglStemmer for Tagalog.

This time, I needed the same solution for Aklanon.

I looked for an existing Aklanon stemmer that I could integrate into Aklish. There wasn’t one. So I built AklStemmer.

AklStemmer is a Python library that recovers the root form of Aklanon words through rule-based transformations. It removes affixes, reduces repeated syllables, applies phonological rules, and filters the results against a dictionary to find the most likely stem.

Why I built it

AklStemmer started as an internal component of Aklish.

The spell checker relied on dictionary lookups, but dictionaries usually contain root words rather than every possible inflected form. Expanding the dictionary wasn’t realistic because there are too many possible combinations. Recovering the root word first was a much cleaner solution.

As the project grew, I split the stemmer into its own library so it could be reused outside Aklish.

There was another reason I cared about building it.

Aklanon is my mother tongue, but it’s also a very low-resource language. There are very few NLP tools available for it, and compared to larger languages, its digital presence is much smaller. I enjoy building software for Filipino languages in general, not just Tagalog, and AklStemmer felt like one small step toward making Aklanon easier to work with computationally.

How I built it

I didn’t start from scratch.

The overall architecture came from TglStemmer, but I couldn’t simply copy the rules. Aklanon has its own morphology, phonological changes, and affix system, so many of the stemming rules had to be rewritten.

At a high level, AklStemmer generates possible root words instead of assuming the input already exists in a dictionary.

Each input first goes through tokenization using NLTK so words can be processed individually. From there, the stemmer applies a sequence of rule-based transformations based on Aklanon grammar.

Most of the linguistic rules came from an Aklanon grammar reference. Because there were no existing stemmers to build on, I had to translate those linguistic descriptions into code myself.

Removing affixes

The first step is stripping prefixes, infixes, and suffixes.

Like Tagalog, removing affixes often requires reversing spelling changes introduced during inflection. The stemmer therefore applies several phonological transformation rules while removing affixes.

For prefixes, that includes examples such as:

  • parayawdayaw
  • pangablitkablit
  • pamahawbahaw
  • pamasyarpasyar
  • panumdumdumdum

It also removes infixes like <in>:

  • sinueatsueat

Suffixes require another collection of transformation rules, including contractions, phoneme changes, vowel loss, and metathesis.

Some examples are:

  • eoteon
  • bayaribayad
  • sugilanonsugid
  • agyanagi
  • tubwantubo
  • buksabukas
  • islanilis

Handling reduplication

The stemmer also recognizes reduplication, another common feature of Aklanon.

Partial reduplication:

  • aabotabot
  • babakaebakae

Full reduplication:

  • ano-anoano
  • anu-anoano
  • ibat-ibaiba

Choosing the best stem

Applying these rules often produces multiple possible stems.

For example, bukot could generate:

  • bukot
  • buko
  • bukon

Instead of selecting the first result, AklStemmer scores every candidate.

The scoring favors stems that require fewer transformations, remove more meaningful affixes or reduplication, and produce more morphologically plausible words. The highest-scoring candidate becomes the final stem.

How it improved Aklish

Adding the stemmer made the spell checker significantly more useful.

Instead of treating many inflected words as unknown, Aklish could recover their root forms before checking the dictionary. That allowed the spell checker to recognize many valid words that would otherwise have been flagged incorrectly.

Separating the stemmer into its own library also kept the architecture cleaner. Aklish could focus on spell checking while AklStemmer handled the language-specific morphology.

What I learned

This project reminded me that supporting a low-resource language is very different from working with one that already has an NLP ecosystem.

When I built TglStemmer, there were at least papers and existing work I could learn from. With Aklanon, there were no libraries I could build on. Much of the work involved reading grammar references and translating linguistic rules into code.

I also discovered rules in my own language that I had never consciously thought about, even though I had been using them my whole life. Writing software forced me to understand Aklanon at a level far beyond simply speaking it.

What I’m most proud of isn’t the stemming algorithm itself. It’s that I built an NLP tool for my own mother tongue.

Languages like English receive most of the attention in NLP. Even Tagalog has a much larger ecosystem than Aklanon. I want to help improve Filipino NLP as a whole, especially for regional languages that rarely receive that kind of support. AklStemmer is one contribution toward that goal.