How Saheed Azeez, a UNILAG Undergraduate, Built a Dataset of 230 Million GPT-2 Tokens from Nairaland

Prior to embarking on his Naijaweb project, Azeez identified that Nigerians lacked sufficient data on prominent and emerging matters on the web. As a natural problem-solving machine learning engineer, he became interested in creating something that would make it easier for Nigerians to access data — in a detailed, accurate, and specific manner.

Saheed Niyi is a final year Mechanical Engineering student from the University of Lagos, Nigeria. Azeez learned about web scraping and data cleaning in a Python class he joined in 2019. These two skills came in very handy while he was building NaijaWeb.

As a young, naive mechanical engineering student, Azeez had always thought of machine learning to be a field that deals with lots of robots, machines, and cyborgs. However, as classes began, he soon realized that there were no such physical robots in ML, but rather than be devastated…

Azeez already picked up an interest in Machine Learning. When COVID-19 struck in 2020, he spent time at his desk taking everything ML seriously — participating in many machine learning competitions he found on Zindi. Azeez won none of these competitions, but he was learning.

He proceeded to learn how to build from scratch.

In 2022, Azeez built Tweet Shot, a screenshot bot, using the Twitter API. Amassing over 174,000 followers and with over 1M+ screenshots saved, Tweet Shot was a viral success.

His first attempt at creating Naijaweb was also in 2022.

Creating a dataset like Naijaweb is not an all-round solution to the unavailability and inaccessibility of data to Nigerians, but it could serve as a major stepping stone.

When OpenAI built GPT-2, they used a dataset called WebText which was built by extracting outbound links from Reddit, a social networking platform used by millions of people around the world.

Azeez creatively thought that since NaijaWeb would cater to Nigerian data needs, extracting outbound links from high-traffic Nigerian sites like Nairaland was a good option. In fact, Azeez claimed to have heard people talking a lot about the amount of value Nairaland possesses, which made him decide to web-scrape the platform.

Unfortunately, his first attempt was unsuccessful. According to him, the script he used back then didn’t support asynchronous programming. This year, 2024, when Azeez tried completing his project, he figured it out—thanks to Hugging Face who had released FineWeb, an easy-to-use library to aid the execution of ML and data science projects.

It was with the same cleaning done on the open-source FineWeb that Azeez cleaned the webpages he extracted from Nairaland.

Now completed, Naijaweb is a 270,000-document dataset (230 Million GPT-2 tokens) of web pages which Nigerians have shown interest in. This dataset opens the door to more possibilities. It can be used to build language models and, according to Azeez, anything to solve Nigerian problems.

There are several hindrances to building these kinds of AI projects for the average Nigerian, which include access to high-end laptops and constant electricity. When Azeez was creating the Naijaweb dataset, for instance, it required him to keep his laptop running for days.

In the same vein, creating large language models (LLM) or even more specific small language models (SLM) is not a one-man job, but it is not impossible.

“There are Nigerians that are very skilled in these things who have gone to do their PhD abroad. I know a UNILAG graduate who built a small LLM one time.”

Azeez himself currently works as a Machine Learning Engineer with HelpMum, a non-profit AI startup dedicated to improving maternal and infant healthcare systems in Africa. Additionally, he is personally building FPL AI, a tool that aids in gameweek predictions for Premier League fixtures.

You can find Azeez Saheed on Twitter @saheedniyi_02

Every day, it gets increasingly exciting to see Nigerians like Saheed Niyi finding their Eureka moments in the tech industry. Will you be the next?