Small language models: What are they and why should FSIs care?

Last year while attending Febraban Tech 2024, a financial services industry event in LATAM, we were intrigued to learn that 96% of surveyed banks have artificial intelligence (AI) initiatives. We wondered–how many banks are actually taking advantage of AI initiatives? And, how can FSIs take advantage of what small language models have to offer? And, can these advantages apply beyond the world of financial services?

What are small language models?

A key aspect of an AI model is the quantity of parameters that are used to train a given model. When exploring models, you’ll find this number to be in the billions for each model. The greater the value, the richer the learning and more advanced its ability to handle more complex language tasks. The bigger, the better? Well, that depends. If you need to customize a hundreds of billions of parameters model, it will take significant (think of days or weeks) computational and GPU capacity to carry a single round of training of that model.

A small language model (SLM) is a model that you can customize, or fine-tune, with your own data within a reasonable time with currently available hardware (assuming you have accelerated compute resources in place or access to an appropriate cloud instance).

If you do your own research into AI models, you‘ll find that some offer parameter variants of the base model. For example, the Granite 3.1 model family offers from sub-billion to 34 billion parameter variants. Consider starting with lower sized variants to find out the capabilities of a model and scaling up in case of need—and to discover the improvements, if any, a larger parameter model can provide. A smaller model will enable you to start more quickly and tune in a reasonable time frame.

SLMs in financial services

While looking into this topic, we came across an interesting article by Drew Breunig. In his article, he breaks down AI into 3 main use cases: Gods, Interns and Cogs. The “God” case is the full human replacement AI or artificial general intelligence (AGI), where lots of funding and research is currently going; “Interns” produce very good content, but must have their content reviewed to confirm that it is accurate and appropriate; and finally, the “cogs”—where I see SLMs existing—are purpose-built models with a low error rate, allowing them to run fairly unsupervised and a domain where users can achieve significant cost savings.

Tests, code development, stress tests and forecasts are strong candidates for “intern”-type AI. Specialist and low error margins in the financial services industry, such as customer service, hyper-personalization, fraud detection and document treatment are natural candidates for purpose-built and specialist “cog” models. These tasks aren’t unique to FSI in principle, so it’s certainly feasible that other industries can take advantage of SLM-based cogs in a similar fashion.

Benefits of SLMs

Is there interest for larger and more complex AI models? Of course. They can be useful for oversight mechanisms, judge/teach base models or where general broad knowledge is desired, but those can come with their associated costs, complexity and requirements.

Instead of taking a monolithic approach to AI and going all-in on a single LLM, organizations should consider a compound solution of purpose-built models, potentially even a mix of SLMs and/or LLMs. This becomes especially interesting if you’re able to pair these models with existing cloud-native or traditional IT applications, creating an interconnected workflow across the hybrid cloud.

A collection of purpose-built SLMs can help:

• Improve the security posture of your content by controlling all the aspects of your model.
• Accelerate innovation through fine-tuning.
• Segregate access by implementing per model access to back-end systems.
• Improve quality control by versioning each of the models.
• Manage costs and enable efficiency by decreasing a model’s training time.

The good news is that open source tools for experimenting with and building your own SLMs are readily available. I recommend trying InstructLab, a groundbreaking project developed by IBM and Red Hat, that provides model alignment tooling to help organizations more efficiently contribute skills and knowledge to their gen AI models in order to address the needs of their AI-enabled applications and business.

There’s a lot about SLMs to love, not just for the FSI world but for the broader business community at large. Explore your options, and with open source tools and open source-licensed models, you can customize and tune an AI model that works for you. So try it out, and you may realize how using your own SLMs can help you solve your company’s problems.