ai-toba/toba-trilingual-1.2B

This TOBA model is a trilingual language model based on GPT-2 architecture with 1.2 billion parameters, trained on a corpus encompassing Indonesian, Batak, and Minangkabau using syllabic-agglutinative tokenization. The architecture integrates an Engram Memory mechanism, an adaptive n-gram-based memory system with a 500,000 x 768 embedding table that captures morphological dependencies through bigram and trigram pathways.

Model file:

model.safetensors

Install

Install PyTorch first according to your CPU/CUDA environment, then install the repo requirements:

pip install -r requirements.txt

Usage

Single prompt, chat mode:

python infer.py --mode chat --prompt "Horas amang inang saluhutna"

Single prompt, completion mode:

python infer.py --mode completion --prompt "Horas amang inang saluhutna"

Interactive:

python infer.py --interactive --mode chat

Quick examples:

python infer.py --prompt "Horas!"
python infer.py --mode completion --prompt "Patorang ma aha do dalihan natolu "
python infer.py --interactive

Ref

• arxiv.org/abs/2601.11643
• arxiv.org/abs/2602.06998
• arxiv.org/abs/2603.10006