phonemetransformers
/

GPT2-85M-CHAR-TXT-SPACELESS

Model card Files Files and versions

GPT2 trained on the BabyLM 2024 training set using a character-based tokenizer with word boundaries removed.

Model trained for From Babble to Words: Pre-Training Language Models on Continuous Streams of Phonemes.

Downloads last month: 5

Safetensors

Model size

85.3M params

Tensor type

F32

·

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for phonemetransformers/GPT2-85M-CHAR-TXT-SPACELESS

Base model

openai-community/gpt2

Finetuned

(2090)

this model

Dataset used to train phonemetransformers/GPT2-85M-CHAR-TXT-SPACELESS

Collection including phonemetransformers/GPT2-85M-CHAR-TXT-SPACELESS

From Babble to Words

The models, tokenizers and datasets used in From Babble to Words, one of the winning BabyLM 2024 submissions, exploring phoneme-based training. • 12 items • Updated Apr 3, 2025

Paper for phonemetransformers/GPT2-85M-CHAR-TXT-SPACELESS

From Babble to Words: Pre-Training Language Models on Continuous Streams of Phonemes

Paper • 2410.22906 • Published Oct 30, 2024