Activity Feed

AI & ML interests

None defined yet.

Recent Activity

codelion 
posted an update 28 days ago
view post
Post
436
SPROG-9M — a 9.37M parameter model trained from scratch to solve GSM8K-style math without using an LLM at inference.

The model, codelion/sprog-9m, predicts symbolic programs over number slots, then a deterministic executor does the arithmetic. With a simple verifier, it reaches ~11.8% on GSM8K test.

We also released the dataset: codelion/gsm8k-synth, 117K validated synthetic GSM8K-style problems.

Tiny model, no pretraining, no LLM at inference, runs on a laptop.
AmelieSchreiber 
posted an update about 1 month ago
view post
Post
2227
Latest OpenAI Parameter Golf Competition Training Run BPB (<1K steps on a single 4090) See: ToricBLM, ToricGT, and TropicalGT methods
  • 1 reply
·
AmelieSchreiber 
posted an update about 1 month ago
view post
Post
169
In algebraic geometry, embedding tropical varieties into toric varieties relies on the process of tropicalization and the use of toroidal embeddings. Rather than embedding as geometric spaces, tropical varieties are embedded as polyhedral fans or cone complexes within the fans of target toric varieties to study compactifications and intersection theory.

Toric Varieties: Geometric varieties constructed from combinatorial data known as fans (collections of polyhedral cones). Tropicalization: A procedure that maps an algebraic variety X (typically a subvariety of an algebraic torus T) to a polyhedral complex in a real vector space via valuations. Toric Closures: When you embed a tropical variety in a compact toric variety, the boundary points correspond to limits of points in the variety going towards the boundary of the torus. Embedding Methodologies...Toric Compactification: Any tropical variety (a rational polyhedral fan) can be compactified by embedding it into a complete toric variety. The fan of the toric variety is chosen to contain the tropical variety as a subcomplex, which encodes the combinatorics of the compactification. Toroidal Embeddings: For varieties that are not strict toric varieties, they can often be mapped into toric varieties using the theory of toroidal embeddings. This involves equipping the tropicalized spaces (cone complexes) with balancing conditions and intersection theories.
its5Q 
posted an update about 1 month ago
view post
Post
246
I have downloaded about 8.7 million of 2048x1024 (or 1664x832 for older captures) panoramas randomly distributed all over the world that I toyed around with training geo-guessing models. Is anybody interested in a dataset of Google Street View panorama images along with latitude-longitude coordinates?
dippatel1994 
posted an update about 2 months ago
view post
Post
1103
To make revising LLM architectures and training methods faster, I created a deck of 180 visual flashcards. It started as a personal hobby, but slowly became cheat code for reviewing LLM concepts before technical interviews. People love it!

Swipe through these samples, and if you want to grab the full set or follow the project, the repo is here: https://github.com/llmsresearch/llm-flashcards.
codelion 
posted an update 2 months ago
view post
Post
3249
Inspired by the Nemotron Diffusion recipe, check out dhara-250m: a 250M experimental language model that supports three decoding modes from one set of weights: autoregressive, block-diffusion, and self-speculation.

It is small, easy to try, and meant for exploring diffusion-style decoding and latency tradeoffs in compact LMs.

Model: codelion/dhara-250m

Try the chat demo here: codelion/dhara-chat
  • 3 replies
·
AmelieSchreiber 
posted an update 2 months ago
BlueNipples 
posted an update 3 months ago
view post
Post
2983
Good news, llama.cpp seems to be close to supporting MTP on qwen models. Bad news, every single gguf will have to be redone when it is.
  • 1 reply
·