update readme

README.md

@@ -4,72 +4,104 @@ We release BriLLM-Chinese and BriLLM-English.
 
 Our paper: https://arxiv.org/pdf/2503.11299
 
+Our Github: https://github.com/brillm05/BriLLM0.5
+
 Our huggingface: https://huggingface.co/BriLLM/BriLLM0.5
 
 
 ## Overview
-This work introduces the first brain-inspired large language model (BriLLM). This is a non-Transformer, non-GPT, non-traditional machine learning input-output controlled generative language model. The model is based on the Signal Fully-connected flowing (SiFu) definition on the directed graph in terms of the neural network, and has the interpretability of all nodes on the graph of the whole model, instead of the traditional machine learning model that only has limited interpretability at the input and output ends. 
 
+BriLLM redefines the foundations of generative language modeling by departing from Transformer architectures, GPT frameworks, and traditional input-output constrained paradigms. Built on the Signal Fully-connected flowing (SiFu) mechanism—a directed graph-based neural network design—BriLLM enables full interpretability across all nodes, in contrast to conventional models limited to input-output interpretability.
+
+In this framework, tokens are represented as graph nodes, with signal flows—either randomly initialized or user-defined—propagating along paths following a "least resistance" principle. The next token to be generated emerges as the target of this signal flow. Theoretically, BriLLM supports infinitely long n-gram modeling, with model size decoupled from input and prediction length. Its signal propagation dynamics mimic human-like cognitive patterns, enabling recall activation and inherent multi-modal compatibility.
+
+![](./figs/tab1.png)
+
+
+## SiFu Mechanism
 
-## SiFU Mechanism
 ![](./figs/fig1.png)
-> As shown in Figure 1, SiFu model is a graph composed of multiple nodes, which are sparsely activated and utilize tensors to transmit a nominal signal.
-Each node (ideally, a layer of neurons) represents a certain concept or word, e.g., a noun, a verb, etc.
-Each edge models the relationship between every node pair.
-The signal is transmitted by the magnitude of the energy. The energy will be strengthened, i.e., maximized, if it is in the right route. Or, at least, the right path always keeps the maximal energy for the transmitted signal.
-Each node is sequentially activated in terms of the maximized energy.
-Route or path is determined in a competitive way, i.e., the next node will be activated only if the energy can be maximally delivered in this node.
 
+The SiFu (Signal Fully-connected Flowing) mechanism addresses fundamental limitations of current machine learning frameworks. Unlike traditional models that process discrete input streams through opaque computations, SiFu operates on a fully connected directed graph where:
+
+- Each node represents an interpretable unit (token, concept, etc.)
+- Signal tensors propagate through the graph following energy dynamics
+- The next token is determined by maximizing signal energy
+- All nodes can serve as both input and output interfaces
 
-## Architecture
 ![](./figs/fig2.png)
-> As shown in Figure 2, BriLLM implements SiFu neural network for language modeling. 
-Each token in the vocabulary is modeled as a node, which is defined by a hidden layer of neurons in the neural network.
 
+Signal propagation follows the principle:
+$v_i = \arg\max_{v'} \left\| r \oplus v_1 \otimes e_{12} \oplus v_2 \ldots \oplus v' \right\|$
+
+where $\oplus$ and $\otimes$ denote tensor operations for node and edge interactions, and $\|\cdot\|$ represents signal energy.
+
+Overall, SiFu's design as a directed fully connected graph with signal propagation confers two key advantages:
+1. **Inherent full interpretability**: User-defined entities (concepts, tokens, or interpretable units) map directly to specific graph nodes;
+2. **Unbounded contextual capacity**: Prediction is framed as signal propagation through node activations. Because signals propagate freely across nodes, sequence prediction naturally supports arbitrarily long contexts without increasing model size. 
+
+
+## Architecture
 
-## Training Network
 ![](./figs/fig3.png)
-> To train a sample in BriLLM, every time we build an individual common neural network to perform the regular BP training. This network consists of two parts, in which the front part connects all input nodes (i.e., tokens), then it follows the rear parts which connect all possible paths in order. At last, a softmax layer collects all paths' energy tensors to indicate the right path with a 0-1 ground truth vector. We adopt a cross-entropy loss for training.
 
+BriLLM implements the SiFu mechanism where each vocabulary token corresponds to a node defined by a GeLU-activated neuron layer with bias $b \in \mathbb{R}^{d_{node}}$. Edges between nodes are modeled as fully connected matrices $W_{u,v} \in \mathbb{R}^{d_{node} \times d_{node}}$, enabling bidirectional signaling.
+
+Signal propagation begins with initial tensor $e_0 = [1, 1, \ldots, 1]^T \in \mathbb{R}^{d_{node}}$ and follows:
 
-## Dataset
-We use the subset from the Chinese version of Wikipedia, which contains over 100M Chinese characters. We truncate the long sentences into small sentences with a maximum length of 16.
-We select a vocabulary of 4,000 tokens consisting of the most frequently used Chinese characters.
+$e_{i+1} = \text{GeLU}(W_{u_i,u_{i+1}} e_i + b_{u_i,u_{i+1}} + PE_i)$
 
+The final prediction maximizes the L2 norm: $v_{predict} = \arg\max_v \|E_{u,v}\|_2$
+
+
+## Training Network
 
-## Implementation Details.
-BriLLM is implemented using PyTorch. 
-It uses sinusoidal positional encoding, GeLU as the activation function, cross-entropy loss for next-token prediction, and an embedding size of $d_{model} = 32$. 
-We used the AdamW optimizer with $\beta_1 = 0.9$, $\beta_2 = 0.999$ and $\epsilon = 10^{-8}$. 
-The model size is about $512 + 4000 * 4000 * (32 * 32 + 32) \approx 16B$.
-We trained our models on one machine with 8 NVIDIA A800 GPUs for 1.5k steps. 
 ![](./figs/fig4.png)
 
+Training BriLLM involves constructing a dedicated neural network for each sequence sample. The network connects input nodes sequentially, with all potential paths integrated into a final softmax layer that identifies the correct path via cross-entropy loss optimization.
 
-## Complexity
-$n$ is the sequence length, $v$ is the vocabulary size, and $d$ is the representation dimension. The computational complexity is $O(n \cdot v \cdot d^2)$.
 
+## Implementation Details
+
+BriLLM is implemented using PyTorch. It uses sinusoidal positional encoding, GeLU as the activation function, cross-entropy loss for next-token prediction, and an embedding size of $d_{model} = 32$. We used the AdamW optimizer with $\beta_1 = 0.9$, $\beta_2 = 0.999$ and $\epsilon = 10^{-8}$. The model size is about $512 + 4000 * 4000 * (32 * 32 + 32) \approx 16B$. We trained our models on one machine with 8 NVIDIA A800 GPUs for 1.5k steps.
 
-## Case Study
 ![](./figs/fig5.png)
-![](./figs/fig7.png)
 
 
-## Comparison of LLM and BriLLM
-![](./figs/fig6.png)
+BriLLM leverages sparse token co-occurrence: most bigrams are low-frequency or absent, allowing shared parameters for inactive edges. Low-frequency bigrams use a fixed, non-updated matrix, reducing model size to 2B (Chinese) and 1B (English)—13.0\% and 5.7\% of the original size, respectively. This reduces parameters by ~90\% while accelerating training.
+
+![](./figs/tab2.png)
+
+
+## Case Study
+
+### Chinese Examples
+![](./figs/tab3.png)
+
+
+### English Examples  
+![](./figs/tab4.png)
+
+
+## Comparison: Traditional LLMs vs BriLLM
+
+![](./figs/tab5.png)
 
 
 ## Installation
+
 ```bash
 pip install torch
 ```
 
 
-## Checkpoint
+## Model Checkpoints
+
 [BriLLM0.5](https://huggingface.co/BriLLM/BriLLM0.5)
 
 
-## Train
+## Training
+
 ### BriLLM-Chinese
 ```bash
 bash run_zh.sh
@@ -81,8 +113,8 @@ bash run_en.sh
 ```
 
 
-
 ## Inference
+
 ### BriLLM-Chinese
 ```python
 import json
@@ -121,7 +153,6 @@ decode_sentence = decode_tuple_list[0][0] + "".join([p[-1] for p in decode_tuple
 print(decode_sentence)
 ```
 
-
 ### BriLLM-English
 ```python
 import json
@@ -141,7 +172,6 @@ def decode_en_sentence(head, max_token=32, do_sample=False):
     decode_sentence = decode_tuple_list[0][0] + "".join([p[-1] for p in decode_tuple_list])
     return decode_sentence
 
-
 with open("./vocab_wiki_4k_en.json") as f:
      node_dict = json.load(f)
 vocab = Vocab.from_node_dict(node_dict)