Dataset Generation Pipeline
In this section, we provide an overview of the question-answer creation pipeline that we developed to create HourVideo.
The pipeline is summarized in Fig.~\<PIC>.
A crucial design consideration for this benchmark is the selection of video sources and types. We chose the Ego4D~\cite{grauman2022ego4d} dataset for our videos for multiple reasons: (1) its egocentric perspective
aligns well with
the typical visual input for autonomous agents and assistants; (2) it features extensive visual narrations, which aid in creating diverse multiple-choice questions; and (3) it is readily accessible under the Ego4D license. We manually reviewed 1,470 videos, ranging from 20 to 120 minutes, from the Ego4D dataset, assessing their potential to generate relevant questions for various tasks in our task suite.
We engaged five human experts for video curation.
Following this process, we curated 500 egocentric videos.
Candidate MCQ Generation, Stage 2.
The objective of this stage is to produce high-quality MCQs for each task,
requiring analysis and synthesis of information across multiple temporal segments in a long-form video. Initially, we manually develop question template(s) for each task in the suite. As shown in Table~\ref{table_main:task_suite}, transforming a question template into an actual question involves incorporating video-specific information tailored to the task and template.
To facilitate this, we utilize the detailed narrations from the Ego4D dataset, transforming them into a structured format that can be processed by an LLM.
Specifically, we segment the video at 20-minute intervals,
with each segment's representation including a summary and a list of tools, food items, technology, humans, pets, and physical locations encountered by the camera wearer in the video.
We note that synthesizing a structured representation and a question template into a valid question with correct and incorrect answers presents a significant challenge, even for advanced LLMs.
Consequently, for each task, we formulate detailed prompts that offer question prototypes, comprehensive instructions, in-context examples, and step-by-step guidance on how to transform a question template into a valid candidate $\QAW_{2}$.
In total, we developed 25 task-specific prompts
Refinement with LLMs using Human Feedback, Stage 3.
The purpose of this phase is to refine $\QAW_{2}$, created in the
previous
stage. $\QAW_{2}$ may contain invalid questions, incorrect answers,
trivial incorrect options,
and various other issues.
We identified that a significant source of these issues stemmed from relying on the noisy narrations in Ego4D.
For example, different narrators within the same video could refer to a dishwasher as a "plate rack" or use other terms, and an individual might be described as an "adult," "person with a red and white shirt," "man Y," or "teenager" at various times in the narration.
These inconsistencies, combined with our automatic question generation in the first stage, could lead to
generation
of invalid
To address noisy {\QAW}s, we implement a human feedback system where trained annotators are tasked with: 1) assessing the validity of each question to ensure it aligns with the video content, 2) verifying the accuracy of the given answer—if found incorrect, they provide the correct answer in free-form text, 3) ensuring that all incorrect options are factually wrong and clearly distinguishable from the correct answer.
We gather human feedback for all $\QAW_{2}$, involving over 400 hours of human
effort.
We then design prompts, to automatically refine $\QAW_{2}$ using this human feedback to produce $\QAW_{3}$.
We engaged seven trained annotators in this stage.
Blind filtering, Stage 4. Modern LLMs possess extensive prior knowledge and can thus easily answer certain questions without needing to analyze the videos. The objective of this phase is to eliminate questions that can be answered through prior knowledge or can be trivially answered without requiring any information from the video. To address this, we do blind filtering of $\QAW_{3}$, utilizing two separate blind LLMs (GPT-4-turbo and GPT-4). Specifically, we exclude any MCQ that is correctly answered by at least one LLM without video input. Although this method may aggressively remove MCQs, it ensures that the remaining $\QAW_{4}$ are of high quality and specifically tailored to test long-form video-language understanding.
Expert Refinement, Stage 5.
The aim of this stage is to enhance the quality of $\QAW_{4}$ by utilizing a selected group of expert human annotators.
This stage serves as a comprehensive step to address various remaining issues that might have persisted through prior stages. Examples of expert refinement include transforming a broad question like "Where did the camera wearer leave the keys?" into a more precise query: "Where did the camera wearer leave the bike keys after returning home from shopping?”
Over 300 hours of expert human effort are employed in this stage to carefully examine and refine $\QAW_{4}$, culminating in a high-quality $\QAW_{5}$.
We engaged four human experts in this stage.
HourVideo Statistics
HourVideo consists of 500 videos from the Ego4D dataset, covering 77 \textit{daily life scenarios} such as cooking, cleaning, eating, watching TV, baking, etc. (Fig.~\<PIC>). The dataset includes 381 hours of video footage, with video durations ranging from 20 to 120 minutes (Figure~\<PIC>). On average, each video is approximately 45.7 minutes long, which 15$\times$ larger than prior work in long-form video-language understanding~\cite{mangalam2024egoschema}. Additionally, 113 videos in our dataset exceed one hour in length. Each video is accompanied by an average of 26 high-quality, five-way multiple-choice questions, totaling  12,976 questions in the dataset.
Finally, we strive to ensure an even distribution of {\QAW}s across all tasks in our suite, with the exception of causal, counterfactual, and navigation tasks, where questions were manually generated for a selected group of videos.


Introduction
Recent advances in video diffusion models demonstrate significant improvements in visual and motion quality ~\citep{VideoDiffusionModels,blattmann2023stable,AlignYourLatents,LVDM}.
These models typically take a text prompt~\citep{VideoDiffusionModels,blattmann2023stable,ImagenVideo} or image~\citep{VideoCrafter1,VideoCrafter2,AnimateDiff,DynamiCrafter} as input and generate video frames of a photorealistic, animated scene.
Current methods can generate videos that are largely consistent with an input text description or image; however, fine-grained adjustment of specific video elements (e.g., object motion or camera movement) is conventionally a tedious process that requires re-running the model with different text prompts or random seeds~\citep{wu2023freeinit,qiu2024freetraj}.
Approaches for controllable video generation aim to eliminate this process of trial-and-error through direct manipulation of generated video elements, such as object motion~\citep{wu2024draganything,yin2023dragnuwa,wang2024boximator}, pose~\citep{AnimateAnyone,MagicAnimate}, and camera movement~\citep{MotionCtrl,li2024image,CameraCtrl,MotionMaster}.
One line of work fine-tunes pre-trained video generators to incorporate control signals such as bounding boxes or point trajectories~\citep{wu2024draganything,MotionCtrl}.
One of the primary challenges with these supervised methods is the expensive training cost, and thus, previous methods usually incorporate trajectory control by fine-tuning at a lower resolution than the original model~\cite{wu2024draganything, yin2023dragnuwa}.
More recently, several methods for zero-shot, controllable text-to-video generation have been developed~\citep{ma2023trailblazer,qiu2024freetraj,jain2024peekaboo}.
They control object trajectories by modulating the cross-attention maps between features within a bounding box and an object-related text token.
Still, it is not always possible to associate a desired edit with the input text prompt (consider, e.g., motion of object parts).
Moreover, these methods cannot be directly applied to animate existing images, as they are only conditioned on text.
In this work, we propose SG-I2V, a new method for controllable image-to-video generation.
Our approach is \textit{self-guided}, in that it offers zero-shot control by relying solely on knowledge present in a pre-trained video diffusion model.
Concretely, given an input image, a user specifies a set of bounding boxes and associated trajectories.
Then, our framework alters the generation process to control the motion of target scene elements.
It is essential to manipulate the structure of the
generated video to achieve precise control over element positions, which is mainly decided by early denoising steps~\citep{balaji2022ediff,DMProcessAnalyze}.
In image diffusion models, it is known that feature maps extracted from the output of upsampling blocks are \textit{semantically aligned}, i.e., pixels belonging to the same object share similar feature vectors on the feature map and thus can be used to control the spatial layout of generated images~\citep{tang2023emergent, shi2024dragdiffusion, namekata2024emerdiff, tumanyan2023plug}.
However, our analysis reveals that feature maps extracted from the upsampling blocks of video diffusion models are only weakly aligned across frames (see~\<PIC>).
This misalignment poses challenges, as directly manipulating these feature maps fails to give useful guidance signals for layout control. %
Instead, we find that feature maps extracted from the self-attention layers can be semantically aligned by replacing the key and value tokens for each frame with those of the first frame (see bottom row of \<PIC>).
After that, we can control the motion of generated videos by optimizing the latent (the input to the denoising network) with a loss that encourages similarity between the aligned features within each bounding box along the input trajectory.
Finally, we apply a post-processing step to enhance output quality by ensuring that our optimization does not disrupt the distribution of high-frequency noise expected by the diffusion model.
In summary, our work makes the following contributions:
In this section, we describe our method for the trajectory control task in image-to-video generation (\cref{sec:3_task}).
Our framework, SG-I2V, builds on the publicly available image-to-video diffusion model Stable Video Diffusion~(SVD)~\citep{blattmann2023stable},
and consists of two main steps.
First, we extract and semantically align the feature maps from a specific layer of SVD during the early steps of the diffusion process (\cref{sec:motion_estimation}); we show that such feature maps are especially effective at influencing motion in the output video.
Second, we optimize the noisy latent (i.e., the input to the denoising network) to enforce similarity between features within the bounding box trajectories (\cref{sec:motion_optimization}).
However, we find that naive optimization of latent is prone to overfitting and often results in low-quality generation.
Thus, we employ frequency-based post-processing to retain an in-distribution noisy latent (\cref{sec:fft}).
Our entire pipeline is summarized in \<PIC>.
Preliminaries: Stable Video Diffusion.
Video diffusion models~\citep{VideoDiffusionodels} learn a data distribution $\bm{x}_0 \sim p_\theta(\bm{x}_0)$ by gradually denoising a video corrupted by Gaussian noise.
The output denoised video is thus drawn from the distribution $p_\theta(\bm{x}_0) = \int p_\theta(\bm{x}_{0:T}) \ d\bm{x}_{1:T}$, where $\bm{x}_0 \in \mathbb{R}^{N \times H \times W}$ is a clean video, and $\bm{x}_{1:T}$ are intermediate noisy samples.
For simplicity, we omit the channel dimension throughout the paper.
To reduce computation, Stable Video Diffusion (SVD)~\citep{blattmann2023stable} performs the diffusion process in a latent space, where a variational autoencoder~\citep{VAE} maps a raw video $\bm{x}_0$ to a latent $\bm{z}_0 \in \mathbb{R}^{N \times h \times w}$. \\
Since this work aims to animate an existing image, we utilize the image-to-video variant of SVD, which concatenates a conditioning frame with noisy latent ($\bm{z}_t$) and runs a 3D U-Net~\citep{U-net} to predict the noise.
The 3D U-Net contains a downsampling and an upsampling path.
Specifically, the upsampling path consists of three stages operating at different resolutions, where each stage contains three blocks with interleaved residual blocks~\citep{ResNet}, spatial, and temporal attention layers~\citep{Transformer}.
We will call these three stages bottom, middle, and top from lower to higher resolution.
For more details, we refer readers to the original paper of SVD~\citep{blattmann2023stable}.
In image diffusion models, prior work has shown that output feature maps of upsampling blocks in the middle stage of the denoising U-Net are \textit{semantically aligned}~\citep{tang2023emergent, namekata2024emerdiff, hedlin2024unsupervised, zhang2023tale, luo2023dhf}, i.e., regions belonging to the same object tend to have similar feature vectors. %
Such semantically aligned feature maps are useful in estimating the layout of generated images, enabling spatial control of objects~\citep{shi2024dragdiffusion, mou2023dragondiffusion}.
Therefore, we first examine whether SVD feature maps are also semantically correlated across \textit{both spatial and temporal dimension}. %
\c<PIC> visualizes the principal components of feature maps extracted from the upsampling block and spatial attention layers.
We observe that SVD feature maps exhibit weak semantic correspondence across frames at early denoising steps, leading to inaccurate object trajectory estimation.
Yet, we want to operate at early steps as they decide the structure of generated videos~\citep{materzynska2023customizing}.
This dilemma prompts us to align these features before applying optimization.{Feature alignment with modified self-attention.}
SVD leverages separate spatial and temporal self-attention to model the entire video.
Since spatial self-attention is only applied per frame, it does not produce cross-frame aligned features.
While temporal attention communicates across frames, it only attends to the same pixel position on the feature map, which may be inadequate for capturing semantic information spatially. %
To address this issue, inspired by \citep{Tune-a-video}, we modify the spatial self-attention on each frame to directly attend to the first frame.
Concretely, for the $n$-th frame, the original spatial self-attention works as $\bm{F}_n =
\mathrm{Softmax}(\frac{\bm{Q}_{n} \cdot \bm{K}_{n}^T}{\sqrt{D}})\cdot \bm{V}_{n}$, where $\bm{F}_n$ %
is the %
outputs of self-attention, $\bm{Q}, \bm{K}, \bm{V}$ are the query, key, and value tokens, respectively, and $D$ is the dimensionality of the key and query tokens~\cite{Transformer}.
Instead, we replace the key $\bm{K}_{n}$ and value $\bm{V}_{n}$ of each frame with $\bm{K}_{1}$ and $\bm{V}_{1}$ from the first frame, leading to a new operation $\tilde{\bm{F}}_n =
\mathrm{SoftMax}(\frac{\bm{Q}_{n} \cdot \mathrm{SG}(\bm{K}_{1})^T}{\sqrt{D}}) \cdot \mathrm{SG}(\bm{V}_{1})$.
Notably, we apply a stop gradient $\mathrm{SG}(\cdot)$ on $\bm{K}_{1}$ and $\bm{V}_{1}$ to stabilize the subsequent optimization process.
Now, all the modified %
feature maps $\tilde{\bm{F}}_n$ are weighted combinations of $\bm{V}_1$, exhibiting a stronger cross-frame correspondence while still maintaining the object layout of each frame, as shown in the bottom row of \c<PIC>.


The ability to learn concepts about a novel task, such as the goal and motion plans, from a few demonstrations is a crucial building block for intelligent agents -- it allows an agent to learn to perform new tasks from other agents (including humans) from little data.
Humans, even from a young age, can learn various new tasks from little data and generalize what they learned to perform these tasks in new situations \cite{lake2019human}.
In machine learning and robotics, this class of problems is referred to as Few-Shot Learning \cite{parnami2022learning}. Despite being a widely studied problem, it remains unclear how we can enable machine learning models to learn concepts of a novel task from only a few demonstrations and generalize the concepts to new situations, just like humans do. Common approaches learn policies either directly, which often suffer from covariate shift \cite{shimodaira2000improving}, or via rewards \cite{ziebart2008maximum,ng2000algorithms, fu2017learning}, which are largely limited to previously seen behavior \cite{ghasemipour2020divergence}.
In a different vein, other work has relied on pretraining on task families and assumes that task learning corresponds to learning similar tasks to ones already seen in the task family \cite{duan2017one,finn2017one}.
Inspired by the success of generative modeling in few-shot visual concept learning \cite{lake2015human,gal2022image,liu2023unsup}, where concepts are latent representations, in this work, we investigate whether and how few-shot task concept learning can benefit from generative modeling as well. Learning concepts from sequential demonstrations rather than images is by nature more challenging due to sequential data often not satisfying the i.i.d. assumption in machine learning \cite{oord2016wavenet}. In particular, we assume access to a large pretraining dataset of paired behaviors and task representations to learn a conditional generative model that synthesizes trajectories conditioned on task descriptions. We hypothesize that by learning a generative model conditioned on explicit representations of behavior, we can acquire strong priors about the nature of behaviors in these domains, enabling us to more effectively learn new behavior that is not within the pretraining distribution, given a limited number of demonstrations, and further generate the learned behavior in new settings.
To this end, we propose Few-Shot Task Learning through Inverse Generative Modeling (\methodname{}). In our approach, we first pretrain a large conditional generative model which synthesizes different trajectories conditioned on different task descriptions. To learn new tasks from a limited number of demonstrations, we then formulate few-shot task learning as an {\it inverse generative modeling problem}, where we find the latent task description, which we refer to as a \textit{concept}, which maximizes the likelihood of generating the demonstrations. This approach allows us to leverage the powerful task priors learned by the generativeen demonstrations without finetuning the model (Figure~\<PIC>). We demonstrate this approach in various domains: object rearrangement, where concepts are relations between objects, goal-oriented navigation, where concepts are target attributes, motion capture, where concepts are human actions, autonomous driving, where concepts are driving scenarios, and real-world table-top manipulation where concepts are manipulation tasks (Figure~\<PIC>).
New concepts are either (1) compositions of training concepts ({\it e.g.}, multiple desired relations between objects that define a new object rearrangement concept) or (2) new concepts that are not explicit compositions in the natural language symbolic space of training concepts ({\it e.g.}, a new human motion `jumping jacks' is not an explicit composition of training concepts `walk', `golf' etc.). Thanks to generative models' compositional properties that enable compositional concept learning \cite{liu2022compositional}, in addition to being able to learn a single concept from demonstrations directly, \methodname{} learns compositions of concepts from demonstrations that, when combined, describe the new concept.
We show that our approach generates diverse trajectories encapsulating the learned concept. We achieve this due to two properties of generative models. First, these models have shown strong interpolation abilities \cite{ramesh2021zero,saharia2022photorealistic}, which allow generating the new concept on new initial states they were not demonstrated from. Second, these models have compositional properties that enable compositional trajectory generation \cite{ajay2023is}, which allow composing learned concepts with training concepts to synthesize novel behavior that was not demonstrated ({\it e.g.}, `jumping jacks' and `walk'), see Figure~\ref{fig:generation}. We further demonstrate that our approach addresses a unique challenge introduced in learning task concepts: we utilize plans generated by learned concepts in a closed-loop fashion.
Our main contributions are (1) formulating the problem of task learning from few demonstrations as Few-Shot Task Learning through Inverse Generative Modeling (\methodname{}), (2) adapting a method for efficient concept learning to this problem based on the new formulation, and (3) a systematic evaluation revealing the ability of our method to learn new concepts across a diverse set of domains.
section{Experiments}
We demonstrate results in four domains where concept representations are T5 \cite{raffel2020exploring} embeddings of task descriptions in natural language for training, and empty string embeddings for the dummy condition. During few-shot concept learning, we are provided with three to five demonstrations of a composition of training concepts or of a novel concept that is not an explicit composition of training tasks in natural language symbolic space. We ask a model to learn the concept from these demonstrations.
{Learning concepts describing goals that are spatial relations between objects.} Object rearrangement is a common task in robotics \cite{shah2018bayesian, yan2020robotic, rowe2019desk} and embodied artificial intelligence (AI) \cite{puig2020watch,srivastava2022behavior}, serving as a foundation for a broader range of tasks such as housekeeping and manufacturing. Here, we use a 2D object rearrangement domain to evaluate the ability of our method to learn task specification concepts. Given a concept representing a relation between objects, we generate a single state describing that relation. The concept in a training example describes the relation (either `right of' or `above') between only one pair of objects (out of three objects) in the environment. Then, a model must learn compositions of these pairwise relations and new concepts such as `diagonal' and `circle' (see Figure~\ref{fig:shapes}). The results in Figures~\ref{fig:shapes-new-concept-compos-qual} and~\<PIC> demonstrate that our method learns unseen compositions of training concepts and new concepts. They further demonstrate how our method composes new concepts with learned concepts. For additional qualitative results, please refer to Appendix~\ref{appendix:add-res}.
While successful in most cases, there are also a few failure examples. The accuracy for the new `circle' concept is low ($0.44$) compared to the mean over task types in Figure~\<PIC> Object Rearrangement New Concept ($0.82\pm0.09$). This is most likely due to this concept lying far out of the training distribution. The task `square right of circle $\land$ triangle above circle' has low accuracy for 2 concepts ($0.32$) compared to the mean in Table~\ref{tab:ablation} Object Rearrangement Training Composition ($0.75\pm0.11$). This may arise from the combined concept-weight optimization process -- as there is no explicit regularization on weights, they may converge to 0 or diverge. In Figure~\ref{fig:shapes-tab4-qual}, we show that concept components may or may not capture new concept relations.
textbf{Learning concepts describing goals based on attributes of target objects.} We test our method in a goal-oriented navigation domain adapted from the AGENT dataset \cite{shu2021agent}, where an agent navigates to one of two potential targets. Conditioned on a concept representing the attributes of the desired target object and initial state, we generate a state-based trajectory describing an agent navigating to the target. Each object has a color and a shape out of four possible colors and four shapes. During training, we provide 16 target-distractor combinations that include all colors and shapes (but not all combinations), and a concept is conditioned on one of the target's attributes ({\it e.g.}, color). We introduce new concepts defined by both target attributes, including (1) unseen color-shape target combinations and (2) new target-distractor combinations. Figure~\ref{fig:AGENT} shows an example. In training, we see bowl and red object targets. A new concept includes a novel composition as the target -- red bowl. The new concept distractor objects (green bowl and red sphere) were introduced during training, but they were not paired with a red bowl as the target. As Figure~\ref{fig:AGENT-open} shows, our method successfully learns concepts where targets are new compositions of target attributes in settings with new target-distractor pairs and generalizes to new initial object states. We further evaluate our model and baselines in closed loop (Figure~\<PIC>) by making an additional assumption that a planner is provided. The planner produces an action given a current state and a future desired state predicted by a model.


IntroductionRecent works~\cite{bitnet2} have shown that 1-bit LLMs can match the full-precision models given the same amount of parameters and training tokens while being significantly cost-effective in terms of latency, memory, throughput, and energy consumption. With model weights represented in 1.58-bit (i.e., \{-1, 0, 1\}), the bottleneck of inference has shifted from the limited memory bandwidth to high computational cost. Low-bit or sparse activations in LLMs served as a promising approach to further reduce the computational budget while maintaining performance on downstream tasks.
One common approach is to utilize activation sparsity~\cite{dejavu, turbosparse, teal}, which reduces the inference FLOPs and the I/O of weight by pruning the activation entries with smaller magnitudes. Sparsification is particularly well-suited for handling activations that exhibit highly imbalanced long-tailed distributions. Recent works~\cite{qsparse} have demonstrated that LLMs with fully sparsely-activated activations can achieve results comparable to dense models while having much less active parameters.
In addition to sparsification, activation quantization is another approach to accelerate the matrix multiplication. However, the optimization of neural networks with low-bit activations is challenging due to the emergence of outlier dimensions as the training progresses and the model size grows. Despite these outliers only account for a very small portion of the activations~\cite{llmint8, smoothquant}, they have much larger magnitude, which leads to significant quantization errors and performance degradation on downstream tasks. Previous works~\cite{int4, quarot, spinquant, duquant} mostly utilize Hadamard or learnable rotary transformation to amortize the outlier features into other entries. However, they are mostly designed for the LLMs of higher precision (e.g., 4-bit). For 1-bit LLMs, the extremely low bit-width of the weights makes it challenging to absorb these transformation matrices directly into the weights, while leaving them as online transformations introduces additional computational overhead and limits overall inference performance.
In this work, we introduce \textbf{BitNet a4.8}, a hybrid quantization and sparsification strategy that enables 4-bit activations for 1-bit LLMs. By carefully analyzing the activation distribution of 1-bit LLMs, we selectively apply 4-bit quantization or sparsification based on the distribution patterns of these activations. Specifically, as shown in Figure~\<PIC>, BitNet a4.8 employs 4-bit activations for the inputs to attention and FFN, while utilizing sparsification with 8 bits for intermediate states. To improve the training efficiency, BitNet a4.8 is trained from 8-bit to 4-bit activations with a two-stage recipe, which requires only a few training tokens to adapt \bitnet{} to the low-bit activations at the end of training. Extensive experiments demonstrate that BitNet a4.8 achieves competitive performance to \bitnet{} with the same training cost while being significantly more efficient at inference time. Furthermore, BitNet a4.8 has only 55\% activated parameters and supports 3-bit KV cache, which further enhances the efficiency of LLM deployment.
paragraph{Down projection of FFN.} We compared 1.3B BitNet a4.8 with different quantization or activation function for the down projection of FFN. All models were trained with the first-stage scheduling for 50B tokens from the RedPajama dataset. To ensure a fair comparison, we leave the other activations at 8-bits. We adopt the absmax quantizer for INT8 quantization and MinMax quantizer for FP4 quantization. The $\beta$ of absmean quantizer is set as $2 \text{mean}(|X|)$. Figure~\<PIC> shows the training loss curves of these models. Squared ReLU achieves slightly better training perplexity than Swish while enabling higher sparsity. Furthermore, applying FP4 quantization for the inputs to the down projection leads to a significant performance degradation, while using INT4 activations with STE causes divergence.


{Numerical Results}\label{sec:Numerical Results}
We present here a thorough evaluation of the proposed Steepest Perturbed Gradient Descent (SPGD) algorithm, comparing its performance against several established optimization methods. The comparison includes traditional gradient descent (GD), Perturbed Gradient Descent (PGD), MATLAB \(fmincon\) function which is a versatile solver for constrained optimization problems \cite{Fmincon}, and Simulated Annealing (SA) \cite{SA}.
Our initial analysis is conducted through the lens of four challenging 2D benchmark functions, selected for their known difficulties and relevance in assessing optimization algorithms' efficacy. These test functions are recognized benchmarks within the optimization community, providing a diverse set of landscapes to evaluate each algorithm's ability to navigate complex, non-convex, and potentially deceptive optimization spaces \cite{simulationlib}. For each test function, we apply \(fmincon\), Simulated Annealing, traditional gradient descent, PGD, and SPGD, meticulously recording and analyzing the results. In both the SPGD and PGD algorithms, the amplitude of the perturbation is set to be the same, ensuring comparability, \(Iter_{P}\) is set to \(5\), and \(N_{P}\) is set to \(10\). Furthermore, the gradient function is explicitly passed to the \(fmincon\) function in each scenario to guide the optimization process.
Key performance indicators include the accuracy of the solution, measured by the proximity to the known global optimum\cite{hazewinkel2001theory}; the computational efficiency, quantified by the number of function evaluations and CPU execution time. The SPGD algorithm's source code, along with comparative analyses against the discussed methods using additional 2D challenging test functions, is publicly accessible on GitHub \footnote{Source code and comparisons available at: \url{https://github.com/Amir-M-Vahedi/SPGD-Benchmark-Functions}}.
subsubsection*{Test function 1} The MATLAB Peaks function \cite{Peaksfunction-MATLAB} presents a formidable challenge for optimization algorithms due to its intricate landscape, which features one global minimum, multiple local minima, a saddle point, and extensive flat regions. This complexity makes the Peaks function a critical benchmark for assessing the capabilities of optimization techniques, particularly those based on gradient descent. Traditional gradient descent methods often struggle with such landscapes, as they can easily become trapped in local minima or stall in flat areas, failing to make significant progress towards the global optimum \cite{machines10010042}. The mathematical expression defining the Peak test function is given as follows:
It has a global minimum point located at \(x= 0.2283\), \(y= -1.6256\) with an optimal function value of \(f(x^{*}) = -6.5511\). The initial condition is chosen randomly to be \((-2.81,-1.47)\), and the \(Amp\) is set to 2.5. Figure ~\<PIC> and ~\<PIC>  illustrate the 3D view and top view of optimization trajectory across the Peaks function surface. The convergence history comparison is shown in Fig~\<PIC>. The total number of function evaluations, the converged optimal value, and CPU execution time for different methods are given in Table \ref{tab:Test1 Performance}. Based on the results depicted in Figures ~\<PIC>, ~\<PIC>, and ~\<PIC>, and performance metrics in Table \ref{tab:Test1 Performance}, it is evident that the GD and PGD algorithms become trapped in local minima. In contrast, the \(fmincon\), SA, and SPGD algorithms successfully converge to the global optimum. Among these three, SPGD demonstrates the lowest computational cost. Notably, despite the \(fmincon\) method having fewer function evaluations, its CPU time is approximately 20 times greater than that of the SPGD algorithm.
subsection{Analysis of Scenario 1: Four Identical Cubes}
In Scenario 1, the initial configuration of the four identical cubes was designed to test each algorithm's ability to navigate a relatively simple scenario where the global optimum involves aligning all cubes in a compact configuration. The results of the final configurations found by the GD and SPGD algorithms are illustrated in Figure~\<PIC> and Figure~\<PIC>, showing both Gradient Descent and Steepest Perturbed Gradient Descent results side by side.
he outcomes depicted in the figures reveal that, due to the collision constraint, GD struggled to converge to the global solution and settled in a suboptimal local minimum. In contrast, SPGD successfully converged to the global optimal configuration, effectively avoiding local minima and fulfilling the collision constraints more efficiently. To further illustrate the performance dynamics over the course of the optimization.


{Pseudo-labeling with Self-supervised ViT}
In order to improve OW-DETR, by leveraging pre-trained ViT features, a first step is to replace Resnet activation maps used for pseudo-labeling by DINOv2 \cite{dinov2} activation maps. We call this variant OW-DETR$^{+}$. Note that DINOv2 is only used for the activation map extraction for the unknown pseudo-labeling while ResNet continues to be the backbone of the detection method.
To further improve {OW-DETR$^{+}$}, we propose the novel pseudo-labeling pipeline illustrated in fig. \<PIC>. We first propose to filter out the background. Given an input image passed through the DINOv2 backbone, we obtain a feature map $F \in \mathbb{R}^{H \times W \times d}$, where each pixel is represented by a $d$-dimensional feature vector. Subsequently, we apply DBSCAN \cite{dbscan} with predefined $eps$ and $min\_samples$ on these features since it can automatically assign the number of clusters. This algorithm provides a great ability to separate between the foreground and the background regions. Hence, we filter out the biggest cluster, likely to be the background, especially for OD benchmarks where usually single objects are not omnipresent in the image. Second, we aim to find different objects in the image by localising the regions having similar and homogeneous semantics. However, DBSCAN offers very coarse semantic clusters. Thus, we apply agglomerative clustering on the same features, refine the clusters boundaries and minimize noisy representations using morphological operations such as erosion and dilation. The feature space is then segmented into $AC$ distinct clusters, according to a distance threshold $d_{th}$. This clustering helps in determining coherent object-like structures. Note that each cluster represents potentially a class, as different objects from the same class have usually similar features.
We aim to keep only regions having high objectness by using the DINOv2 activation maps. In fact, $h_a$ attention maps $A_i =\{a_{h,w}, 1 \leq h \leq H,  1 \leq w \leq W \}$ are derived from the ViT backbone, where $h_a$ represents the number of attention heads. Each pixel value $a_{h,w}$ for each attention map is normalized to prevent outlier activations: $a_{h,w} = \frac{a_{h,w}}{\sum_{l=1}^{H} \sum_{k=1}^{W} a_{l,k}}$. These maps are averaged to obtain: $A_{avg} = \frac{1}{h_a} \sum_{i=1}^{h_a} A_i$. Using $A_{avg}$, we compute the average attention activation $A_{ac}$ for each cluster $ac$. Intuitively, higher activations correspond to regions containing objects. Given this insight, we only keep clusters having an average activation greater than the mean average attention of the whole image ($A_{ac}>mean(A_{avg})$). We then search for spatially isolated instances in these clusters and surround the largest $N$ regions with bounding boxes. Finally, we apply Non-Maximum Suppression (NMS) to the generated bounding boxes and filter out boxes that overlap with ground truth boxes ($IoU>T$). Note that in our case, we can have zero unknown object if top activated regions overlap with ground truth objects. We can also have a large number of unknown objects, if different clusters are largely activated. Following the aforementioned method, pseudo-labels for unknown objects are pre-computed in an offline manner for each image of the training dataset. It is important to note that this is a lengthy process. The D-DETR architecture is then trained end-to-end to detect objects from $|\mathcal{K}|+1$ classes by predicting $|\mathcal{K}|+1$ softmax probabilities  ($|\mathcal{K}|$ known class and one unknown class).
{OpenImagesRoad benchmark}
We compare in tab. \ref{tab:openimages} performances of OpenDet \cite{OpenDet}, RandBox \cite{randbox} and OW-DETR \cite{OW-DETR} alongside its different improvements. ORE is unreported as it shows poor performance in unknown detection in tab. \ref{tab:voc_coco}. For fairness, ORE (MAVL \cite{MAVL}) is also unreported as it uses external supervision to extract pseudo-labels.
Interesting performance trends emerge depending on the nature of each method. Indeed, pseudo-labeling methods such as RandBox, OW-DETR,  OW-DETR, OW-DETR$^{+}$ and OW-DETR$^{++}$ all have lower $AP_{u}$ than the contrastive method OpenDet which indicates that contrastive methods seem to learn a broader unknown representation which is good for classification. On the other hand, OW-DETR$^{++}$  have higher $AP_{all}$ and $AP_{sc}$ than OpenDet, demonstrating that this pseudo-labeling based method seems more apt for object localization. The higher $AP_{sc}$ also indicates a good ability of pseudo-labeling methods to understand high granularity classification but at the same time, an inability to understand smaller grained differences between known and unknown as shown by the lower $AP_{u}$. The surprising drop between $AP_{u}$ and $AP_{all}$ for OpenDet can be explained by frequent double detections of unknown objects: as unknown and known objects. Fig. \<PIC> illustrates the behaviours of OpenDet and OW-DETR$^{++}$, further samples of all methods are included in the appendix.
Our proposed pseudo-labeling improvements on OW-DETR, namely OW-DETR$^{+}$ and OW-DETR$^{++}$ bring impressive performance gains. Note that no knowledge on unknown objects is transferred from DINOv2 since it is only used to pseudo-label the training images and that no unknown object is present in these images for the \textit{unseen} scenario. Indeed, OW-DETR$^{++}$ achieves best performance for known OD ($mAP_{k}$), object localization $(AP_{all}$), and high granularity classification unknown OD ($AP_{sc}$). However, despite the drastic performance increase, OpenDet remains better for correctly detecting unknown objects and classifying them as unknown ($AP_{u}$).
To conclude, the proposed OpenImagesRoad benchmark  permitted to shine new light on the detection performance of these methods. It appears that there is not a single best method among those we tested. Rather, the choice of a method can be driven by the envisaged use-case. If unknown objects should be both correctly detected and classified as unknown, the contrastive learning based OpenDet detector appears to be an appropriate choice, especially for such $S_{unseen}$ scenario. Conversely, if the correct localisation of unknown objects is paramount, and coarse classification is sufficient, then OW-DETR$^{++}$ seems best indicated. It is also worth noting that pseudo-labeling techniques are expected to perform better in the $S_{unlabeled}$ scenario where unknown objects are potentially seen and pseudo-labeled during training. Indeed, we remark that in tab. \ref{tab:voc_coco} (VOC-COCO/$S_{unlabeled}$), OW-DETR$^{++}$ seems to be relatively more preferment compared to Opendet, than in tab. \ref{tab:openimages} (OpenImagesRoad/$S_{unseen}$).


{Introduction}
{I think this paper will benefit from more motivation provided in this section. Much more could be said why such a visualisation is important providing specific examples from different applications}
High-throughput experiments are becoming extremely common in applied sciences.
Now more than ever, large high-dimensional datasets are generated in almost every laboratory calling for an automated and reliable way to extract new knowledge from them.
Let us fix a dataset $X$, usually embedded in a high dimensional space.
Standard dimension reduction techniques, including PCA~\cite{pca}, t-SNE~\cite{tsne}, \textsc{UMAP}~\cite{umap} and \textsc{PHATE}~\cite{phate} aim to find a low dimensional embedding of $X$ so that points that are close in $X$, are also close in the embedding.
However, preservation of the global organization of $X$ in general, and information about distances of far away points in particular, is a challenge for this methods.
Clustering techniques~\cite{statLearning, SAXENA2017664} on the other hand, based on a fixed similarity measure, provide a partition of the input dataset $X$.
However, clustering itself does not provide information about either intra-- or inter-- cluster organization of points, therefore is not used to asses the global structure of the data.
The aim of this work is two bridge this two approaches by enriching the output of a clustering algorithm with additional information on the data's global organization.
The first contribution of this paper is the construction of a \emph{ClusterGraph}; a graph--based structure on a top of a partition $\mathcal{C}(X)$ of the data obtained from a clustering algorithm $\mathcal{C}$ applied to $X$.
In the ClusterGraph $G = (V,E)$, each vertex corresponds to a single cluster from $\mathcal{C}(X)$.
Two vertices $u,v \in V$ are connected by an edge whose length corresponds to the distance between their respective clusters in $\mathcal{C}(X)$.
For the purpose of this construction, a number of inter--cluster distances defined in the ambient space are considered.
ClusterGraph has a number of advantages compared to alternative dimension reduction methods.
One of them is based on the fact that the distances, computed in the ambient space, are represented by labels on edges and not subjected to distortions made by standard dimension reduction techniques that force the projected data points to be embedded in an Euclidean space.
This  allows to visualize the \emph{global} distances in the dataset.
This is important as many datasets cannot be embedded into low dimensional Euclidean spaces without perturbing the distances between points.
As an example of such a situation consider a collection of points in four clusters: $0$, $1$, $2$ and $3$. Points in each cluster are infinitesimally close. Distance between cluster $0$ and clusters $1$, $2$ and $3$ are $1$, while distance between  clusters $1$, $2$ and $3$ is $2$.
It is well known~\cite{morgan_embedding_1974, bourgain_lipschitz_1985}
that such a graph cannot be isometrically embedded to any Euclidean space $\mathbb{R}^n$ for any $n$.
As a consequence, all dimension reduction techniques will distort the distances between clusters, as can be observed in Fig~\<PIC>. ClusterGraph, on the contrary, provides the correct graph even in this case.
{Metric distortion}
Given a dataset $X$, different choices of the clustering algorithm $\mathcal{C}$, as well as the metrics $d_X$ and $d_\mathcal{C}$ can lead to significantly different ClusterGraphs. The aim of this section is to introduce a score to assess the quality of a given
ClusterGraph $G$ by comparing it to the underlying geometric structure of the dataset $X$.
For this purpose let us assume that the considered point cloud $X$ is sampled from a compact and connected manifold $\mathcal{M}$ equipped with an  \emph{intrinsic distance} $d_{\mathcal{M}}$. Informally, the intrinsic distance between two points $x,y \in \mathcal{M}$ is  defined to be the infimum of the length of a curve $\gamma \subset \mathcal{M}$ joining $x$ and $y$, this is also known as \emph{geodesic} distance.
In most applications, the underlying manifold is not known.
Consequently, the intrinsic distance needs to be estimated from the point cloud. This is a well-studied problem in computational geometry and computer graphics, and multiple methods have been proposed~\cite{isomap, klein_point_2004, ruggeri_approximating_2006,yu_geodesics_2014}.
Below, we follow the approach of~\cite{isomap, Bernstein2000GraphAT} using the shortest path in the $k$-nearest neighbor graph as estimator.
Note that any other estimator of intrinsic distance can be also used in the proposed construction.
Let $G_{knn}(X)$ be the $k$-nearest neighbor graph on $X$ constructed in the following way: each point of $X$ corresponds to a vertex of $G_{knn}(X)$; it is connected to its $k$-nearest neighbors (in the chosen distance $d_X$, typically Euclidean), with $k$ being a parameter of the method. Weights corresponding to the distance between endpoints are assigned to the edges of $G_{knn}(X)$. We define a distance $d_X^k$ on $G_{knn}(X)$, estimating the intrinsic distance on $X$, as
    It may happen that $G_{knn}(X)$ is not connected.
        There are two possible reason for this, depicted in Fig.~\<PIC>.
        There are two possible reason for this.
    In the first case  points are indeed sampled from a compact and connected manifold but the parameter $k$ is too low. This can be easily solved by increasing $k$. In the second case the underlying manifold is not connected. This will result in the $k$-nn graph being disconnected even for very high values of $k$, especially if many points are sampled. In this case, we will threat each connected component separately, splitting the input dataset $X$ (and the output of the clustering algorithm) into disjoint sets, each one corresponding to a different connected component and analyze each of them separately\footnote{By performing the construction in Sec.~\ref{sec:cg} we obtain a ClusterGraph for each connected component, each of them being a fully connected graph. In graph theory such a disjoint union of complete graphs is sometimes called a \say{cluster graph}. This unexpected but pleasing agreement in nomenclature motivates our choice of referring to our construction in camel case, to avoid confusion.}. For the rest of the Section we  therefore assume, without lack of generality, that the $k$-nn graph is fully connected. We discuss how to investigate the relations between different connected components in Section~\ref{sec:connectivity}.\\
    Whenever an estimator is used, it is natural to ask how good such estimator is. The choice of a $k$-nn graph as an estimator of the geodesic distance is motivated by the following theorem by Bernstein,  Vin de Silva, Langford and Tenenbaum.
Theorem A in~\cite{Bernstein2000GraphAT}]
        Let $\mathcal{M}$ be a compact submanifold of $\mathbb{R}^n$, $X$ a finite set of data points in $\mathcal{M}$ and $G$ a graph on $X$ (for example, a $k$-nn graph). Then the inequalities
            (1-\lambda_1) d_\mathcal{M}(x, y) \leq d_G(x,y) \leq (1+\lambda_2) d_\mathcal{M}(x, y)
        are valid for all $x,y$ in $X$, where $\lambda_1, \lambda_2 < 1$ are two positive real numbers that depends on $G$, $\mathcal{M}$ and some technical assumptions on the density of $X$.
For each point $x \in X$, we denote with $C_x$ the cluster in $\mathcal{C}(X)$ that contains $x$. We can then use the distance between clusters described in \ref{sec:distances_clusters} to define a distance between points $d_{CG}$ in the ClusterGraph $G$ as following
Recall that $G$ is fully connected; one could wonder why the length of the shortest path between two vertices is used instead of the weight of the edge connecting them given by $d_\mathcal{C}$. First, the triangle inequality might not hold for $d_\mathcal{C}$. Secondly, we want the definition of $d_{CG}$ to hold also in the case of a pruned ClusterGraph, which we will discuss in Section~\ref{sec:pruning}.
This notion of $d_{CG}$ is well defined only when $\mathcal{C}(X)$ is a partition. In the more general case of a division, when a point can belong to more than one cluster, we take $d_{CG}(x,y)$ to be the length of the shortest path between any cluster containing $x$ and any cluster containing $y$.


The main issues that may easily lead to smart contract vulnerabilities are twofold. First, the programming languages and tools are still new and crude, which leaves plenty of rooms for misunderstandings in the built-in functions and tools [Luu et al., 2016]. Second, due to the immutable nature of smart contracts, developers are required to anticipate all possible states (e.g., stack status) and environments that the code may encounter in the future, which is obviously difficult. Existing methods on smart contract vulnerability detection can be roughly cast into two categories. The first line of work [Luu et al., 2016; Tsankov et al., 2018; Jiang et al., 2018] utilized classical static analysis and dynamic execution techniques to identify vulnerabilities. Unfortunately, they fundamentally rely on several fixed expert rules, while the manually defined patterns bear the inherent risk of being errorprone and some complex patterns are non-trivial to be covered. Meanwhile, crafty attackers may easily bypass the fixed patterns using small tricks. Another line of work [Tann et al., 2018; Zhuang et al., 2020] explored using deep learning models to deal with complex contract data, having achieved much improved accuracy. Due to the black-box nature, however, they fail to encode useful expert knowledge and mostly have poor interpretability. This motivates us to consider whether we could combine neural networks with classical expert patterns, where neural networks contribute their ability to handle the complex code semantic graph while expert patterns contribute precise and valuable local information. More importantly, we seek an explainable solution which could tell the weights of different features. In this paper, we propose a new system beyond pure neural networks that can automatically detect vulnerabilities and incorporate expert patterns into networks in an explainable fashion. In particular, (1) we develop automatic tools to extract vulnerability-specific expert patterns. (2) Then, we exploit a graph structure to frame the rich control-flow and data-flow semantics of the function code. Upon the graph, a graph neural network is employed to extract the deep graph feature. (3) Finally, we propose an attentive multi-encoder network to interpretably fuse the global graph feature and local expert patterns. Extensive experiments are conducted on all the 40k contracts in two benchmark datasets, demonstrating significant improvements over state-of-the-art: accuracy from 84% to 90%, 83% to 87%, 75% to 80% on three types of vulnerabilities respectively. More importantly, our model is able to explain its label prediction, give warnings of high weighted local patterns, and provide a grand picture of the significance of different features. <PIC>: The attentive multi-encoder network, consisting of a self-attention mechanism and a cross-attention mechanism. It combines local pattern features and the global graph feature for vulnerability detection, and outputs interpretable weights for all features. Contributions. The key contributions of this work are: 1) We investigate combining vulnerability-specific expert patterns with neural networks in an explainable way. To the best of our knowledge, we are the first to prob the combination interpretably. 2) In the method, we present a simple but effective multi-encoder network for feature fusion. 3) Our method sets the new state-of-the-art and provides novel insights. To facilitate future research, our implementations are released at https://github.com/Messi-Q/AMEVulDetector. We would like to point out that different from [Liu et al., 2021], this work focuses mainly on the explainability of the expert pattern and deep graph feature combination, and offers a grand picture on the importance of different features. The overview of the proposed system is illustrated in <PIC> which consists of three components: 1) a local expert pattern extraction tool, which extracts expert patterns of a specific vulnerability from the function code; 2) a graph construction and normalization module, which transforms the function code into a code semantic graph; and 3) an attentive multi-encoder network that combines local expert patterns and the global graph feature for vulnerability detection and outputs explainable weights. In what follows, we introduce the three components one by one. 3.1 Local Expert Pattern Extraction
Following [Liu et al., 2021], we design corresponding expert patterns for three types of vulnerabilities respectively. Then, we implement a fully automatic tool to extract expert patterns from the function code. Specifically, the patterns for different vulnerabilities are defined as: Reentrancy. Technically, the reentrancy vulnerability occurs when a call.value invocation (i.e., a built-in money transfer function) can call back to itself through a chain of calls.
That is, call.value is successfully re-entered to perform unexpected repeat money transfers. For the reentrancy vulnerability, we design three local patterns. (1) enoughBalance concerns whether there is a check on the sufficiency of the user balance before transferring to a user. (2) callValueInvocation models whether there exists an invocation to call.value in the function. (3) balanceDeduction checks whether the user balance is deducted after money transfer, which considers the fact that the money stealing can be avoided if the user balance is deducted each time before money transfer. Block timestamp dependence. Generally, the timestamp dependency vulnerability exists when a smart contract is conventionally considered as using block.timestamp as part of the conditions to perform critical operations [Jiang et al., 2018]. We design three local patterns for the timestamp dependence vulnerability. (1) timestampInvocation models whether there exists an invocation to opcode block.timestamp in the function. (2) timestampAssign checks whether the value of block.timestamp is assigned to other variables or passed to a function as a parameter, namely whether block.timestamp is actually used. (3) timestampContaminate verifies if block.timestamp may contaminate the triggering condition of a critical operation (e.g., money transfer). 3.2 Graph Construction and Normalization
One may directly use combinations of aforementioned patterns to predict whether the function has a certain vulnerability. However, these fixed patterns are shown to have difficulties in handling relatively complex attacks and are trivial to be bypassed by adversaries [Qian et al., 2020]. Therefore, we further propose to model the control flow and data flow of the entire source code into a semantic graph, and adopt graph neural networks to handle it. Thereafter, the extracted global graph feature and the local expert patterns could supplement each other towards a more precise and explainable label prediction. Graph construction. Different program elements in a function are not of equal importance in detecting vulnerabilities. Therefore, we extract two categories of nodes, i.e., core nodes and normal nodes. (1) Core nodes symbolize key invocations and variables in the function code, which are critical for detecting a specific vulnerability. For instance, for reentrancy vulnerability, (i) an invocation to call.value, (ii) an invocation to a money transfer function that contains a call.value invocation, and (iii) the variable that corresponds to user balance, are treated as core nodes. We summarize
the core nodes for detecting the three vulnerabilities in Table 1. (2) Invocations and variables that are not extracted as core nodes are modeled as normal nodes, which play an auxiliary role in detecting vulnerabilities. We also construct an extra fallback node to stimulate the fallback function of a virtual attack contract. The fallback node can interact with the function under test and is considered as a normal node. It is worth mentioning that distinct from prior works such as
[Zhuang et al., 2020], which merely model key invocations as core nodes, we propose to further extract key variables as core nodes, given that they are undoubtedly important in detecting
vulnerabilities. To characterize rich connections between different nodes, we construct three categories of semantic edges, namely control flow, data flow, and fallback edges. Graph normalization. It is worth mentioning that different functions corresponding to distinct code semantic graphs, bringing difficulties in training a graph neural network. Moreover, current graph neural networks are inherently flat when propagating information, ignoring that different nodes are not of equal importance. Therefore, we propose to normalize the graph following that of [Zhuang et al., 2020] to remove all normal nodes and merge their features to the nearest core nodes. A simplified example for graph construction and normalization is given in Fig. 1(b).


As motivated in Sec. I, iAudit employs a novel two-stage fine-tuning approach and combines it with LLM-based agents for intuitive smart contract auditing with justifications. As shown in  <PIC>, iAudit has the following four roles:
• Detector is the key component for achieving intuitive smart contract auditing. By fine-tuning an LLM with vulnerable and non-vulnerable code, Detector can discern whether a piece of code is vulnerable, much like how a human hacker perceives a potential vulnerability.
• Reasoner takes the initial vulnerability perception from Detector to further analyze the potential causes of the vulnerability based on Detector’s decision. By connecting Detector’s output with Reasoner’s reasoning during both training and inference, iAudit achieves two-stage fine-tuning.
• To identify the optimal cause of a vulnerability during the inference phase, we further introduce the concept of LLM-based agents into the fine-tuning paradigm in iAu dit. Specifically, Ranker evaluates the reasons for each potential vulnerability, selecting a top explanation, while Critic further assesses Ranker’s output to debate and determine the most appropriate cause of the vulnerability.  Challenges. While iAudit’s four roles in  <PIC> are intuitive,
training and coordinating them well for effective smart con tract auditing with reasonable justifications is difficult. More specifically, we encountered the following four challenges during the design and implementation of iAudit: C1: How to collect and derive high-quality training data?
For a fine-tuned model like iAudit, obtaining high-quality training data is always crucial. We propose leveraging reputable auditing reports to collect posi tive samples and employing our own data enhance ment method to derive negative samples. Since this part is independent of iAudit’s design, we defer its presentation to the end of this section in Sec. III-D.
C2: How to make effective vulnerability judgements? While fine-tuning a model with vulnerable and non-vulnerable code is straightforward, tuning it to be effective with limited data presents a challenge. We make an effort towards addressing this problem in Sec. III-A by opting to use multiple prompts for fine-tuning rather than a single prompt. The advantages of this approach are twofold: (i) it enriches the training dataset by increasing the volume of data, and (ii) it diminishes the bias associated with a single prompt, thereby enhancing the reliability of the results [48]. Optimal vulnerability perception
could thus be achieved through majority voting.
C3: How to effectively connect Detector’s vulnerabil ity sensing with Reasoner’s vulnerability reason ing? The fine-tuning of iAudit is unique because it
employs a two-stage fine-tuning approach with the Detector and Reasoner models. Therefore, how to effectively connect these two models becomes a new issue not encountered in traditional fine-tuning. We present this aspect of iAudit’s design in Sec. III-B.
C4: How to obtain the optimal vulnerability cause from Reasoner’s output? Since Reasoner also employs multi-prompt fine-tuning, it is necessary to identify the optimal cause of vulnerability among the multiple causes output by Reasoner. We introduce two LLM-based agents, namely the Ranker and Critic compo nents, in Sec. III-C, to iteratively select and debate the most appropriate cause of vulnerability. An Example of Workflow. To wrap up,  <PIC> also illustrates an example of iAudit’s workflow. Initially, Detector perceives code vulnerabilities using five different inference paths (prompts). The perceived results are then subjected to majority voting to determine a consensus label. Based on the voting result, Reasoner interprets this outcome according to different inference paths, resulting in ten answers (each considering the context of the code location or not). Next, Ranker selects Reason 1 with a confidence score 9/10 and explains this choice. Critic challenges this choice and advises Ranker to re-evaluate. Taking Critic’s feedback into account, Ranker re-ranks the ten reasons and selects Reason 3 with a confidence score of 10/10. Critic reviews Ranker’s choice again and agrees with this decision. The loop is completed, and the final reason is returned to the user. As shown in <PIC>, iAudit achieves the highest scores of F1, Recall, and Accuracy among all methods, 0.9121, 0.8934 and 0.9111. CodeLlama-13b-class is second only to our method regarding vulnerability detection rate, and the performance is relatively close. GraphCodeBERT and UnixCoder perform worse than CodeLlama-13b-class. Although CodeT5 achieves the highest precision at 0.9431, its other metrics are lower than GraphCodeBERT and UnixCoder. CodeBERT has the worst performance. Additionally, the accuracy scores of these models are relatively high (all are more than 0.91), indicating that many of the predicted risky vulnerabilities are indeed risky. The results, as shown in <PIC>, indicate that our step-wise approach outperforms the integration models across four key performance metrics. While the three methods are similar in precision, the differences in other metrics are notable. Analyzing these results, we found that the integration methods have higher accuracy for negative samples but a lower detection rate for positive samples (i.e., lower recall). This may be attributed to the generative loss optimization, where the output sequence is longer, making the label-related loss occupy a smaller proportion of the total loss, thus preventing the model from adequately focusing on the label. To test this hypothesis, we added data that includes only label generation to the dataset during the integration training process, guiding the model to focus more on the label. In the evaluation phase, we still required the model to output both labels and explanations simultaneously. Through this mixed training approach, we observed a significant improvement in the model’s vulnerability detection performance, with an F1 score of 0.8433, a recall rate of 0.8164, a precision of 0.8723, and an accuracy of 0.8434. Answer for RQ3: iAudit achieved better detection performance than the integration model that outputs labels and reasons simultaneously. We confirmed that the model struggles to focus on the labels when required to output both types of information, as evidenced by our inclusion of label-only data in the verification process. F. RQ4 - Effectiveness of Majority Voting Our research explored a method using multiple prompts and a voting mechanism for Detector to determine the final label. This method aims to enhance the model’s precision and credibility. During the evaluation process, we continued to use metrics such as the F1 score, recall, precision, and accuracy. We calculated these metrics for each prompt individually for comparative analysis, as shown in <PIC>. It should be noted that the first row Single-prompt indicates that we used only one prompt format to train Detector. Prompt-1, Prompt-2, Prompt-3, Prompt-4, and Prompt-5 represent the results for each
prompt after multiple-prompt training. The last row shows the results after majority voting, indicating that majority voting can improve the overall performance of iAudit, with both the F1 score and accuracy being the highest. At the same time, except for Single-prompt, we noticed minimal performance differences among multiple prompts. Single-prompt performed much worse than the others. Training with multiple prompts can improve model performance compared to using only one prompt during training. Additionally, we divided the test set into two groups based on whether  the predictions were correct or incorrect, named 'correct prediction'  and 'incorrect prediction' groups, respectively, and analyzed the  distribution of confidence scores within these two groups. We found that in the incorrect prediction group, the proportion of confidence scores  within the range of 0.6 to 0.8 is significantly higher than in the  correct prediction group (11% vs 2%, 10% vs 3%, respectively), as shown  in <PIC>. The confidence score can reflect the reliability of the  prediction results to a certain extent. When the confidence score is  low, the prediction results are less credible.


<PIC>: The overall architecture of our proposed GraphGPT with graph instruction tuning paradigm. Contributions. To address these challenges, we propose a novel framework called GraphGPT, which aims to align Large Language Models (LLMs) with Graphs using a carefully designed graph instruction tuning paradigm. (C1) Our framework introduces a textgraph grounding paradigm as the initial step to align the encoding of graph structures with the natural language space. By incorporating textual information in a contrastive manner, we enable effective alignment of graph structure information within language models. (C2) In our proposed dual-stage graph instruction tuning paradigm, we leverage self-supervised signals through the graph matching task, which is derived from unlabeled graph structures, to serve as instructions for guiding model tuning of LLMs. By incorporating this self-supervised instruction tuning, the language model acquires domain-specific structural knowledge related to graphs, thereby enhancing its understanding of graph structures. To further customize the LLM’s reasoning behavior for diverse downstream graph learning tasks, the second stage of our graph instruction tuning paradigm involves fine-tuning the LLM with task-specific graph instructions, to improve the model’s adaptability. (C3) By incorporating the Chain-of-Thought (COT) distillation into our framework, GraphGPT enhances its step-by-step reasoning abilities and improves its performance in the face of distribution shift. In summary, our work makes the following contributions: • This work aims to align graph domain-specific structural knowledge with the reasoning ability of Large Language Models (LLMs) to improve the generalization of graph learning. • Our approach aims to align LLMs with Graphs through a graph instruction tuning paradigm. This paradigm incorporates selfsupervised instruction tuning, enhancing the LLM’s comprehension of graph structural knowledge and its reasoning capabilities. Additionally, we introduce task-specific instruction tuning to improve the model’s adaptability across diverse graph tasks. • We evaluate our proposed GraphGPT on supervised and zeroshot graph learning tasks. We conduct thorough analyses of its component-wise effects and generalization ability. By comparing it with state-of-the-art baselines, we demonstrate the superior generalization power of our approach across various settings.  To improve the understanding of graph structural information by large language models, our framework focuses on aligning the encoding of graph structures with the natural language space. This alignment enables language models to effectively comprehend the graph’s structural elements using their language understanding capabilities. To achieve this, we introduce a text-graph grounding paradigm that generates prompts preserving the graph’s structural context for language models. This paradigm acts as a bridge, connecting the semantic understanding of textual information with the inherent structural relationships in the graph. In our GraphGPT, we design the graph encoder to be highly flexible, allowing it to leverage a wide range of backbone GNN architectures obtained from diverse graph pre-training paradigms. We incorporate a message-passing neural network architecture, which can be a graph transformer [60] or a graph convolutional network [17], as the structure-level pre-trained graph model. In each message-passing step, the graph encoder aggregates information from neighboring nodes, considering their relationships:  Text-Structure Alignment. To enhance the alignment of graph structure information with Language Models (LLMs), our focus is on exploring effective encoding methods that can collaborate seamlessly with LLMs. Building upon previous works [30, 49], we adopt a contrastive approach by incorporating textual information into the graph structure encoding process. We directly integrate a pre-trained graph encoder into our GraphGPT framework, enabling the seamless utilization of its capabilities. <PIC>: Workflow of text-structure alignment.  The dual-stage graph instruction tuning paradigm proposed in this work builds upon the concept of instruction tuning, which has been recently introduced to enhance the adaptability of language models for specific domains [45]. In this paradigm, we aim to align the language capacity of the model with the nuances of graph learning tasks, enabling the language model to generate more accurate and contextually appropriate responses for graph-structured data.  Self-Supervised Instruction Tuning. In the initial stage of our graph instruction tuning, we introduce self-supervised instruction tuning. This mechanism enhances the language model’s reasoning abilities by incorporating graph domain-specific structural knowledge and effectively understanding contextual information within the graph’s structure. To achieve this, we utilize self-supervised signals derived from unlabeled graph structures as instructions for model tuning. Specifically, we design a structureaware graph matching task that guides the language model in differentiating between graph tokens using language tokens. This instruction task plays a vital role in accurately associating graph tokens with their corresponding textual descriptions, deepening the model’s comprehension of the graph with the provided guidance.  Obs.2: Benefits with Structure-aware Graph Matching. The presence of the first stage, which involves self-supervised graph matching tasks for instruction tuning, plays a crucial role in enhancing the zero-shot transferability of our GraphGPT. The first stage focuses on aligning the graph tokens, which encode rich structural information, with the language tokens. This alignment enables the model to develop a deeper understanding of the inherent structural characteristics of the graph data. Without the first stage, if we only conduct the second stage of task-specific instruction tuning, the model tends to be more prone to overfitting on the specific dataset. In such cases, the model’s performance may be heavily reliant on dataset-specific patterns and characteristics, rather than a genuine understanding of the underlying graph structure. This can limit the model’s ability to generalize to new, unseen datasets.  More Data Boost Model Transfer Ability. In our preliminary investigation, we examine the influence of data quantity on the transfer capability of our GraphGPT, as illustrated in the "(Arxiv + PubMed)-Cora" column of Table 1. In this experiment, we train models using a combination of the Arxiv and PubMed datasets and perform zero-shot testing on the Cora dataset. The results reveal that by incorporating a relatively smaller PubMed dataset (with 20,000+ items) alongside Arxiv, our GraphGPT exhibits a significant improvement in transfer performance on Cora. In contrast, the transfer performance of GNN-based models, trained separately on Arxiv and PubMed, actually deteriorates.  More Data Yet No Forgetting. We further validate the performance of the combined Arxiv and PubMed instruction data on the original Arxiv data, as demonstrated in the "(Arxiv + PubMed)- Arxiv" column in Table 1. The results indicate that most traditional GNN-based approaches experience a significant decline in performance on Arxiv after iterative training. In contrast, our model exhibits improved performance. We attribute this phenomenon to the occurrence of catastrophic forgetting in GNN-based models, where the structural modeling competence of the model trained solely on the smaller PubMed dataset is compromised. However, our model effectively mitigates this issue through our unified graph instruction tuning paradigm. This enables our model to maintain and even enhance its performance by retaining the generalized graph structure patterns despite incorporating additional data.


For the CL module, we incorporate correlation labels to guide the training process. These labels are constructed based on the relationships between sampled contracts. Therefore, employing a suitable sampling strategy is crucial as it can greatly enhance the performance by ensuring the utilization of high-quality sample pairs for training, while minimizing the introduction of bias into the learning process. Motivated by this, our sampling strategy is as follows: There are three types of relationships for contract pairs, i.e. "V-V", "N-N", and "V-N", where V and N denote Vulnerable and Non-vulnerable contracts, respectively. Our intuition is that the relationships of "V-V" and "V-N" are more important, because we would like to discover the commonality in "V-V" and differences in "V-N" by CL. In contrast, the "N-N" is not substantially helpful in identifying SCV. Therefore, our sampling strategy is to extract all vulnerable contracts  from the original dataset and create a new set called the POS set. Then, for each contract in the original dataset, we randomly select a  contract from the POS set to form a pair of contracts as input for the  CL model. It should be noted that this sampling strategy does not have  'N-N' relationship. Finally, the correlation labels LCL of the contract  pairs are constructed to guide the training of the CL model. <PIC>: Overview of Clear, which encompasses both the CL process, depicted by solid lines indicating the data flow, and the subsequent vulnerability detection process, represented by dotted arrows indicating the data flow. Finally, in Table 1, line 13 reports the performance of the state-of-the-art general method, LineVul, which uses CodeBERT to detect vulnerabilities. The quantitative results suggest that simple migration of general methods to the SCVD field may not yield satisfactory results. Even the LineVul, which performs the best among the general methods, only achieves an average precision, recall and F1-score of 82.92%, 71.56%, and 76.57% for the three vulnerability detection scenarios, respectively. In comparison, Clear outperforms LineVul across all three metrics, surpassing LineVul by more than
12.93%, 33.37%, and 23.44%, respectively. Answer to RQ1: The proposed Clear outperforms the state-of-the-art methods across all metrics. On average, Clear achieves an F1-score of 94.52%, showcasing a 9.73% increase in F1-score compared to the existing best-performing method. 5.2 RQ2: Impact of Different Modules
To answer RQ2, we conduct comprehensive ablation tests to examine and understand the impact of different modules on Clear’s overall effectiveness. In Section 3.1, we have described that Clear consists of three stages, and therefore, we have specifically designed distinct ablation tests for each of these stages. The results of all ablation tests are presented in Table 2, in which the metrics P, R, and F represent precision, recall, and F1-score, respectively. To begin with, for stage 1, we focus our data sampling strategy on two specific types of contract relationships, namely "V-V" and "V-N". We selectively mask these relationships in order to evaluate the influence of the labels generated by these two relationships on the overall effectiveness of vulnerability detection. The "Clear-MVV" indicates the masked "V-V" relationship and "Clear-MVN" indicates the masked "V-N". As shown in Table 2, Clear-MVN and ClearMVV achieve 81.18% and 84.76% average F1-score, respectively. In comparison, Clear outperforms both of them and has an F1-score of 94.52%. That is to say, learning only one of the contract relations within the CL stage does not yield satisfactory results. It is only by simultaneously learning both relations that we observe a significant improvement in the performance of SCVD. Moving on to stage 2, we intentionally remove the MLM module
that is integrated into the CL stage. This allows us to analyze the overall effectiveness of the CL stage without the presence of the MLM module. This particular test is referred to as "Clear-RMLM". We observe that the MLM module has a substantial impact on the effectiveness of the CL module. Specifically, when the MLM module is removed, there is an average decrease in precision, recall, and F1-score by 4.15%, 6.13%, and 5.14% respectively. Therefore, we believe that the MLM module can enhance the performance of Clear and is an essential component. Lastly, for stage 3, we remove the CL stage altogether and directly performed the vulnerability detection stage. This test, known as "Clear-RCL", enables us to evaluate the performance of vulnerability detection in the absence of the CL stage. In comparison to Clear-RCL, we observe a significant improvement in performance for all
three types of vulnerability detection tasks with the addition of the CL module. The F-score increased by 35.47% for RE, 34.21% for TD, and 19.00% for IO. This notable improvement can be attributed to the synergistic effects of the CL stage itself and our unique sampling strategy. Specifically, the CL module facilitates the convergence of dispersed vulnerability samples in the feature space, resulting in increased proximity among them. By utilizing our unique sampling strategy, we further reinforce the correlation among samples belonging to the same vulnerability category, thereby promoting their clustering behavior. This process enables the model to more effortlessly identify and discover potential SCVs, leading to a significant improvement in the performance of SCVD. To substantiate our assertion, we thoroughly examine the derivation process of the sample distribution during the CL stage. In particular, we analyze the evolution of the output of the CL stage (denoted as   in Eq. 7) at each epoch and employ principal component analysis [29] to project each output onto a two-dimensional space. Subsequently, these outputs are visualized as scatter plots and displayed in <PIC>, where the horizontal and vertical axes represent linear combinations of the vectors v obtained through PCA. Each point denotes a contract sample, with purple indicating vulnerability samples and yellow representing non-vulnerability samples. The figure clearly depicts the progression of smart contract sample distribution throughout the CL stage and yields the following finding. First, during the training process of the CL module, the samples of  vulnerability contracts exhibit a tendency to cluster together, while  being distinctly separated from non-vulnerability samples, indicating a  clear distinction between the two groups. Second, this distribution  enhances the ability to differentiate and detect SCVs. Clear exhibits a  higher proficiency in recognizing this particular cluster and accurately classifying contracts within its proximity as vulnerable. This leads to an improved capability for identifying SCVs.


With the continuous growth of bitcoin addresses, an accurate yet efficient method for address classification is much coveted. Existing methods for bitcoin address classification can be roughly cast into two categories, i.e., off-chain informationbased method and on-chain behavior-based method. The offchain information-based method focuses on tagging addresses by gathering real-world data from address users. For example, Ermilov et al. [18] crawl the user profiles from relevant forums (e.g., Bitcointalk.com, Twitter, and Reddit) and darknet markets (e.g., Silkroad, Hub Marketplace, and Alphabay) to obtain the association between some bitcoin addresses and users, and then analyze the behavior of other addresses using a clustering method. Kang et al. [19] obtain the IP address by receiving bitcoin protocol message packets, and then use static analysis to infer other addresses which the current IP address may be associated with, realizing the mapping between bitcoin addresses and IP addresses. Unfortunately, these solutions are highly dependent on off-chain collected information and human experience, and hence cannot be used for all bitcoin addresses. Additionally, there are usually many mistakes in off-chain information, which might lead to low accuracy of the address behavior analysis. The on-chain behavior-based method concentrates on extracting transaction characteristics of bitcoin addresses and analyzing their behavior with the guidance of machine learning. For instance, [20]–[25] directly extract transaction features from bitcoin addresses and then feed them into models, e.g., Random Forest, SVM, and LightGBM. Such methods do not need to rely on off-chain information, thus avoiding potential problems caused by missing off-chain information. However, they also suffer from inherent drawbacks. On one hand, direct feature extraction from transactions in the addresses may lead to large deviations. On the other hand, the traditional classification model has difficulties in utilizing the temporal feature and topology of transactions in the addresses. These problems impose a significant impact on the accuracy of address behavior analysis, which motivates us to design a novel and effective address behavior analysis model. In this paper, we present BAClassifier, a fully automatic framework for bitcoin address classification. In particular, BAClassifier consists of three key components. (1) Address Graph Construction. For each given bitcoin address, BAClassifier constructs a chronological transaction graph that reflects the behavior of that address. Specifically, we engage in a graph node compression technique and a graph structure augmentation method to transfer the transactions of each bitcoin address into a unified graph representation. (2) Graph Representation Learning. BAClassifier utilizes graph neural networks to learn the graph representation of each address and generate the graph embeddings. (3) Address Classification. BAClassifier aggregates all graph embeddings of each address into the address-level representations, and engages in a classification model to give the final address behavior classification. • We propose BAClassifier, a tool that can automatically classify bitcoin address behaviors. Particularly, we investigate a scalable and generic manner of analyzing bitcoin address behaviors using graph neural networks. • Within this system, we come up with three key components, i.e., address graph construction, graph representation learning, and address classification. Specifically, we introduce graph node compression and graph structure augmentation techniques to translate bitcoin address transactions into unified graph representations. Furthermore, we adopt graph feature networks to extract address graph features and select the combination of LSTM+MLP as the behavior classification model. • We construct a large-scale labeled dataset that consists of over 2 million bitcoin addresses as well as their transactions and concerns 4 types of address behaviors, which can serve as a benchmark for evaluating bitcoin address classification methods. • Extensive experiments on the collected dataset show that our proposed system is indeed useful in identifying bitcoin address behaviors. BAClassifier surpasses stateof-the-art address classifiers and overall provides interesting insights. We have released our code and dataset at https://github.com/AwesomeHuang/BAClassifier, hoping to facilitate future research. Method Overview.  The overall architecture of BAClassifier is outlined in <PIC>. Generally, BAClassifier consists of three key components:  • Address Graph Construction: For each given bitcoin address, BAClassifier will construct a chronological transaction graph that reflects the behavior of that address.  • Graph Representation Learning: BAClassifier utilizes a graph neural network to learn the graph representation of each address and generate the graph embeddings.  • Address Classification: BAClassifier aggregates all graph embeddings to produce the address-level representations, and confirms the classification model to output the predictions of address classifications. In what follows, we will elaborate on the details of these components one by one. The first step in BAClassifier is to transfer the address transactions into graph structures. To obtain a unified address graph, we have to solve three key problems. (1) Different bitcoin addresses have a distinct number of transactions, thus yielding different sizes of graphs. Moreover, the transactions of one address are performed in temporal order. Therefore, we must guarantee that the generated graphs have a unified structure while still preserving the temporal order of transactions. (2) There is a significant disparity in the number of addresses involved in various transactions. For instance, a transaction issued by certain exchanges may only have several associated addresses, while a transaction generated by a mining pool may have thousands of associated addresses. Hence, the graph size must be limited. (3) Since bitcoin transactions provide an insufficient amount of information, we thus need to go deeper into the global graph structural feature to elicit further information. To tackle the above problems, BAClassifier incorporates three key modules into the address graph construction, namely original graph extraction, graph node compression, and graph structure augmentation. When a transaction occurs, the bitcoin wallet will zero off the balance in the original address, and transfer any leftover funds to a new address. Naturally, the address that receives the change could be set to the original address of the originating transaction. Then, the bitcoin wallet automatically generates a new address to receive the change after the transferred amount and the fee are deducted. Such a mechanism safeguards the user’s privacy by ensuring that no one other than the user could know which address is the change address and which is the receipt address. However, all these facts make the analysis of bitcoin address behavior more difficult and challenging than that of traditional account models. Single-Transaction Address Compression. The single-transaction address  compression is proposed to merge the address nodes that have only a  single transaction, reducing the number of address nodes in the graph.  <PIC> illustrates the specific procedure. For the single-transaction  address connected to the same transaction node, we merge these nodes  into a hyper node, termed a single-transaction hyper node. To retain the input or output value of these addresses, we adopt the statistical  feature extraction method to retrieve the transfer feature for these  single-transaction hyper nodes, which is the features of the  single-transaction hyper node in the graph. After compressing address nodes, a transaction node can link to a maximum of two single-transaction hyper nodes on the input and output side of the transaction, respectively.


TRIALMASTER conducts inference on itself without any help from a  backtracking system like DFS or BFS. It outputs two kinds of tactics:  tactics in Lean and backtrack instructions. When TRIALMASTER is doing a  proof search on the test set, it is prompted with all history paths,  including previous tactics, states, the backtracking it made before, and the failed search path. It then outputs the entire proof path after.  Nonetheless, we only utilize the first tactic in the output and employ  Lean as a calculator to determine the next state, thereby ensuring the correctness of the state following the tactic. If the tactic outputted by TRIALMASTER is a backtrack instruction, it is then prompted with all the proof search history including the backtrack instruction and the state that the backtrack instruction says to return to. If that tactic is not a backtrack instruction, the tactic and the current state will be fed into Lean for producing the state after. TRIALMASTER is then prompted with the entire proof tree including the state that Lean calculated, and it should output a tactic again. This process is repeated until Lean identifies that the proof is complete or any Lean error occurs. We also note that TRIALMASTER only outputs one tactic at each state using greedy search. <PIC>: Method comparison. (a) A conventional system: The tactic generator (i.e., LLM) is fine-tuned on correct proof paths only. During inference, the trained tactic generator produces Nsampled (e.g., 2 in the example) tactics at a time. If Lean decides that the current tactic is wrong, the system backtracks to the last valid state and tries other candidate tactics. (b) Our methodology: The tactic generator is fine-tuned on proofs with trial-and-error. During inference, we take the first tactic it generates and feed that into Lean for state checking at each step. Why do we choose DFS over BFS? While the breadth-first-search (BFS) system is also popular for building neural provers in Automated Theorem Proving, we have opted for DFS as our baseline over BFS in the context of propositional logic theorem proving. This is due to the finite number (around 20) of tactics available at any step for the search process of intuitionistic propositional logic theorems, making DFS more efficient than BFS without compromising the success rate. 5.3 Results and Analysis
TRIALMASTER outperforms conventional DFS system. We begin by evaluating the methods of the in-distribution test set. Table 2 illustrates that both our method and the DFS system perform exceptionally well, achieving a success rate of nearly 100% in most configurations. This suggests that Llama-7b effectively masters in-distribution intuitionistic propositional logic theorems. Then, we compare the performance of the methods on the out-of-distribution task. The results are presented in <PIC>. Our method with trial-and-error significantly outperforms the DFS system across various
hyperparameter configurations. Additionally, we observe that feeding more proofs without trial-and-error for LLM fine-tuning does not further improve
the performance. Model learns backtracking capability from trial-and-error data. In the experiments, we find out that our TRIALMASTER successfully acquires the
backtracking capability from proofs with trial-anderror information. This is evidenced by the fact that during TRIALMASTER’s proof search for theorems in the test set, all backtracking instructions produced by the LLM adhere to the correct format and point to existing state numbers.
Including failed search paths helps TRIALMASTER to learn. The following experiment shows that adding failed search paths to the training data
for TRIALMASTER results in an overall gain. In this experiment, the model is only trained to learn the correct search paths and the backtracking instructions. The model is not trained to learn the failed search paths (we don’t compute the loss for the failed search paths during the training stage in
this case). The proof success rate in this case is 75.6%, which is lower than TRIALMASTER’s proof success rate of 88.7%. The NLean for the model is 13,600, which is lower than that of TRIALMASTER. This is expected since the model does not learn to predict failed search paths. Our explanation for why TRIALMASTER has a higher proof search success rate than the model trained in the previously mentioned experiment is that the failed search paths also contribute to improving the proof search success rate of the model. TRIALMASTER strategically tried some potentially failed search paths to gain a more comprehensive view of the problem, which then led to the correct search paths. 5.4 Ablation Study
To evaluate the effectiveness of training with trial-and-error, we craft an ablated version of our method where the LLM is fined-tuned with data of the correct path only and do inference in the same way as our method (i.e., producing one tactic at a time and applying Lean for state checking). We denote the ablated version as Model - proof w/o t.a.e.. For both methods, we mark the search attempt as failed if the tactic induces a Lean error, or the search exceeds the 1500-word limit. The result is shown in the Table 3. The difference between the success rates of the two models is 29.4%, which is significant. This clearly shows that failed search states and trial-and-error information tremendously enhance the model’s capability to solve theorem-proving tasks. 5.5 Exploratory Study: Effect of Training
Proof Length on Model Performance Since the FPS algorithm of PropL dataset can generate multiple proofs with variable length, we conduct an exploratory study to assess the impact of proof length on model performance. We fine-tune two models using proofs with different lengths of trial-and-error information. For the first model, which is our TRIALMASTER, the training data is derived by randomly selecting two out of the shortest four proofs from the ten available proofs for each theorem in PropL. We denote it as Model - short proof w/ t.a.e. In contrast, the training data of the second model is formed by randomly selecting two proofs from the ten available for each theorem, irrespective of their lengths. We denote it as Model - long proof w/ t.a.e. For both models, we use greedy search to let them generate one tactic for each state. We evaluate the models on our 1000 OOD test set. The results are shown in the <PIC>. A higher success rate is observed in the model trained with shorter  proofs. This can be attributed to the fact that as the proof with  trial-and-error information becomes longer, there is too much  trial-and-error information that may detrimentally affect the model's  performance, as too many failed search paths may lower the quality of  the training data. 6 Conclusion and Future Work
In this paper, we study Automated Theorem Proving in formalized environments. We create a complete, scalable, and representative dataset of intuitionistic propositional logic theorems in Lean. We demonstrate that leveraging information from failed search states and backtracking not only teaches models how to backtrack effectively, but also helps in developing better tactics than those generated by models trained without access to backtracking insights. We release our datasets on GitHub and Huggingface.