In this paper, we shed light on smart contract fuzzing by employing a sequence-aware mutation and seed mask guidance strategy. In particular, we first utilize data-flow-based feedback to determine  transaction orders in a meaningful way and further introduce a  sequence-aware mutation technique to explore deeper states. Thereafter,  we design a mask-guided seed mutation strategy that biases the generated transaction inputs to hit target branches. In addition, we develop a  dynamic-adaptive energy adjustment paradigm that balances the fuzzing  resource allocation during a fuzzing campaign. <PIC>. A high-level architecture and analysis pipeline of MuFuzz. We design and implement MuFuzz, a novel fuzzing framework for smart  contracts, which consists of a sequence-aware mutation module, a seed  mask guidance module, and a dynamic energy adjustment module. In summary, we make the following key contributions:
• We design a sequence-aware mutation and seed mask guidance strategy for blockchain smart contract fuzzing.
• We propose a new fuzzing framework for smart contracts that consists of three key components: a sequence-aware mutation, a mask-guided seed mutation, and a dynamic adaptive energy adjustment, which increases the probabil ity of exploring deep contract states. The proposed mod ules hold the potential to be transferable to the fuzzing of smart contracts on alternative blockchain platforms.
• We implement MuFuzz1 and conduct extensive exper iments on three benchmarks. Not only does MuFuzz outperform state-of-the-art smart contract fuzzers in terms of both coverage and runtime, but it is also able to detect more vulnerabilities than existing bug detection tools.
• We release both our system and benchmarks to facilitate future research, in the spirit of open science. II. BACKGROUND
A. Ethereum Smart Contract
The notion of smart contracts was originally put forth in 1994 by Nick Szabo [42], who described the concept of a trustless system containing self-executing computer programs. However, the concept did not become a reality until the emergence of the Ethereum in 2014 [43]. Ethereum is currently
the most popular blockchain, which employs the Ethereum Virtual Machine (EVM) to execute smart contracts. Smart contracts maintain Storage, which is organized as a key value store to persistently hold state variables [18]. Since smart contracts are stored on the blockchain, they inherit certain properties. The immutability of the blockchain ensures that the dapp/program execution strictly adheres to the rules defined in the smart contract, which are unalterable once deployed on chain. Attributed to the decentralized nature, smart contracts allow transactions to transpire between anonymous parties without relying on a trusted third party. To date, more than 60 million smart contracts have been created on Ethereum [4],
giving rise to a variety of decentralized applications, such as decentralized finance (DeFi) [44], non-fungible tokens (NFT) [45], Internet of Things (IoT) [46], healthcare [47], crowdsale [48], and many others [1], [2], [49]. B. Smart Contract Vulnerability Smart contracts on the Ethereum blockchain have been
subject to numerous destructive attacks. The most notable ones include the DAO attack in 2016 [50], the Parity Multisig Wallet attack in 2017 [5], the Beauty Chain attack in 2018 [51], the Cream Finance attack in 2021 [9], and the Rari Fuse Pool attack in 2022 [52], which together resulted in huge financial losses. Prior works such as [53], [18] have studied and defined various defects in Ethereum smart contracts. Here, we mainly focus on detecting nine types of vulnerabilities, which are summarized in Table I. The reasons for considering these bug classes are: (1) A large body of previous research has demonstrated that these vulnerabilities account for a significant portion of the existing bugs in the Ethereum ecosystem [12], [13]. (2) Over 90% of the financial losses in Ethereum smart contracts are caused by these vulnerabilities [6], [30], [54]. (3) We have investigated the bug classes handled by existing smart contract security tools and found that these vulnerabilities are of the highest concern [18], [15], [19], [33]. <PIC> shows the experimental results, where each bar represents the percentage of achieved coverage or detected vulnerabilities compared to the results when all three com ponents were enabled (i.e., the gray bar in the back). It can be seen from the figure that each component is essential to the performance improvement of MuFuzz. In particular, we can observe that generating a meaningful transaction sequence plays the most critical role in achieving branch coverage and detecting more vulnerabilities. Quantitatively, without the three components, the achieved coverage decreases by 18%, 9%, 10% on small contracts and 26%, 19%, 25% on large contracts, respectively. Moreover, MuFuzz can discover 14%, 6%, 11% more bugs on small contracts and 27%, 22%, 24% more bugs on large contracts than without the three components, respectively. E. Real-World Case Study (Answering RQ4)
We also pay attention to the scalability and practicality of MuFuzz. We randomly select 100 contracts from D3, where each contract contains more than 30,000 transactions in Ethereum. We manually check the bug detection results and classify them into true positives (TP) and false positives (FP). In addition, we present the overall branch coverage (i.e., the average of the 100 contract runs) of MuFuzz. Table IV summarizes the experimental results. From the table, we can see that: (1) MuFuzz reports a total of 86 bug alarms. Out of the 100 contracts, 39 contracts are flagged as having at least one of these alarms. We manually verify the alarms and con firm that 94% of them are true positives. (2) MuFuzz achieves an average coverage of 80.71% on the 100 contracts, showing inspiring practicality in testing real-world large contracts. Case Study. To provide further insight on how MuFuzz achieves high coverage and detects bugs, we illustrate the fuzzing strategy of MuFuzz by through its workflow. <PIC> shows a contract where MuFuzz achieves 100% coverage while sFuzz and ConFuzzius achieve only 50%. Function withdraw has an if condition at line 30. If the condition cannot be satisfied, the bug inside the condition cannot be exposed. To cover line 31, one must call function invest twice to set the phase = 1. sFuzz and ConFuzzius fail to
cover line 31 because they cannot generate a sequence that runs invest twice. MuFuzz, instead, incorporates a sequence aware mutation strategy to create a sequence that can handle such conditions. Particularly, MuFuzz works in four main steps: 1) MuFuzz parses a contract source code into bytecode, ABI, and AST. By analyzing the ABI and AST, MuFuzz identifies which state variables are defined. 2) MuFuzz derives the dataflow dependencies of state variables and formulates a sequence of transactions. 3) MuFuzz determines the test inputs for each transaction in the sequence and activates
the mutation masking strategy to evolve the seed inputs. 4) MuFuzz analyzes the fuzzing log and identifies if there is a vulnerability by matching it against the bug oracles. The vulnerability in the crowdsale case can be exposed by MuFuzz with a sequence of transactions:


Transformers succeed in learning from Teaching Sequences. For all 5 tasks, we find that Transformers achieve perfect accuracy on subsequent points after receiving the teaching sequence. <PIC> shows the accuracy curve of Transformers and baselines on three representative tasks after receiving the first t points containing the teaching sequence. Note that, the accuracy of Transformers stays (close to) 100% after receiving the teaching sequence. It is interesting to see that Transformers succeed in learning with teaching sequences for tasks like DNFs and sparse parities where they struggled to learn in the vanilla setting. By definition, the teaching sequence is the smallest sequence required to learn the target function. Hence, it can often be much smaller than the sample complexity for a learning algorithm to learn from arbitrary distributions such as the ones considered in the previous section. Thus, our experiments show that models predict perfectly with much fewer examples when provided with these teaching sequences. This is not true for FFNs trained with gradient descent which fail to predict accurately given only the teaching sequence during training (see <PIC> right in Appendix). FFNs require a larger number of additional points apart from the teaching sequence to predict with high accuracy. Two distinct algorithms to learn one task. An interesting finding is that Transformers seem to learn two different algorithms to learn Conjunctions depending on the data distribution DX during the training process. See that (<PIC> left) when Transformers are trained to perform in-context learning with Conjunctions in the vanilla setting (as in Section 3) without teaching sequences, and tested on examples (or Prompts) containing teaching sequences, they are not able to leverage the teaching sequence and require more examples to predict accurately. The algorithm learned with standard Conjunctions still works on examples with teaching sequences even if it is not as sample-efficient. On the other hand, when Transformers are trained with teaching sequences and are tested in the standard setting without teaching sequences, they do not perform well since the learning algorithm relies on using the first t examples containing the teaching sequence (<PIC> center-left). These results indicate that models can learn two distinct algorithms depending on the distribution of inputs provided during training. We conducted another experiment where we trained a Transformer on a mixture of examples with and without teaching sequence. During training, we sample an example (or Prompt sequence) with a teaching sequence with probability 1 2 , and without a teaching sequence with equal probability. We evaluate the model on both tasks separately, one which contains examples with teaching sequences and one without them. We find that the same Transformer model could achieve near-optimal performance for both tasks – when provided with teaching sequences, it behaves like the model that is trained on just the Teach Conjunction task (<PIC> Right) and when provided with examples without teaching sequences, it behaves like a model trained on just the Conjunction task (<PIC> center-right). This indicates that Transformers can learn two distinct learning algorithms for Conjunctions and implement the optimal one depending on the sequence of in-context examples. This highlights the versatility of neural sequence models in being able to find separate learning algorithms with respect to the data distribution for solving the same task. Results with Frozen GPT. We evaluate the performance of a frozen GPT-2 model on tasks such as Conjunctions and Disjunctions. <PIC> depicts the performance on Conjunctions (the behavior on Disjunctions is almost identical). We find that the GPT-2 model performs relatively better than the nearest neighbor baseline and much better than a randomly initialized model on these tasks. However, it still does not come close to achieving the near-perfect accuracy obtained by fully trainable Transformer networks as in Section 3. GPT-2 can implement Nearest Neighbor. Given the observation that the performance of GPT-2 was close to the nearest neighbor algorithm, we examined if a frozen GPT-2 model can implement the nearest neighbor (NN) algorithm. We designed the NN task to test this hypothesis. In this task, each prompt contains 100 points where the first 20 points are labeled 0 or 1 uniformly at random and the subsequent 80 points are labeled according to the nearest neighbor algorithm. We then evaluate the GPT-2 model in the frozen-GPT setup described earlier and find that it can achieve near-perfect accuracy on the 80 points labeled with the nearest-neighbor algorithm (see <PIC> center). Moreover, upon analyzing the attention heads we found heads which closely implemented nearest neighbors — for an input xi , the attention head attended over yj where xj is the nearest neighbor of xi among x1, . . . , xi−1 (illustrated in <PIC> right). Further details about these experiments are provided in Appendix G. These heads are reminiscent of induction heads (Olsson et al., 2022) where instead of matching prefixes, they can find the nearest neighbor over the input space. Since the weights of the Transformers are frozen and only the input embedding and output layers are updated during the training process, the mechanisms to solve tasks such as NN and Conjunctions must have been learned during pretraining on language data. Moreover, a Transformer of the same size with randomly initialized weights is unable to perform much better than chance-level accuracy. Results with Direct Evaluation. The performances of all LLMs for the Conjunction and Majority tasks with varying number of dimensions are provided in <PIC>. It is clear that all models are able to perform as well as or better than the nearest neighbor baseline when the number of dimensions is up to 7. Note that in this setup, the LLM is essentially unaware of the task (such as Conjunction) when it is provided directly with the prompt at inference time. Even with n = 7, there are 2 128 Boolean functions, and so the in-context learning problem remains challenging, and the observed performance of these models is impressive. It is perhaps surprising to see that the open-source LLaMA-2 model performs quite similar to GPT-3.5-Turbo. It is also particularly interesting to see the strong performance of GPT-4; apart from outperforming all other models, it also slightly outperforms the nearest neighbor baseline even in the 15-dimensional case.


One of the most critical problems in BCFL is the resource allocation on local devices. Firstly, local devices with heterogeneous computational power usually have their own tasks to finish, so a universal resource allocation scheme for all the mobile devices is not practical. In addition, the whole system may not work effectively and sustainably if there are no reasonable rewards allocated to clients. Furthermore, both training and mining in the framework of BCFL consume significant amount of resources and time, and thus it is difficult for clients to appropriately allocate their limited resources to ensure the performance of the global model during the required time period. Lastly, since the system may not be aware of the amount of training data that each client owns, it can be challenging for the model owner (MO), i.e., the BCFL task publisher, to make proper decisions regarding the reward distribution. Inspired by [20], we consider a fully coupled BCFL system, which runs FL on a consortium blockchain network. In such a decentralized BCFL system, the participants in FL work as the blockchain nodes as well. Specifically, there are multiple local devices, termed as clients, working collaboratively to train a machine learning model, i.e., the global model. The set of clients can be denoted as N = {1, · · · , i, · · · , N} with N representing the total number of clients in the BCFL system. For simplicity, we refer to the work done by the blockchain for FL as mining in a uniform way, which does not imply that clients perform mining jobs consuming excessive amount of computing power like Proof of Work (PoW) [21]. In our system, we consider that lightweight consensus algorithms, such as Practical Byzantine Fault Tolerance (PBFT) [22] and Delegated Proof of Stake (DPoS) [23], are utilized in the consortium blockchain system. In our considered BCFL system, each client should be responsible for both training and mining. Since the workflow of the fully coupled BCFL is that mining starts only after the training is completed, we assume that training and mining are not parallel in this paper. Once the training is finished, all the clients will upload their local model updates to the blockchain network so as to be recorded on the blockchain. Here we define one round of BCFL as finishing both the training and mining processes. Since mobile devices usually have their own tasks to finish rather than only contributing to BCFL, we assume that they will not use all of their computation resources. In other words, CPU cycles per second for training and mining can be adjusted strategically in each round of BCFL. The MO is the requester of the FL task, aiming at to receive a well-trained final global model from the BCFL system. After the FL task is published on the blockchain, clients start to train their local models and then broadcast the obtained model updates to the blockchain network once the local training process is finished. By this means, the MO can only access the model updates from all clients rather than raw data of devices, thus preventing the leakage of private information for participants. An illustration of our system model is shown in <PIC>. The detailed workflow to finish one BCFL task is as below:
• The MO publishes an FL task, with the rewards for training and mining.
• Clients determine the computational resources, i.e., CPU cycles per second, used to train the model and mine for the blockchain based on the rewards provided by the MO.
• Each client trains the local model, and then broadcasts the model updates to the blockchain network. Then, clients start to mine the block.
• Once the block is generated, the model updates are stored on the blockchain, and the rewards will be delivered to each client.
• Clients calculate the global model with the verified model updates on chain. As long as the expected performance of the global modal is not reached, clients will start the next round of training based on the aggregated global model. Before we formulate the game model, we should clarify the fair reward allocation scheme first. In our system, we consider that each client has an equal chance to participate in both the training and mining processes with fair rewards. And since the allocation of rewards to each client in training and mining has a significant impact on the system fairness and further the participation willingness, we need to design a fair reward allocation scheme. Although we have already defined the payoff of each client during the training and mining processes in the above section, it is necessary to investigate their upper bounds based on the MO’s rewards budget. And the rewards distribution should not only be associated with the computing power of the device, but also take into account the performance of its work. On the one hand, the reward budget of the MO and the rewards that each client can get are limited; on the other hand, if the resources are allocated only based on the computing power devoted, it could lead to the situation where devices with sufficient computing power take most of the rewards, while devices with less power cannot get enough rewards, making the system unstable and unsustainable. In this section, we will conduct numerical experiments to verify and support our designed mechanism. We first clarify the experimental settings and then illustrate the results. We implement the simulations using Matlab 2019b in macOS 11.0.1 running on Intel i7 processor with 32 GB RAM and 1 TB SSD. First, we prove the correctness of the optimal strategies derived from our models. We assume there are 50 clients in total and each client has the same  ata size, so we set µi = 10. In our experiments, for clients and the MO, there are four strategy combinations, i.e., both sides choose the optimal strategies, one chooses the random strategies while the other chooses optimal strategies, and both choose the random strategies. For example, we define the strategy combination Random vs. Optimal as the clients choose the random strategies and the MO chooses the optimal strategy. We compare the utilities of clients and the MO with random strategies and optimal strategies, respectively. The results in <PIC> illustrate that clients and the MO can obtain the higher utilities than all other strategies when they both choose the optimal strategies, proving the validity of our proposed optimal strategies.


Abstract
Large Language Models (LLMs) have emerged as powerful tools across var ious domains within cyber security. Notably, recent studies are increasingly
exploring LLMs applied to the context of blockchain security (BS). How ever, there remains a gap in a comprehensive understanding regarding the
full scope of applications, impacts, and potential constraints of LLMs on blockchain security. To fill this gap, we undertake a literature review focus ing on the studies that apply LLMs in blockchain security (LLM4BS).
Our study aims to comprehensively analyze and understand existing re search, and elucidate how LLMs contribute to enhancing the security of
blockchain systems. Through a thorough examination of existing literature, we delve into the integration of LLMs into various aspects of blockchain secu rity. We explore the mechanisms through which LLMs can bolster blockchain security, including their applications in smart contract auditing, transaction anomaly detection, vulnerability repair, program analysis of smart contracts, and serving as participants in the cryptocurrency community. Furthermore, we assess the challenges and limitations associated with leveraging LLMs for enhancing blockchain security, considering factors such as scalability, pri vacy concerns, and ethical concerns. Our thorough review sheds light on the opportunities and potential risks of tasks on LLM4BS, providing valuable insights for researchers, practitioners, and policymakers alike. In this section, we introduce a thematic taxonomy devised to  systematically categorize the body of literature about tasks associated  with large language models for blockchain security (LLM4BS), emphasizing the function of the LLM within these contexts. <PIC> depicts the five  applications of LLM4BS task, involving code audit of smart contracts  §3.1, analysis of abnormal transactions §3.2, dynamic analysis of smart  contracts §3.3, development of smart contracts §3.4, participants of  cryptocurrency community §3.5, and other potential directions §3.6. Governance emerges as a major theme, as LLMs could contribute signifi cantly to the structuring and transparency of this largely unregulated space. The first document outlines the broader governance challenges faced by AI systems, suggesting blockchain as a viable solution to introduce verifiability
and accountability. On the other hand, the limitations of LLMs in capturing the complexities of legal reasoning are highlighted, a concern that is echoed
across the three studies to varying degrees. The practical applications of these models in legal settings, specifically detailed in the second and third documents, emphasize their innovative role in drafting legal complaints. This development is promising for the future of legal work related to cryptocurrency regulations and litigation, as it sug gests that LLMs could alleviate some of the workload from human experts, although the need for human oversight remains. While governance and legal assistance dominate the discourse, there’s a tone of cautious optimism throughout the texts. There is recognition of the transformative potential of LLMs in the cryptocurrency sector, but also a clear acknowledgment of the need for further advancement in AI technology to fully integrate into complex decision-making processes where legal and ethical considerations are paramount. In essence, the collective narrative from the three documents converges on the premise that LLMs hold transformative potential for the cryptocurrency
community’s governance and legal sectors but must overcome challenges in understanding before they can be fully trusted in autonomous roles. 3.6. Miscellaneous
As displayed in Table 7, LLM is also used in other blockchain security fields, involving smart contract compilers [57], zero-knowledge proofs [58], model training [20, 59], NFT generation [60]. We will introduce their appli cations in detail in the future. 4. Case study of LLM4BS
In this section, we engage in an in-depth examination through three dis tinct case studies, each serving to illustrate and shed light on the diverse
and concrete applications of Large Language Models for Blockchain Systems (LLM4BS). These cases in Table.8 have been meticulously selected to encom pass a broad range of scenarios, comprising LLM4FUZZ [44] §4.1, SMART INV [29] §4.2, BLOCKGPT [26] §4.3. LLM4FUZZ emerges as an innovative technique in the cybersecurity  landscape, specifically in the niche of smart contract security within  blockchain networks. It intricately combines the prowess of Large  Language Models (LLMs) with fuzz testing methodologies to proactively  unearth vulnerabilities that could potentially compromise the integrity  of smart contracts. LLMs are highly sophisticated AI models that have made significant  strides in understanding and generating human-like text, and more  recently, they have proven to be adept at comprehending programming  languages and code structure. LLM4FUZZ exploits this capacity by  deploying LLMs to guide fuzzing processes intelligently. This results in a more incisive and nuanced exploration of smart contracts, focusing  testing efforts on areas that LLMs determine to be most likely to  contain security flaws. By doing so, LLM4FUZZ succeeds in not only streamlining the anomaly detection process but also in enhancing its accuracy and depth. <PIC>: The architecture of LLM4FUZZ. In the world of blockchain technology, where smart contracts serve as immutable agreements that execute automatically based on coded conditions,
the potential negative impact of a security breach is heightened. Smart contracts control significant digital assets and are essential to the functioning of distributed applications (dApps). The immutable nature of blockchain adds a layer of complexity as deployed smart contracts, once committed to the blockchain, cannot be altered. Therefore, preemptive security assurances become crucial to ensuring their reliability and safeguarding the assets and
processes they govern. LLM4FUZZ provides a novel layer of security analysis by identifying and prioritizing potential problem areas within smart contract code. This prior itization is achieved through the LLM’s learned understanding of code pat terns that are historically or commonly associated with vulnerabilities. The methodology enhances traditional fuzzing strategies, which typically adopt a more scattergun approach by bombarding the code with random data inputs. LLM4FUZZ’s targeted testing is not just more efficient but also more effec tive in discovering complex vulnerabilities that might otherwise be missed. Following implementation, LLM4FUZZ has been benchmarked against existing fuzzing techniques and has consistently demonstrated superior performance. It expedites the vulnerability detection process and increases the breadth of security flaws that can be detected, thereby reinforcing the overall security posture. The case of LLM4FUZZ is emblematic of the foresight in AI integration into cybersecurity regimes. It encapsulates the transformative effects of AI on improving and redefining existing technological processes, particularly in areas critical to the burgeoning digital economy. Through its lens, we catch a glimpse of the future of smart contract security – a future where AI-driven tools not only anticipate but actively engage in the continuous battle against cyber threats.


We compare the performance of Astute RAG and baselines across different subsets partitioned by their retrieval precision, on our collected data with Claude as the LLM. As shown in \<PIC>, Astute RAG achieves consistently better performance than all baselines across different retrieval precision, indicating its effectiveness in improving RAG trustworthiness in broad scenarios. Notably, Astute RAG does not sacrifice performance gain under high retrieval quality in exchange for improvement under low retrieval quality. When the retrieval quality is extremely low (close to zero retrieval precision), all other RAG variants underperforms the 'No RAG' baseline, except for the proposed Astute RAG. This observation aligns with the worst-case results on RGB. It demonstrates the difficulty in overcoming imperfect retrieval augmentation, and verify the effectiveness of Astute RAG in doing so. title{Effectiveness in addressing knowledge conflicts}. We split our collected data in to three subset according to the answers from Claude, with and without RAG. The answers from two inference methods can be both correct, both incorrect, or conflicting with one being correct. These three subsets represents the three situations between internal and external knowledge. The results are shown in
ef{<PIC>}. On the conflicting subset, Astute RAG successfully chooses the correct answer in approximately 80//% of cases, being the most effective method in addressing knowledge conflicts. Notably, Astute RAG even brings performance improvement on the subset where neither internal nor external knowledge alone leads to the correct answer. This indicates that Astute RAG can effectively combine partially-correct information from LLM-internal and external knowledge, to achieve the correct answer through collective information across them. title{Worst-case performance on RGB.} \<PIC> presents the results under the worst-case setting on RGB where all retrieved documents are negative. It demonstrates the noise robustness of Astute RAG and baseline RAG methods. The performance gap between RAG and No RAG exceeds 50 points, highlighting the detrimental impact of imperfect retrieval results and emphasizing the importance of providing robust safeguards against worst-case scenarios. While the baseline RAG methods outperform the original RAG, they still obviously fall behind No RAG. Astute RAG is the only RAG method that reaches a performance close to No RAG under the worst-case scenario, further supporting its effectiveness in addressing imperfect retrieval augmentation. title{Qualitative study}. In \<PIC> we present two representative examples showing the intermediate outputs of Astute RAG. In the first example, LLM without RAG generates a wrong answer, while RAG returns a correct answer. Astute RAG successfully identified the incorrect information in its generated passage and an external passage, avoiding confirmation bias \cite{tan2024blinded}. In the second example, LLM alone is correct, while RAG is incorrect due to the noisy retrieval results. Astute RAG detected the correct answer from noisy retrieved information by checking with its internal knowledge.