xpqiu commited on
Commit
4235d7d
·
verified ·
1 Parent(s): 75cdf21

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -5
README.md CHANGED
@@ -61,12 +61,22 @@ Yunfan Shao, Zhichao Geng, Yitao Liu, Junqi Dai, Fei Yang, Li Zhe, Hujun Bao, Xi
61
  **Note: Please use BertTokenizer for the model vocabulary. DO NOT use original BartTokenizer.**
62
 
63
  ## Citation
 
 
64
 
65
  ```bibtex
66
- @article{shao2021cpt,
67
- title={CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation},
68
- author={Yunfan Shao and Zhichao Geng and Yitao Liu and Junqi Dai and Fei Yang and Li Zhe and Hujun Bao and Xipeng Qiu},
69
- journal={arXiv preprint arXiv:2109.05729},
70
- year={2021}
 
 
 
 
 
 
 
 
71
  }
72
  ```
 
61
  **Note: Please use BertTokenizer for the model vocabulary. DO NOT use original BartTokenizer.**
62
 
63
  ## Citation
64
+ Shao, Y., Geng, Z., Liu, Y. et al. CPT: a pre-trained unbalanced transformer for both Chinese language understanding and generation. Sci. China Inf. Sci. 67, 152102 (2024).
65
+ https://www.sciengine.com/SCIS/doi/10.1007/s11432-021-3536-5
66
 
67
  ```bibtex
68
+ @Article{Shao2024a,
69
+ author = {Shao, Yunfan and Geng, Zhichao and Liu, Yitao and Dai, Junqi and Yan, Hang and Yang, Fei and Li, Zhe and Bao, Hujun and Qiu, Xipeng},
70
+ journal = {Science China Information Sciences},
71
+ title = {CPT: a pre-trained unbalanced transformer for both Chinese language understanding and generation},
72
+ year = {2024},
73
+ issn = {1869-1919},
74
+ number = {5},
75
+ pages = {152102},
76
+ volume = {67},
77
+ abstract = {In this paper, we take the advantage of previous pre-trained models (PTMs) and propose a novel Chinese pre-trained unbalanced transformer (CPT). Different from previous Chinese PTMs, CPT is designed to utilize the shared knowledge between natural language understanding (NLU) and natural language generation (NLG) to boost the performance. CPT consists of three parts: a shared encoder, an understanding decoder, and a generation decoder. Two specific decoders with a shared encoder are pre-trained with masked language modeling (MLM) and denoising auto-encoding (DAE) tasks, respectively. With the partially shared architecture and multi-task pre-training, CPT can (1) learn specific knowledge of both NLU or NLG tasks with two decoders and (2) be fine-tuned flexibly that fully exploits the potential of the model. Moreover, the unbalanced transformer saves the computational and storage cost, which makes CPT competitive and greatly accelerates the inference of text generation. Experimental results on a wide range of Chinese NLU and NLG tasks show the effectiveness of CPT.},
78
+ doi = {10.1007/s11432-021-3536-5},
79
+ refid = {Shao2024},
80
+ url = {https://doi.org/10.1007/s11432-021-3536-5},
81
  }
82
  ```