A large-scale microblog dataset and stock movement prediction based on Supervised Contrastive Learning model

A large-scale microblog dataset; Natural Language Processing; Stock market; Supervised Contrastive Learning; Language processing; Large-scales; Learning methods; Micro-blog; Natural language processing; Natural languages; Neural-networks; Supervised contrastive learning; Textual data; Computer Science Applications; Cognitive Neuroscience; Artificial Intelligence

Abstract :

[en] The integration of Deep Neural Networks (DNN) with Natural Language Processing (NLP) technologies has opened new avenues in financial market prediction, particularly through the utilization of textual information. This study represents a significant advancement, which offers two primary contributions to stock trend prediction: (i) the exploitation of textual data (news, comments, microblogs) using advanced DNN architectures, enhancing market information utilization; (ii) significant improvement of the accuracy of predicting the direction of stock volatility by integrating textual and neural network technologies. Meanwhile, we have crawled, filtered, and constructed a large-scale microblog dataset. This dataset includes approximately 114,992 microblog textual data from 40 Science and Technology Innovation Board (STIB) companies in China during 2021. We conducted a comprehensive analysis using various DNN techniques, including Feedback Neural Networks (FNN), Supervised Contrastive Learning (SCL), Cross Entropy (CE), and Dual Contrastive Learning (DualCL), in conjunction with bag of words models, BERT, and Roberta compilers. Our findings reveal that the SCL method, when combined with microblog data, significantly increases prediction accuracy, particularly during the COVID-19 period. Furthermore, we discovered that using a cross-stock dataset enhances the accuracy of all prediction methods, and random allocation of microblog data leads to better results than sequential allocation. Additionally, we compared the efficacy of traditional models like the CAPM, three-factor, and five-factor models against neural network-based methods. Our results suggest a notable superiority of the SCL method in increasing prediction accuracy. Finally, applying our findings to real-world trading strategies, we demonstrated the practical advantages of using the SCL method in trading, evidenced by significant improvements across all performance indicators.

Disciplines :

Computer science

Author, co-author :

Yang, Song; GuiZhou University of Finance and Economics, China

TANG, Xunzhu ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > TruX

External co-authors :

yes

Language :

English

Title :

A large-scale microblog dataset and stock movement prediction based on Supervised Contrastive Learning model

Publication date :

June 2024

Journal title :

Neurocomputing

ISSN :

0925-2312

eISSN :

1872-8286

Publisher :

Elsevier B.V.

Volume :

584

Pages :

127583

Peer reviewed :

Peer Reviewed verified by ORBi

Additional URL :

https://api.elsevier.com/content/article/PII:S0925231224003540?httpAccept=text/xml

Funding text :

The authors thank the editors and reviewers for their comments, which led to the improvement of this paper. This work is supported by the Guizhou Province Philosophy and Social Science Foundation of China (Grant No. 22GZQN03).

Available on ORBilu :

since 02 September 2025

Statistics

Number of views

31 (2 by Unilu)

Number of downloads

60 (1 by Unilu)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

OpenAlex citations

WoS citations^™

Bibliography

Fama, E.F., French, K.R., A five-factor asset pricing model. J. Financ. Econ. 116 (2013), 1–22.
Rui Cheng, Qing Li, Modeling the momentum spillover effect for stock prediction via attribute-driven graph attention networks, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, 2021, pp. 55–62.
Guven, Murat, Cetinguc, Basak, Guloglu, Bulent, Calisir, Fethi, The effects of daily growth in COVID-19 deaths, cases, and governments’ response policies on stock markets of emerging economies. Res. Int. Bus. Finance, 61, 2022, 101659.
Liheng Zhang, Charu Aggarwal, Guo-Jun Qi, Stock price prediction via discovering multifrequency trading patterns, in: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2017, pp. 2141–2149.
Li, Yelin, Bu, Hui, Li, Jiahong, Wu, Junjie, The role of text-extracted investor sentiment in Chinese stock price prediction with the enhancement of deep learning. Int. J. Forecast. 36:4 (2020), 1541–1562.
Li, Bing, Chan, Keith C.C., Ou, Carol, Ruifeng, Sun, Discovering public sentiment in social media for predicting stock movement of publicly listed companies. Inf. Syst. 69 (2017), 81–92.
Mehtab, S., Sen, J., A robust predictive model for stock price prediction using deep learning and natural language processin. 2019 arXiv preprint arXiv:1912.07700.
Alshemali, B., Kalita, J., Improving the reliability of deep neural networks in NLP: A review. Knowl.-Based Syst., 191, 2020, 105210.
Min, B., Ross, H., Sulem, E., et al. Recent advances in natural language processing via large pre-trained language models: A survey. ACM Comput. Surv. 56:2 (2023), 1–40.
Xing, F.Z., Cambria, E., Welsch, R.E., Natural language based financial forecasting: a survey. Artif. Intell. Rev. 50 (2018), 49–73.
Khosla, P., Teterwak, P., Wang, C., et al. Supervised contrastive learning. Adv. Neural Inf. Process. Syst. 33 (2020), 18661–18673.
S. Wan, S. Pan, J. Yang, et al., Contrastive and generative graph convolutional networks for graph-based semi-supervised learning, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, 2021, pp. 10049–10057, (11).
Beliz Gunel, Jingfei Du, Alexis Conneau, Veselin Stoyanov, Supervised contrastive learning for pre-trained language model fine-tuning, in: International Conference on Learning Representations, ICLR, 2021.
Sailer, M., Bauer, E., Hofmann, R., et al. Adaptive feedback from artificial neural networks facilitates pre-service teachers’ diagnostic reasoning in simulation-based learning. Learn. Instr., 83, 2023, 101620.
Chen, Q., Zhang, R., Zheng, Y., et al. Dual contrastive learning: Text classification via label-aware data augmentation. 2022 arXiv preprint arXiv:2201.08702.
Sharpe, W.F., Capital asset prices: A theory of market equilibrium under conditions of risk. J. Finance 19:3 (1964), 425–442.
Fama, E.F., French, K.R., Common risk factors in the returns on stocks and bonds. J. Financ. Econ. 33 (1993), 3–56.
Patel, Jigar, Shah, Sahil, Thakkar, Priyank, Kotecha, K., Predicting stock and stock price index movement using Trend Deterministic Data Preparation and machine learning techniques. Expert Syst. Appl. 42:1 (2015), 259–268.
Pang, Bowen, Wei, Wei, Li, Xing, Feng, Xiangnan, Li, Chao, A representation-learning-based approach to predict stock price trend via dynamic spatiotemporal feature embedding. Eng. Appl. Artif. Intell., 126(Part A), 2023, 106849.
Mane, O., Stock market prediction using natural language processing–A survey. 2022 arXiv preprint arXiv:2208.13564.
Bustos, O., Pomares-Quimbaya, A., Stock market movement forecast: A Systematic review. Expert Syst. Appl., 156, 2020, 113464.
Wu, Junran, Xu, Ke, Zhao, Jichang, Predicting long-term returns of individual stocks with online reviews. Neurocomputing 417 (2020), 406–418.
Zou, J., Cao, H., Liu, L., et al. Astock: A new dataset and automated stock trading based on stock-specific news analyzing model. 2022 arXiv preprint arXiv:2206.06606.
Huang, Heyan, Liu, Xiao, Zhang, Yue, Feng, Chong, News-driven stock prediction via noisy equity state representation. Neurocomputing 470 (2022), 66–75.
Li, Xiaodong, Huang, Xiaodi, Deng, Xiaotie, Zhu, Shanfeng, Enhancing quantitative intra-day stock return prediction by integrating both market news and stock prices information. Neurocomputing 142 (2014), 228–238.
R. Sawhney, S. Agarwal, A. Wadhwa, et al., Deep attentive learning for stock movement prediction from social media text and company correlations, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP, 2020, pp. 8415–8426.
Zhang, Ming, Yang, Jiahao, Wan, Meilin, Zhang, Xuejun, Zhou, Jun, Predicting long-term stock movements with fused textual features of Chinese research reports. Expert Syst. Appl., 210, 2022, 118312.
Ma, Yu, Mao, Rui, Lin, Qika, Wu, Peng, Cambria, Erik, Multi-source aggregated classification for stock price movement prediction. Inf. Fusion 91 (2023), 515–528.
Engle, R.F., Autoregressive conditional heteroskedasticity with estimates of the variances of U.K. inflation. Econometrica 50 (1982), 987–1008.
Bollerslev, Tim, Generalized autoregressive conditional heteroskedasticity. J. Econometrics 21 (1986), 307–328.
Kara, Yakup, Boyacioglu, Melek Acar, Baykan, Ömer Kaan, Predicting direction of stock price index movement using artificial neural networks and support vector machines: The sample of the Istanbul Stock Exchange. Expert Syst. Appl. 38:5 (2011), 5311–5319.
Long, Wen, Lu, Zhichen, Cui, Lingxiao, Deep learning-based feature engineering for stock price movement prediction. Knowl.-Based Syst. 164 (2019), 163–173.
Wang, Lu, Zhao, Chenchen, Liang, Chao, Jiu, Song, Predicting the volatility of China's new energy stock market: Deep insight from the realized EGARCH-MIDAS model. Finance Res. Lett., 48, 2022, 102981.
Zhao, Yanli, Yang, Guang, Deep Learning-based Integrated Framework for stock price movement prediction. Appl. Soft Comput., 133, 2023, 109921.
Li, Shicheng, Huang, Xiaoyong, Cheng, Zhonghou, Zou, Wei, Yi, Yugen, AE-ACG: A novel deep learning-based method for stock price movement prediction. Finance Res. Lett., 58(Part A), 2023, 104304.
Huang, Yusheng, Mao, Xiaoyan, Deng, Yong, Natural visibility encoding for time series and its application in stock trend prediction. Knowl.-Based Syst., 232, 2021, 107478.
Li, Shangzhe, Wu, Junran, Jiang, Xin, Xu, Ke, Chart GCN: Learning chart information with a graph convolutional network for stock movement prediction. Knowl.-Based Syst., 248, 2022, 108842.
Gao, Ruize, Cui, Shaoze, Xiao, Hongshan, et al. Integrating the sentiments of multiple news providers for stock market index movement prediction: A deep learning approach based on evidential reasoning rule. Inform. Sci. 615 (2022), 529–556.
Xu, Hongfeng, Chai, Lei, Luo, Zhiming, Li, Shaozi, Stock movement predictive network via incorporative attention mechanisms based on tweet and historical prices. Neurocomputing 418 (2020), 326–339.
Xu, Hongfeng, Cao, Donglin, Li, Shaozi, A self-regulated generative adversarial network for stock price movement prediction based on the historical price and tweets. Knowl.-Based Syst., 247, 2022, 108712.
Devlin, Jacob, Chang, Mingwei, Lee, Kenton, Toutanova, Kristina, BERT: Pre-training of deep bidirectional transformers for language understanding. 2018 arXiv:1810.04805.
Liu, Yinhan, Ott, Myle, Goyal, Naman, et al. Roberta: A robustly optimized bert pretraining approach. 2019 arXiv:1907.11692.
Chen, Xingtong, Ma, Xiang, Wang, Hua, Li, Xuemei, Zhang, Caiming, A hierarchical attention network for stock prediction based on attentive multi-view news learning. Neurocomputing 504 (2022), 1–15.
Tang, Daniel, Chen, Zhenghan, Kim, Kisub, Song, Yewei, Tian, Haoye, Ezzini, Saad, Huang, Yongfeng, Bissyande, Jacques Klein Tegawende F, Collaborative agents for software engineering. 2024 arXiv preprint arXiv:2402.02172.
Kachuee, M., Yuan, H., Kim, Y.B., Lee, S., Self-supervised contrastive learning for efficient user satisfaction prediction in conversational agents. 2020 arXiv preprint arXiv:2010.11230.
Falcon, W., Cho, K., A framework for contrastive self-supervised learning and designing a new approach. 2020 arXiv preprint arXiv:2009.00104.
Ilya Loshchilov, Frank Hutter, Decoupled weight decay regularization, in: International Conference on Learning Representations, ICLR, 2018.
Wolf, Thomas, Debut, Lysandre, Sanh, Victor, et al. Huggingface’ s transformers: State-of-the-art natural language processing. 2019 arXiv preprint: arXiv:1910.03771.
Fama, E.F., French, K.R., International tests of a five-factor asset pricing model. J. Financ. Econ. 123:3 (2017), 441–463.