网站首页期刊简介编委会过刊目录投稿指南广告合作征订与发行联系我们English
基于工业实时时序海量测点数据的高效存储方法
英文标题:Efficient storage method on industrial real-time time-series massive measurement point data
作者:汤育玺1 袁超2 3 褚正全2 翟江波1 张浩2 张晓鹏1 刘昊松2 石一磬2 
单位:1. 陕西宏远航空锻造有限责任公司 陕西 咸阳 713800 2.中国机械总院集团北京机电研究所有限公司 北京 100083 3.华中科技大学 机械科学与工程学院 湖北 武汉 430074 
关键词:时序数据 数据存储优化 高频数据处理 工业大数据 锻压行业 
分类号:TP316
出版年,卷(期):页码:2025,50(6):268-276
摘要:

针对关系型数据库和时序数据库在处理锻压行业高频时序数据时存在的存储效率低、写入与查询性能不足等问题,提出了一种高效的工业实时时序海量测点数据存储方法。该方法创新性地采用JSON格式进行单列全量存储,有效地规避了传统数据库的列数限制,同时降低了存储空间占用。通过利用数据库底层的JSON操作符逻辑进行数据查询与检索,提高了时序数据的查询效率,尤其在大规模数据场景下展现出显著优势。实验结果表明,该方法在大数据量下的写入效率和存储空间占用分别较传统方法提升了5.6倍和节约了170倍,并且在数据查询速度上具备3~5倍的性能优势。

 

For the problems of low storage efficiency, insufficient writing and query performance of relational databases and time-series databases when processing high-frequency time-series data in the forging industry, an efficient industrial real-time time-series massive measurement point data storage method was proposed. The method innovatively used JSON format for single-column full-storage strategy, effectively circumventing the column number limitations of traditional databases and reducing the storage space occupancy. By using the underlying JSON operator logic of the database for data query and retrieval, the query efficiency of time-series data was improved, especially showing significant advantages in large-scale data scenarios. Experimental results demonstrate that compared with the traditional methods, this method improves the writing efficiency by 5.6 times and saves 170 times of storage space consumption under large data volumes, and has a performance advantage of 3-5 times in data query speed.  

 
基金项目:
国家重点研发计划资助项目(2022YFB3706904)
作者简介:
作者简介:汤育玺(1982-),男,硕士,高级工程师,E-mail:tyxsl@sohu.com;通信作者:袁超(1992-),男,博士,高级工程师,E-mail:804785930@qq.com
参考文献:

[1]周济.智能制造——“中国制造2025”的主攻方向[J].中国机械工程,2015,26(17):2273-2284.


 

Zhou J. Intelligent manufacturing—Main direction of “Made in China 2025” [J]. China Mechanical Engineering,2015,26(17):2273-2284.

 

[2]袁超,张浩,凌云汉,等.基于小波变换和S-G滤波的多尺度平滑预处理方法[J].锻压技术,2023,48(6):140-155.

 

Yuan C, Zhang H, Ling Y H, et al. Multiscale smoothing preprocessing method based on wavelet transform and S-G filtering[J]. Forging & Stamping Technology, 2023, 48(6):140-155.

 

[3]丁小欧,于晟健,王沐贤,等.基于相关性分析的工业时序数据异常检测[J].软件学报,2020,31(3):726-747.

 

Ding X O, Yu S J, Wang M X, et al. Anomaly detection on industrial time series based on correlation analysis[J]. Journal of Sofrware,2020,31(3):726-747.

 

[4]李潇睿,班晓娟,袁兆麟,等.工业场景下基于深度学习的时序预测方法及应用[J].工程科学学报,2022,44(4):757-766.

 

Li X R,Ban X J,Yuan Z L, et al. Review on deep learning models for time series forecasting in industry[J]. Chinese Journal of Engineering,2022,44(4):757-766.

 

[5]刘帅,乔颖,罗雄飞,等.时序数据库关键技术综述[J].计算机研究与发展,2024,61(3):614-638.

 

Liu S, Qiao Y, Luo X F,et al. Key techniques of time series databases: A survey[J]. Journal of Computer Research and Development,2024,61(3):614-638.

 

[6]郑孟蕾,田凌.基于时序数据库的产品数字孪生模型海量动态数据建模方法[J].清华大学学报(自然科学版), 2021,61(11): 1281-1288. 

 

Zheng M L, Tian L. Digital product twin modeling of massive dynamic data based on a time-series database[J]. Journal of Tsinghua University(Science and Technology),2021,61(11):1281-1288. 

 

[7]陈通,韩雪君,马延路.时序数据库在海量地震波形数据分布式存储与处理中的应用初探[J].中国地震,2022,38(4):799-809.

 

Chen T, Han X J, Ma Y L. Preliminary application of time series database in distributed storage and processing of massive seismic waveform data[J]. Earthquake Research in China,2022,38(4):799-809.

 

[8]张伟雄,唐娉,张正.基于时序自注意力机制的遥感数据时间序列分类[J].遥感学报,2023,27(8):1914-1924.

 

Zhang W X,Tang P, Zhang Z. Time series classification of remote sensing data based on temporal self-attention mechanism[J]. National Remote Sensing Bulletin,2023,27(8):1914-1924.

 

[9]谢伟,卢士达,时宽治,等.面向工业物联网时序数据的异常检测方法[J].计算机工程与应用,2024,60(12):270-282.

 

Xie W, Lu S D, Shi K Z, et al. Anomaly detection method for industrial IoT timing data[J]. Computer Engineering and Applications, 2024,60(12):270-282.

 

[10]Ahmed I. PostgreSQL数据库的特点[J].软件和信息服务, 2021(6):63.

 

Ahmed I. Features of PostgreSQL database[J].Software and Integrated Circuit,2021(6):63.

 

[11]王林彬,黎建辉,沈志宏.基于NoSQL的RDF数据存储与查询技术综述[J].计算机应用研究,2015,32(5):1281-1286.

 

Wang L B, Li J H, Shen Z H. Overview of NoSQL databases for large scaled RDF data management[J]. Application Research of Computers,2015,32(5):1281-1286.

 

[12]刘晓光.基于MySQL的分布式SQL数据库的设计与实现[D].北京:中国科学院大学,2016.

 

Liu X G. Design and Implementation of a Distributed Database Based on MySQL Database[D].Beijing:University of Chinese Academy of Sciences,2016.
服务与反馈:
文章下载】【加入收藏
《锻压技术》编辑部版权所有

中国机械工业联合会主管  中国机械总院集团北京机电研究所有限公司 中国机械工程学会主办
联系地址:北京市海淀区学清路18号 邮编:100083
电话:+86-010-82415085 传真:+86-010-62920652
E-mail: fst@263.net(稿件) dyjsjournal@163.com(广告)
京ICP备07007000号-9