引用本文: | 张 磊,刘辛彤,蔡 硕,刘红艳,刘 蕾.基于Storm架构的电力物联网流数据处理[J].电力系统保护与控制,2021,49(20):112-119.[点击复制] |
ZHANG Lei,LIU Xintong,CAI Shuo,LIU Hongyan,LIU Lei.Stream data processing of the power internet of things based on Storm architecture[J].Power System Protection and Control,2021,49(20):112-119[点击复制] |
|
摘要: |
数据的采集、分析和处理是目前电力系统重点关注的问题之一。电力物联网系统中数据业务处理需求多样,尤其是流数据的处理对时延要求严格。然而,现有的数据处理方法形式单一,不能很好地满足低时延处理要求。因此,提出了基于Storm架构的电力物联网流数据处理方法。首先,基于Storm拓扑结构提出分布式流数据处理框架。进而采用流水线式的处理方式,从而达到缩短处理时间的效果。在数据接入后,采用循环队列和流转算子的方法。然后再通过边缘计算处理海量的数据,实现高效的协同工作。此外,通过支持向量机预测算法预测数据的发展趋势,采用清洗技术对脏数据进行处理,解决数据传输过程中的污染情况。仿真结果表明,与传统数据处理方法相比,所提流数据处理方法大大缩短了数据处理时间,满足了大量流数据处理的需求。 |
关键词: 流数据处理 数据清洗 Storm结构 支持向量机 电力物联网 |
DOI:DOI: 10.19783/j.cnki.pspc.201655 |
投稿时间:2020-12-31修订日期:2021-08-19 |
基金项目:国家自然科学基金项目资助(61971190);国网河北省电力有限公司科技项目资助(SGHEXT00GCJS2000167) |
|
Stream data processing of the power internet of things based on Storm architecture |
ZHANG Lei,LIU Xintong,CAI Shuo,LIU Hongyan,LIU Lei |
(1. State Grid Hebei Electric Power Co., Ltd. Information and Communication Branch, Shijiazhuang 050000, China;
2. School of Electrical & Electronic Engineering, North China Electric Power University, Baoding 071003, China) |
Abstract: |
Data collection, analysis and processing is one of the most important problems in power systems. There are various requirements for data business processing in the power internet of things (IoT) systems. In particular, there are strict requirements on delay for stream data processing. However, the existing data processing methods are single in form and cannot well meet the requirements of low-delay processing. Therefore, a stream data processing method of power IoT based on Storm architecture is proposed. First, a distributed stream data processing framework is proposed based on Storm topology. Then the pipeline-based processing method is adopted to shorten the processing time. Then data access, a circular queue and a flow operator are adopted. Then, through the edge computing processing of massive data, efficient collaborative work is achieved. In addition, the support vector machine prediction algorithm is used to predict the development trend of data, and the dirty data is processed by cleaning technology to solve the pollution in the process of data transmission. The simulation results show that, compared with the traditional data processing methods, the proposed method can greatly shorten data processing time and meet the needs of processing a large number of data streams.
This work is supported by the National Natural Science Foundation of China (No. 61971190) and the Science and Technology Project of State Grid Hebei Electric Power Co., Ltd. (No. SGHEXT00GCJS2000167). |
Key words: stream data processing data cleaning construction of Storm support vector machine power internet of things |