Journal of Modern Power Systems and Clean Energy

ISSN 2196-5625 CN 32-1884/TK

Big data compression processing and verification based on Hive for smart substation
Author:
Affiliation:

1. School of Electrical & Electronic Engineering, East China Jiaotong University, Nanchang, 330013, China

Fund Project:

National Natural Science Foundation of China (No. 51267005) and Jiangxi Province University Visiting Scholar Special Funds for Young Teacher Development Plan (No. G201415, No. GJJ13350)

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
    Abstract:

    The capacity and the scale of smart substation are expanding constantly, with the characteristics of information digitization and automation, leading to a quantitative trend of data. Aiming at the existing processing shortages in the big data processing, the query and analysis of smart substation, a data compression processing method is proposed for analyzing smart substation and Hive. Experimental results show that the compression ratio and query time of RCFile storage format are better than those of TextFile and SequenceFile. The query efficiency is improved for data compressed by Deflate, Gzip and Lzo compression formats. The results verify the correctness of adjacent speedup defined as the index of cluster efficiency. Results also prove that the method has a significant theoretical and practical value for big data processing of smart substation.

    Reference
    Related
    Cited by
Get Citation
Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:
  • Revised:
  • Adopted:
  • Online: August 26,2015
  • Published: