| Peer-Reviewed

Fast Type Recognition of Missive Insulator Leakage Current Data Using Spark

Received: 21 May 2016     Published: 24 May 2016
Views:       Downloads:
Abstract

Performance is the key issue in power big data applications. One of main challenges is how to exploit these technologies in building power big data processing platform and facilitating science discoveries such as those in electric power systems. This paper explores how Spark and Cloud computing can accelerate performance of missive insulator leak current data pattern recognition. We have designed and implemented the Parallel KNN(k-NearestNeighbor) algorithm using Spark and then deployed onto the Aliyun E-MapReduce cloud computing platform. The results from experiments shows the performance and scalability can be enhanced through these advanced technologies.

Published in Journal of Electrical and Electronic Engineering (Volume 4, Issue 3)
DOI 10.11648/j.jeee.20160403.12
Page(s) 51-56
Creative Commons

This is an Open Access article, distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium or format, provided the original work is properly cited.

Copyright

Copyright © The Author(s), 2016. Published by Science Publishing Group

Keywords

Insulator Leakage Current, Electric Power Big Data, Spark

References
[1] Zhou, G., Zhu, Y., Wang, G., & Song, Y. “Real-time big data processing technology application in the field of state monitoring”. Diangong Jishu Xuebao/transactions of China Electrotechnical Society, vol.29, pp. 432-437.
[2] Uri Hasson, Jeremy I Skipper, Michael J Wilde, Howard C Nusbaum, and Steven L Small. Improving the analysis, storage and sharing of neuroimaging data using relational databases and distributed computing. NeuroImage, 39(2):693–706, 2008.
[3] Kai-Wei Chang, Deka, B Hwu, W.-M.W, etc. Efficient Pattern-based Time Series Classification on GPU[C]. 2012 IEEE 12th International Conference on Data Mining (ICDM 2012). Los Alamitos, CA, USA, 2012: 131-40.
[4] Zhao Jun, Zhu Xiaoliang, Wang Wei, etc. Extended Kalman filter-based Elman networks for industrial time series prediction with GPU acceleration [J]. Neurocomputing, 2013, 118: 215-224.
[5] Haimonti Dutta, Alex Kamil, Manoj Pooleery, Simha Sethumadhavan, and John Demme. Distributed Storage of Large-Scale Multidimensional Electroencephalogram Data Using Hadoop and HBase [J]. Grid and Cloud Database Management.2011.9.
[6] White T. Hadoop: The definitive guide [M]. O'Reilly Media, Inc, 2012:260-261.
[7] Christophe Bisciglia. The smart grid: Hadoop at the Tennessee Valley Authority (TVA) [EB/OL]. 2009.6 [2013.2]. http://www.cloudera.com/blog/2009/06/smart-grid-hadoop-tennessee-valley-authority-tva/
[8] Dean J, Ghemawat S. MapReduce: simplified data processing on large clusters [J]. Communications of the ACM, 2008, 51(1): 107-113.
[9] Zaharia M, Chowdhury M, Das T, et al. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing [A]. Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation[C]. USENIX Association, 2012: 2-2
[10] Zaharia M, Chowdhury M, Das T, et al. Fast and interactive analytics over Hadoop data with Spark[J]. USENIX; login, 2012, 37(4): 45-51
[11] Yan Y, Huang L, Yi L. Is Apache Spark scalable to seismic data analytics and computations? [C]// IEEE International Conference on Big Data. IEEE, 2015.
[12] Shyam R, Bharathi Ganesh H. B, Sachin Kumar S, et al. Apache Spark a Big Data Analytics Platform for Smart Grid[J]. Procedia Technology, 2015, 21:171-178.
[13] Mushtaq H, Al-Ars Z. Cluster-based Apache Spark implementation of the GATK DNA analysis pipeline[C]// Bioinformatics and Biomedicine (BIBM), 2015 IEEE International Conference on. IEEE, 2015:1471-1477.
[14] Ram&#, Rez-Gallego S, Garc&#, et al. Distributed Entropy Minimization Discretizer for Big Data Analysis under Apache Spark [C]// IEEE Trustcom/bigdatase/ispa. IEEE Computer Society, 2015.
[15] Cover, T., Hart, P. Nearest neighbor pattern classification [J]. IEEETrans. Inf. Theory, 1967, 30(1): 21–27
[16] Suda T. Frequency characteristics of leakage current waveforms of an artificially polluted suspension insulator [J]. Dielectrics & Electrical Insulation IEEE Transactions on, 2001, 8(4): 705-709.
Cite This Article
  • APA Style

    Song Yaqi. (2016). Fast Type Recognition of Missive Insulator Leakage Current Data Using Spark. Journal of Electrical and Electronic Engineering, 4(3), 51-56. https://doi.org/10.11648/j.jeee.20160403.12

    Copy | Download

    ACS Style

    Song Yaqi. Fast Type Recognition of Missive Insulator Leakage Current Data Using Spark. J. Electr. Electron. Eng. 2016, 4(3), 51-56. doi: 10.11648/j.jeee.20160403.12

    Copy | Download

    AMA Style

    Song Yaqi. Fast Type Recognition of Missive Insulator Leakage Current Data Using Spark. J Electr Electron Eng. 2016;4(3):51-56. doi: 10.11648/j.jeee.20160403.12

    Copy | Download

  • @article{10.11648/j.jeee.20160403.12,
      author = {Song Yaqi},
      title = {Fast Type Recognition of Missive Insulator Leakage Current Data Using Spark},
      journal = {Journal of Electrical and Electronic Engineering},
      volume = {4},
      number = {3},
      pages = {51-56},
      doi = {10.11648/j.jeee.20160403.12},
      url = {https://doi.org/10.11648/j.jeee.20160403.12},
      eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.jeee.20160403.12},
      abstract = {Performance is the key issue in power big data applications. One of main challenges is how to exploit these technologies in building power big data processing platform and facilitating science discoveries such as those in electric power systems. This paper explores how Spark and Cloud computing can accelerate performance of missive insulator leak current data pattern recognition. We have designed and implemented the Parallel KNN(k-NearestNeighbor) algorithm using Spark and then deployed onto the Aliyun E-MapReduce cloud computing platform. The results from experiments shows the performance and scalability can be enhanced through these advanced technologies.},
     year = {2016}
    }
    

    Copy | Download

  • TY  - JOUR
    T1  - Fast Type Recognition of Missive Insulator Leakage Current Data Using Spark
    AU  - Song Yaqi
    Y1  - 2016/05/24
    PY  - 2016
    N1  - https://doi.org/10.11648/j.jeee.20160403.12
    DO  - 10.11648/j.jeee.20160403.12
    T2  - Journal of Electrical and Electronic Engineering
    JF  - Journal of Electrical and Electronic Engineering
    JO  - Journal of Electrical and Electronic Engineering
    SP  - 51
    EP  - 56
    PB  - Science Publishing Group
    SN  - 2329-1605
    UR  - https://doi.org/10.11648/j.jeee.20160403.12
    AB  - Performance is the key issue in power big data applications. One of main challenges is how to exploit these technologies in building power big data processing platform and facilitating science discoveries such as those in electric power systems. This paper explores how Spark and Cloud computing can accelerate performance of missive insulator leak current data pattern recognition. We have designed and implemented the Parallel KNN(k-NearestNeighbor) algorithm using Spark and then deployed onto the Aliyun E-MapReduce cloud computing platform. The results from experiments shows the performance and scalability can be enhanced through these advanced technologies.
    VL  - 4
    IS  - 3
    ER  - 

    Copy | Download

Author Information
  • Department of Computer Science, North China Electric Power University, Baoding, China

  • Sections