Skip to content
Research Article Open access CC BY 4.0

A Comprehensive Survey for Hadoop Distributed File System

Karwan Jameel Merceedi, Nareen Abdulla Sabry

Asian Journal of Research in Computer Science · pp. 46–57 · Published 23 Aug 2021

10.9734/ajrcos/2021/v11i230260

Abstract

In the last few days, data and the internet have become increasingly growing, occurring in big data. For these problems, there are many software frameworks used to increase the performance of the distributed system. This software is used for available ample data storage. One of the most beneficial software frameworks used to utilize data in distributed systems is Hadoop. This software creates machine clustering and formatting the work between them. Hadoop consists of two major components: Hadoop Distributed File System (HDFS) and Map Reduce (MR). By Hadoop, we can process, count, and distribute each word in a large file and know the number of affecting for each of them. The HDFS is designed to effectively store and transmit colossal data sets to high-bandwidth user applications. The differences between this and other file systems provided are relevant. HDFS is intended for low-cost hardware and is exceptionally tolerant to defects. Thousands of computers in a vast cluster both have directly associated storage functions and user programmers. The resource scales with demand while being cost-effective in all sizes by distributing storage and calculation through numerous servers. Depending on the above characteristics of the HDFS, many researchers worked in this field trying to enhance the performance and efficiency of the addressed file system to be one of the most active cloud systems. This paper offers an adequate study to review the essential investigations as a trend beneficial for researchers wishing to operate in such a system. The basic ideas and features of the investigated experiments were taken into account to have a robust comparison, which simplifies the selection for future researchers in this subject. According to many authors, this paper will explain what Hadoop is and its architectures, how it works, and its performance analysis in a distributed systems. In addition, assessing each Writing and compare with each other.

Hadoop HDFS distributed file system

References (42)

  1. 1 MapReduce [DOI]
  2. 2 Hive [DOI]
  3. 3 A comparison of approaches to large-scale data analysis [DOI]
  4. 4 Building a high-level dataflow system on top of Map-Reduce [DOI]
  5. 5 The Hadoop distributed filesystem: Balancing portability and performance [DOI]
  6. 6 Graph Databases: New Opportunities for Connected Data
  7. 7 The Design and Implementation of Modern Column-Oriented Database Systems [DOI]
  8. 8 Implementing WebGIS on Hadoop: A case study of improving small file I/O performance on HDFS [DOI]
  9. 9 Cloud Computing Virtualization of Resources Allocation for Distributed Systems [DOI]
  10. 10 Task Scheduling Algorithms in Cloud Computing: A Review [DOI]
  11. 11 GFS [DOI]
  12. 12 Distributed Cloud Computing and Distributed Parallel Computing: A Review [DOI]
  13. 13 Comparison Among Cloud Technologies and Cloud Performance [DOI]
  14. 14 Comparison of Software Defined Networking with Traditional Networking [DOI]
  15. 15 A Survey of Optical Fiber Communications: Challenges and Processing Time Influences [DOI]
  16. 16 Multi-dimensional Index on Hadoop Distributed File System [DOI]
  17. 17 Comprehensive Survey for Cloud Computing Based Nature-Inspired Algorithms Optimization Scheduling [DOI]
  18. 18 A State of Art: Survey for Concurrent Computation and Clustering of Parallel Computing for Distributed Systems [DOI]
  19. 19 Unified Ontology Implementation of Cloud Computing for Distributed Systems [DOI]
  20. 20 IoT Provisioning QoS based on Cloud and Fog Computing [DOI]
  21. 21 A Distributed Cache for Hadoop Distributed File System in Real-Time Cloud Services [DOI]
  22. 22 Improving performance of small-file accessing in Hadoop [DOI]
  23. 23 A novel indexing scheme for efficient handling of small files in Hadoop Distributed File System [DOI]
  24. 24 An Efficient Binary Locally Repairable Code for Hadoop Distributed File System [DOI]
  25. 25 An approach for big data security based on Hadoop distributed file system [DOI]
  26. 26 Cache Coherence Protocols in Distributed Systems [DOI]
  27. 27 Data security in Hadoop distributed file system [DOI]
  28. 28 Enhancing throughput of the Hadoop Distributed File System for interaction-intensive tasks [DOI]
  29. 29 A dynamic replica strategy based on Markov model for hadoop distributed file system (HDFS) [DOI]
  30. 30 State of Art Survey for Significant Relations between Cloud Computing and Distributed Computing [DOI]
  31. 31 The life and times of a zookeeper [DOI]
  32. 32 Performance analysis of MapReduce programs on Hadoop cluster [DOI]
  33. 33 Performance Evaluation of Read and Write Operations in Hadoop Distributed File System [DOI]
  34. 34 Performance evaluation and tuning for MapReduce computing in Hadoop distributed file system [DOI]
  35. 35 Hadoop Distributed File System for Big data analysis [DOI]
  36. 36 Hadoop-MapReduce: A platform for mining large datasets
  37. 37 Capturing Hadoop Storage Big Data Layer Meta-Concepts [DOI]
  38. 38 Modeling and Simulation of Hadoop Distributed File System in a Cluster of Workstations [DOI]
  39. 39 Zput: A speedy data uploading approach for the Hadoop Distributed File System [DOI]
  40. 40 The Evolution of the Hadoop Distributed File System [DOI]
  41. 41 A Novel Approach of Fair Scheduling to Enhance Performance of Hadoop Distributed File System [DOI]
  42. 42 Remote Controlling Distributed Parallel Computing System over the Cloud (RCDPCSC) [DOI]

Cited by 60

Design of Multi-priority Message Sending Method Based on Bandwidth State Control

Zhong-Jun-Dong-Zhong-Jun Dong · Journal of Science & Technology · 2023

Quantification and analysis of performance fluctuation in distributed file system

Yuchen Zhang, Xiao Zhang, Jin-Yang Yu · Cluster Computing · 2023

Construction of a rural tourism information service management system for multi-source heterogeneous data processing

Xuefei Wu, Jiahui Huang · PeerJ Computer Science · 2023

Survey of Distributed Computing Frameworks for Supporting Big Data Analysis

Xudong Sun, Yulin He, Dingming Wu · Big Data Mining and Analytics · 2023

Building an Effective Classifier for Phishing Web Pages Detection: A Quantum-Inspired Biomimetic Paradigm Suitable for Big Data Analytics of Cyber Attacks

S. Darwish, D. A. Farhan, Adel A. Elzoghabi · Biomimetics · 2023

High-performance fuzzy optimized deep convolutional neural network model for big data classification based on the social internet of things

B. Shaji, R. L. R. Singh, K. L. Nisha · Journal of Supercomputing · 2023

Research of the methods of creating content aggregation systems

Denis Aleksandrovich Kiryanov · Программные системы и вычислительные методы · 2022

A Column Encryption-Based Privacy-Preserving Framework for Hadoop Big Data Sets

Hidayath Ali Baig

Research on Mass Image Data Storage Method for Data Center

Sen Pan, Jing Jiang, Hongbin Qiu · Smart Innovation, Systems and Technologies · 2023

A Support Vector Machine-Based Cache Replacement Method for Efficient MapReduce Execution

Rana Ghazali, Ali Rezaee, Sahar Adabi · 2025

Showing 32 of 60 known citations — external sources report more than can currently be individually listed.

Related research

Article metrics

Real usage data collected on this platform.

60

Page views

0

PDF downloads

3

Outbound clicks

60

Citations

Views over time

Views by country

Approximate, from request IP at view time — not citizenship or institution. Countries with fewer than 5 views are grouped as "Other".

Traffic sources

Referring site, by host.

Views and downloads exclude known bots/crawlers. Citations combines this platform's own DOI-resolved index with each external source's own reported total — see Cited by above for individually listed citing works. Last refreshed 0 seconds ago.