Skip to content
Research Article Open access CC BY 4.0

A Comprehensive Survey for Hadoop Distributed File System

Karwan Jameel Merceedi, Nareen Abdulla Sabry

Asian Journal of Research in Computer Science · pp. 46–57 · Published 23 Aug 2021

10.9734/ajrcos/2021/v11i230260

Abstract

In the last few days, data and the internet have become increasingly growing, occurring in big data. For these problems, there are many software frameworks used to increase the performance of the distributed system. This software is used for available ample data storage. One of the most beneficial software frameworks used to utilize data in distributed systems is Hadoop. This software creates machine clustering and formatting the work between them. Hadoop consists of two major components: Hadoop Distributed File System (HDFS) and Map Reduce (MR). By Hadoop, we can process, count, and distribute each word in a large file and know the number of affecting for each of them. The HDFS is designed to effectively store and transmit colossal data sets to high-bandwidth user applications. The differences between this and other file systems provided are relevant. HDFS is intended for low-cost hardware and is exceptionally tolerant to defects. Thousands of computers in a vast cluster both have directly associated storage functions and user programmers. The resource scales with demand while being cost-effective in all sizes by distributing storage and calculation through numerous servers. Depending on the above characteristics of the HDFS, many researchers worked in this field trying to enhance the performance and efficiency of the addressed file system to be one of the most active cloud systems. This paper offers an adequate study to review the essential investigations as a trend beneficial for researchers wishing to operate in such a system. The basic ideas and features of the investigated experiments were taken into account to have a robust comparison, which simplifies the selection for future researchers in this subject. According to many authors, this paper will explain what Hadoop is and its architectures, how it works, and its performance analysis in a distributed systems. In addition, assessing each Writing and compare with each other.

Hadoop HDFS distributed file system

References (42)

  1. 1 MapReduce [DOI]
  2. 2 Hive [DOI]
  3. 3 A comparison of approaches to large-scale data analysis [DOI]
  4. 4 Building a high-level dataflow system on top of Map-Reduce [DOI]
  5. 5 The Hadoop distributed filesystem: Balancing portability and performance [DOI]
  6. 6 Graph Databases: New Opportunities for Connected Data
  7. 7 The Design and Implementation of Modern Column-Oriented Database Systems [DOI]
  8. 8 Implementing WebGIS on Hadoop: A case study of improving small file I/O performance on HDFS [DOI]
  9. 9 Cloud Computing Virtualization of Resources Allocation for Distributed Systems [DOI]
  10. 10 Task Scheduling Algorithms in Cloud Computing: A Review [DOI]
  11. 11 GFS [DOI]
  12. 12 Distributed Cloud Computing and Distributed Parallel Computing: A Review [DOI]
  13. 13 Comparison Among Cloud Technologies and Cloud Performance [DOI]
  14. 14 Comparison of Software Defined Networking with Traditional Networking [DOI]
  15. 15 A Survey of Optical Fiber Communications: Challenges and Processing Time Influences [DOI]
  16. 16 Multi-dimensional Index on Hadoop Distributed File System [DOI]
  17. 17 Comprehensive Survey for Cloud Computing Based Nature-Inspired Algorithms Optimization Scheduling [DOI]
  18. 18 A State of Art: Survey for Concurrent Computation and Clustering of Parallel Computing for Distributed Systems [DOI]
  19. 19 Unified Ontology Implementation of Cloud Computing for Distributed Systems [DOI]
  20. 20 IoT Provisioning QoS based on Cloud and Fog Computing [DOI]
  21. 21 A Distributed Cache for Hadoop Distributed File System in Real-Time Cloud Services [DOI]
  22. 22 Improving performance of small-file accessing in Hadoop [DOI]
  23. 23 A novel indexing scheme for efficient handling of small files in Hadoop Distributed File System [DOI]
  24. 24 An Efficient Binary Locally Repairable Code for Hadoop Distributed File System [DOI]
  25. 25 An approach for big data security based on Hadoop distributed file system [DOI]
  26. 26 Cache Coherence Protocols in Distributed Systems [DOI]
  27. 27 Data security in Hadoop distributed file system [DOI]
  28. 28 Enhancing throughput of the Hadoop Distributed File System for interaction-intensive tasks [DOI]
  29. 29 A dynamic replica strategy based on Markov model for hadoop distributed file system (HDFS) [DOI]
  30. 30 State of Art Survey for Significant Relations between Cloud Computing and Distributed Computing [DOI]
  31. 31 The life and times of a zookeeper [DOI]
  32. 32 Performance analysis of MapReduce programs on Hadoop cluster [DOI]
  33. 33 Performance Evaluation of Read and Write Operations in Hadoop Distributed File System [DOI]
  34. 34 Performance evaluation and tuning for MapReduce computing in Hadoop distributed file system [DOI]
  35. 35 Hadoop Distributed File System for Big data analysis [DOI]
  36. 36 Hadoop-MapReduce: A platform for mining large datasets
  37. 37 Capturing Hadoop Storage Big Data Layer Meta-Concepts [DOI]
  38. 38 Modeling and Simulation of Hadoop Distributed File System in a Cluster of Workstations [DOI]
  39. 39 Zput: A speedy data uploading approach for the Hadoop Distributed File System [DOI]
  40. 40 The Evolution of the Hadoop Distributed File System [DOI]
  41. 41 A Novel Approach of Fair Scheduling to Enhance Performance of Hadoop Distributed File System [DOI]
  42. 42 Remote Controlling Distributed Parallel Computing System over the Cloud (RCDPCSC) [DOI]

Cited by 60

Toward smarter redundancy in Hadoop: a systematic review of replication and erasure coding approaches

Helmi Tlich, Hanene Chettaoui, T. Hamrouni · Cluster Computing · 2026

A Systematic Review of Artificial Intelligence Techniques for Distributed Storage and Data Processing in Cloud Computing

R. Gandhi, Jimmy Singla · 2026 5th International Conference on Sentiment Analysis and Deep Learning (ICSADL) · 2026

Predicting Adolescent Suicide Risk in Smart Cities: An AI-Driven, Privacy-Preserving Architecture

Serik Aliaskarov, Orazmukhamed Bekmurat, Vassily Serbin · HighTech and Innovation Journal · 2025

Semantic Data Federated Query Optimization Based on Decomposition of Block-Level Subqueries

Yuan Yao, Yang Zhang · Future Internet · 2025

Performance Evaluation of Distributed Framework for Healthcare Data Storage and Analysis Using Hadoop-HBase

A. Khaleel, Ahmed Adnan Mohammed Al-Azzawi, Abas Wisam Mahdi Abas · 2025 International Conference on Big Data, Knowledge and Control Systems Engineering (BdKCSE) · 2025

A Scalable Big Data-Driven Distributed Deep Learning Framework for Breast Cancer Diagnosis Using Big Data Analytics

Muhammad Babar, Sarah Kaleem, M. El-Affendi · Engineering, Technology & Applied Science Research · 2025

A comprehensive survey on big data privacy and Hadoop security: Insights into encryption mechanisms and emerging trends

Youness Filaly, Nisrine Berros, Fatna El Mendili · Results in Engineering · 2025

Smart Data Prefetching Using KNN to Improve Hadoop Performance

Rana Ghazali, Douglas G. Down · EAI Endorsed Transactions on Scalable Information Systems · 2025

Efficient small file management in Hadoop distributed file system for enhanced e-government services

F. Ishengoma · Technological Sustainability · 2025

Exploring a digital music teaching model integrated with recurrent neural networks under artificial intelligence

Yang Han · Scientific Reports · 2025

Showing 32 of 60 known citations — external sources report more than can currently be individually listed.

Related research

Article metrics

Real usage data collected on this platform.

61

Page views

0

PDF downloads

3

Outbound clicks

60

Citations

Views over time

Views by country

Approximate, from request IP at view time — not citizenship or institution. Countries with fewer than 5 views are grouped as "Other".

Traffic sources

Referring site, by host.

Views and downloads exclude known bots/crawlers. Citations combines this platform's own DOI-resolved index with each external source's own reported total — see Cited by above for individually listed citing works. Last refreshed 0 seconds ago.