Hadoop HDFS

non-cncf

First Research Pass

GitHub Repository Official Website

Content Lifecycle & Evolution State

1

Initial Seed

Metadata Loaded

2

First Research

Automated Gathering

3

Tech Writing

Content Structured

4

Approval

Editorial Verified

First Research Pass Completed: Initial facts, key features, and documentation links have been gathered. Detailed technical writing and synthesis are currently in queue.

Overview

The Hadoop Distributed File System (HDFS) is a highly fault-tolerant distributed file system designed to run on commodity hardware, providing high throughput access to application data and suitable for applications with large data sets.

Key Features

Highly fault-tolerant and designed to be deployed on low-cost hardware.
Provides high throughput access to application data, suited for batch processing rather than interactive use.
Tuned to support large files, scaling to hundreds of nodes and tens of millions of files in a single instance.
Simplifies data coherency issues with a write-once-read-many access model for files.

Use Cases

HDFS is primary distributed storage used by Hadoop applications. A MapReduce application or a web crawler application fits perfectly with its write-once-read-many model.

Getting Started

The command `bin/hdfs dfs -help` lists the commands supported by Hadoop shell to interact with HDFS.

Recent Updates

The most recent tag is release-3.5.0-RC2, released in March 2026.

Did You Know?

HDFS was originally built as infrastructure for the Apache Nutch web search engine project.