Apache Hadoop
non-cncf
First Research Pass
Content Lifecycle & Evolution State
1
Initial Seed
Metadata Loaded
2
First Research
Automated Gathering
3
Tech Writing
Content Structured
4
Approval
Editorial Verified
First Research Pass Completed: Initial facts, key features, and documentation links have been gathered. Detailed technical writing and synthesis are currently in queue.
Overview
A foundational open-source framework that allows for the distributed processing and storage of large data sets across clusters of computers.
Key Features
HDFS (Hadoop Distributed File System) for fault-tolerant distributed storage.
YARN for cluster resource negotiation and workload scheduling.
Hadoop MapReduce for parallel data computation.
Ecosystem compatibility with Hive, Spark, and cloud object stores.
Use Cases
Batch data warehousing, historical log archiving, and petabyte-scale data pipelines.
Getting Started
Configure `core-site.xml`, format namenode with `hdfs namenode -format`, and run `start-dfs.sh`.
Recent Updates
HDFS erasure coding cutting storage overhead by 50% and GPU scheduling in YARN.
Did You Know?
Apache Hadoop is widely utilized across the cloud native and open source ecosystem.