HDFS applications need a write-once-read-many access model for files. A file once created, written, and closed need not be changed except for appends and truncates. Appending the content to the end of the files is supported but cannot be updated at arbitrary point. This assumption simplifies data coherency issues and enables high throughput data access. A MapReduce application or a web crawler application fits perfectly with this model.
HDFS应用需要一个一次写入多次读取的文件访问模型。一个文件一旦创建，写入和关系都不需要改变。支持在文件的末端进行追加数据而不支持在文件的任意位置进行修改。这个假设简化了数据一致性问题和支持高吞吐量的访问。支持在数据尾部增加内容而不支持在任意位置更新。一个Map/Reduce任务或者web爬虫完美匹配了这个模型。
“Moving Computation is Cheaper than Moving Data”（移动计算比移动数据更划算）A computation requested by an application is much more efficient if it is executed near the data it operates on. This is especially true when the size of the data set is huge. This minimizes network congestion and increases the overall throughput of the system. The assumption is that it is often better to migrate the computation closer to where the data is located rather than moving the data to where the application is running. HDFS provides interfaces for applications to move themselves closer to where the data is located.。
如果应用的计算在它要操作的数据附近执行那就会更高效。尤其是数据集非常大的时候。这将最大限度地减少网络拥堵和提高系统的吞吐量。这个假设是在应用运行中经常移动计算到要操作的数据附近比移动数据数据更好HDFS提供接口让应用去移动计算到数据所在的位置。
Portability Across Heterogeneous Hardware and Software Platforms（轻便的跨异构的软硬件平台）HDFS has been designed to be easily portable from one platform to another. This facilitates widespread adoption of HDFS as a platform of choice for a large set of applications.
HDFS被设计成可轻便从一个平台跨到另一个平台。这促使HDFS被广泛地采用作为应用的大数据集系统。
NameNode and DataNodesHDFS has a master/slave architecture. An HDFS cluster consists of a single NameNode, a master server that manages the file system namespace and regulates access to files by clients. In addition, there are a number of DataNodes, usually one per node in the cluster, which manage storage attached to the nodes that they run on. HDFS exposes a file system namespace and allows user data to be stored in files. Internally, a file is split into one or more blocks and these blocks are stored in a set of DataNodes. The NameNode executes file system namespace operations like opening, closing, and renaming files and directories. It also determines the mapping of blocks to DataNodes. The DataNodes are responsible for serving read and write requests from the file system’s clients. The DataNodes also perform block creation, deletion, and replication upon instruction from the NameNode.
The NameNode and DataNode are pieces of software designed to run on commodity machines. These machines typically run a GNU/Linux operating system (OS). HDFS is built using the Java language; any machine that supports Java can run the NameNode or the DataNode software. Usage of the highly portable Java language means that HDFS can be deployed on a wide range of machines. A typical deployment has a dedicated machine that runs only the NameNode software. Each of the other machines in the cluster runs one instance of the DataNode software. The architecture does not preclude running multiple DataNodes on the same machine but in a real deployment that is rarely the case.
The existence of a single NameNode in a cluster greatly simplifies the architecture of the system. The NameNode is the arbitrator and repository for all HDFS metadata. The system is designed in such a way that user data never flows through the NameNode.
HDFS使用主/从架构。一个HDFS集群包含一个NameNode，一个服务器管理系统的命名空间和并控制客户端对文件的访问。此外，有许多的DataNode，通常是集群中的每个节点，用来管理它们所运行的节点的内存。HDFS暴露文件系统的命名空间和允许用户数据存储在文件中。在系统内部，一个文件被切割成一个或者多个块而这些块将储存在一系列的DataNode中。NameNode执行文件系统的命名空间操作例如打开、关闭和从命名文件和路径。它也指定数据块对应的DataNode。DataNode负责提供客户端对文件的读写服务。DataNode也负责执行NameNode的创建、删除和复制指令。