HBase选择Store file做compaction的算法(2)

long totalSize = 0;
      if (!majorcompaction && !references) {
        // we're doing a minor compaction, let's see what files are applicable
        int start = 0;
        double r = this.compactRatio;

/* Start at the oldest file and stop when you find the first file that
        * meets compaction criteria:
        * 从老的storefile到新的storefile进行遍历,停止的条件是当遇到一个storefile的
        * 大小小于minCompactSize的时候,或者是小于后面maxFilesToCompact个storefile
        * 大小的和乘以compactRatio(默认1.2)
        *
        * X <= minCompactSize || X <= SUM_ * compactRatio ==> 停止
        * X > minCompactSize && x > SUM_ * compactRation  ==> 继续扫描
        * X > max(minCompactSize, SUM_ * compactRation)  ==> 继续扫描
        *  (1) a recently-flushed, small file (i.e. <= minCompactSize)
        *      OR
        *  (2) within the compactRatio of sum(newer_files)
        * Given normal skew, any newer files will also meet this criteria
        *
        * Additional Note:
        * If fileSizes.size() >> maxFilesToCompact, we will recurse on
        * compact().  Consider the oldest files first to avoid a
        * situation where we always compact [end-threshold,end).  Then, the
        * last file becomes an aggregate of the previous compactions.
        */
        /*
        * 至少有compactionThreshold这么多个store files
        * 至少满足停止条件(1)(2)的时候
        * ==>
        * 才能进行min compaction
        */
        while(countOfFiles - start >= this.compactionThreshold &&
              fileSizes[start] >
                Math.max(minCompactSize, (long)(sumSize[start+1] * r))) {
          ++start;
        }
       
        // 确定我们一次min compaction最多只能有maxFilesToCompact个store file
        int end = Math.min(countOfFiles, start + this.maxFilesToCompact);
        // 包含在这次min compaction里面的store file总大小
        totalSize = fileSizes[start]
                  + ((start+1 < countOfFiles) ? sumSize[start+1] : 0);

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:http://www.heiqu.com/f1a07c90478d39354a606428e68eaeb5.html