Git:在此树中查找重复的blob(文件)

这是对这个问题的后续行动.

如果有多个blob具有相同的内容,它们只会在git存储库中存储一次,因为它们的SHA-1将是相同的.如何找到给定树的所有重复文件？

您是否必须遍历树并查找重复的哈希值,或者git是否提供每个blob的反向链接到引用它的树中的所有文件？

[alias]
    # find duplicate files from root
    alldupes = !"git ls-tree -r HEAD | cut -c 13- | sort | uniq -D -w 40"

    # find duplicate files from the current folder (can also be root)
    dupes = !"cd `pwd`/$GIT_PREFIX && git ls-tree -r HEAD | cut -c 13- | sort | uniq -D -w 40"

在我工作的代码库上运行这个让我大开眼界,我可以告诉你!

#!/usr/bin/perl

# usage: git ls-tree -r HEAD | $PROGRAM_NAME

use strict;
use warnings;

my $sha1_path = {};

while (my $line = ) {
    chomp $line;

    if ($line =~ m{ \A \d+ \s+ \w+ \s+ (\w+) \s+ (\S+) \z }xms) {
        my $sha1 = $1;
        my $path = $2;

        push @{$sha1_path->{$sha1}}, $path;
    }
}

foreach my $sha1 (keys %$sha1_path) {
    if (scalar @{$sha1_path->{$sha1}} > 1) {
        foreach my $path (@{$sha1_path->{$sha1}}) {
            print "$sha1  $path\n";
        }

        print '-' x 40, "\n";
    }
}

只做了一个单行,突出了git呈现的重复ls-tree.
可能有用

git ls-tree -r HEAD |
    sort -t ' ' -k 3 |
    perl -ne '$1 && / $1\t/ && print "\e[0;31m" ; / ([0-9a-f]{40})\t/; print "$_\e[0m"'