Skip to content

Commit 7642fcf

Browse files
committed
backfill: add --batch-size=<n> option
Users may want to specify a minimum batch size for their needs. This is only a minimum: the path-walk API provides a list of OIDs that correspond to the same path, and thus it is optimal to allow delta compression across those objects in a single server request. We could consider limiting the request to have a maximum batch size in the future. Signed-off-by: Derrick Stolee <[email protected]>
1 parent 12da515 commit 7642fcf

File tree

3 files changed

+30
-2
lines changed

3 files changed

+30
-2
lines changed

Documentation/git-backfill.txt

+9-1
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ git-backfill - Download missing objects in a partial clone
99
SYNOPSIS
1010
--------
1111
[verse]
12-
'git backfill' [<options>]
12+
'git backfill' [--batch-size=<n>]
1313

1414
DESCRIPTION
1515
-----------
@@ -38,6 +38,14 @@ delta compression in the packfile sent by the server.
3838
By default, `git backfill` downloads all blobs reachable from the `HEAD`
3939
commit. This set can be restricted or expanded using various options.
4040

41+
OPTIONS
42+
-------
43+
44+
--batch-size=<n>::
45+
Specify a minimum size for a batch of missing objects to request
46+
from the server. This size may be exceeded by the last set of
47+
blobs seen at a given path. Default batch size is 16,000.
48+
4149
SEE ALSO
4250
--------
4351
linkgit:git-clone[1].

builtin/backfill.c

+3-1
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@
2121
#include "path-walk.h"
2222

2323
static const char * const builtin_backfill_usage[] = {
24-
N_("git backfill [<options>]"),
24+
N_("git backfill [--batch-size=<n>]"),
2525
NULL
2626
};
2727

@@ -112,6 +112,8 @@ int cmd_backfill(int argc, const char **argv, const char *prefix, struct reposit
112112
.batch_size = 16000,
113113
};
114114
struct option options[] = {
115+
OPT_INTEGER(0, "batch-size", &ctx.batch_size,
116+
N_("Minimun number of objects to request at a time")),
115117
OPT_END(),
116118
};
117119

t/t5620-backfill.sh

+18
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,24 @@ test_expect_success 'do partial clone 1, backfill gets all objects' '
5959
test_line_count = 0 revs2
6060
'
6161

62+
test_expect_success 'do partial clone 2, backfill batch size' '
63+
git clone --no-checkout --filter=blob:none \
64+
--single-branch --branch=main \
65+
"file://$(pwd)/srv.bare" backfill2 &&
66+
67+
GIT_TRACE2_EVENT="$(pwd)/batch-trace" git \
68+
-C backfill2 backfill --batch-size=20 &&
69+
70+
# Batches were used
71+
test_trace2_data promisor fetch_count 20 <batch-trace >matches &&
72+
test_line_count = 2 matches &&
73+
test_trace2_data promisor fetch_count 8 <batch-trace &&
74+
75+
# No more missing objects!
76+
git -C backfill2 rev-list --quiet --objects --missing=print HEAD >revs2 &&
77+
test_line_count = 0 revs2
78+
'
79+
6280
. "$TEST_DIRECTORY"/lib-httpd.sh
6381
start_httpd
6482

0 commit comments

Comments
 (0)