Files
buildroot/support/download/git
Yann E. MORIN 91e776b5af core/pkg-download: change all helpers to use common options
Currently all download helpers accepts the local output file, the remote
locations, the changesets and so on... as positional arguments.

This was well and nice when that's was all we needed.

But then we added an option to quiesce their verbosity, and that was
shoehorned with a trivial getopts, still keeping all the existing
positional arguments as... positional arguments.

Adding yet more options while keeping positional arguments will not be
very easy, even if we do not envision any new option in the foreseeable
future (but 640K ought to be enough for everyone, remember? ;-) ).

Change all helpers to accept a set of generic options (-q for quiet and
-o for the output file) as well as helper-specific options (like -r for
the repository, -c for a changeset...).

Maxime:
Changed -R to -r for recurse (only for the git backend)
Changed -r to -u for URI (for all backend)
Change -R to -c for cset (for CVS and SVN backend)
Add the export of the BR_BACKEND_DL_GETOPTS so all the backend wrapper
can use the same option easily
Now all the backends use the same common options.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-04-02 12:22:50 +02:00

106 lines
3.7 KiB
Bash
Executable File

#!/usr/bin/env bash
# We want to catch any unexpected failure, and exit immediately
set -e
# Download helper for git, to be called from the download wrapper script
#
# Options:
# -q Be quiet.
# -r Clone and archive sub-modules.
# -o FILE Generate archive in FILE.
# -u URI Clone from repository at URI.
# -c CSET Use changeset CSET.
# -n NAME Use basename NAME.
#
# Environment:
# GIT : the git command to call
verbose=
recurse=0
while getopts "${BR_BACKEND_DL_GETOPTS}" OPT; do
case "${OPT}" in
q) verbose=-q; exec >/dev/null;;
r) recurse=1;;
o) output="${OPTARG}";;
u) uri="${OPTARG}";;
c) cset="${OPTARG}";;
n) basename="${OPTARG}";;
:) printf "option '%s' expects a mandatory argument\n" "${OPTARG}"; exit 1;;
\?) printf "unknown option '%s'\n" "${OPTARG}" >&2; exit 1;;
esac
done
shift $((OPTIND-1)) # Get rid of our options
# Caller needs to single-quote its arguments to prevent them from
# being expanded a second time (in case there are spaces in them)
_git() {
eval ${GIT} "${@}"
}
# Try a shallow clone, since it is faster than a full clone - but that only
# works if the version is a ref (tag or branch). Before trying to do a shallow
# clone we check if ${cset} is in the list provided by git ls-remote. If not
# we fall back on a full clone.
#
# Messages for the type of clone used are provided to ease debugging in case of
# problems
git_done=0
if [ -n "$(_git ls-remote "'${uri}'" "'${cset}'" 2>&1)" ]; then
printf "Doing shallow clone\n"
if _git clone ${verbose} "${@}" --depth 1 -b "'${cset}'" "'${uri}'" "'${basename}'"; then
git_done=1
else
printf "Shallow clone failed, falling back to doing a full clone\n"
fi
fi
if [ ${git_done} -eq 0 ]; then
printf "Doing full clone\n"
_git clone ${verbose} "${@}" "'${uri}'" "'${basename}'"
fi
pushd "${basename}" >/dev/null
# Try to get the special refs exposed by some forges (pull-requests for
# github, changes for gerrit...). There is no easy way to know whether
# the cset the user passed us is such a special ref or a tag or a sha1
# or whatever else. We'll eventually fail at checking out that cset,
# below, if there is an issue anyway. Since most of the cset we're gonna
# have to clone are not such special refs, consign the output to oblivion
# so as not to alarm unsuspecting users, but still trace it as a warning.
if ! _git fetch origin "'${cset}:${cset}'" >/dev/null 2>&1; then
printf "Could not fetch special ref '%s'; assuming it is not special.\n" "${cset}"
fi
# Checkout the required changeset, so that we can update the required
# submodules.
_git checkout -q "'${cset}'"
# Get date of commit to generate a reproducible archive.
# %cD is RFC2822, so it's fully qualified, with TZ and all.
date="$( _git log -1 --pretty=format:%cD )"
# There might be submodules, so fetch them.
if [ ${recurse} -eq 1 ]; then
_git submodule update --init --recursive
fi
# We do not want the .git dir; we keep other .git files, in case they
# are the only files in their directory.
# The .git dir would generate non reproducible tarballs as it depends on
# the state of the remote server. It also would generate large tarballs
# (gigabytes for some linux trees) when a full clone took place.
rm -rf .git
popd >/dev/null
# Generate the archive, sort with the C locale so that it is reproducible
find "${basename}" -not -type d >"${basename}.list"
LC_ALL=C sort <"${basename}.list" >"${basename}.list.sorted"
# Create GNU-format tarballs, since that's the format of the tarballs on
# sources.buildroot.org and used in the *.hash files
tar cf - --numeric-owner --owner=0 --group=0 --mtime="${date}" --format=gnu \
-T "${basename}.list.sorted" >"${output}.tar"
gzip -6 -n <"${output}.tar" >"${output}"