lgp: fix sources.list corruption when using lgp shell (closes: #72434)

No more use the OTHERMIRROR env variable due to conflictual combination with
the command line parameter "--othermirror" used in shell command.

New:
- dynamic lgp sources.list entries are now managed by E00lgpmirrors
and write now the entries in the chrooted file:
/etc/apt/sources.list.d/lgp.list
- add explicit message from hook result

Fixes:
- no more override the static mirror list managed by lgp from
/etc/lgp/sources.list.* by using lgp shell
- rewrite shell function merge_other_mirrors() with sed
function is now called merge_lgp_sources_list()
- creation of distribution with non-existent mirror entries is now possible
- drop buggy hooks D90sources and F90sources (replaced by --othermirror option)

Note: a new LGP_OTHERMIRRORS env variable will be visible in chroot shell.

authorJulien Jehannet <julien.jehannet@logilab.fr>
changeset1a064c88dda2
branchdefault
phasepublic
hiddenno
parent revision#5e695b561d98 lgp: fix string expansion in shell command when script are invoked (closes: #72414)
child revision#d596c339fd64 0.18.3
files modified by this revision
doc/lgp/settings.rst
etc/lgp/hooks/D90sources
etc/lgp/hooks/E00lgpmirrors
etc/lgp/hooks/F90sources
etc/lgp/pbuilderrc.sh
etc/lgp/sources.list.debian
etc/lgp/sources.list.ubuntu
# HG changeset patch
# User Julien Jehannet <julien.jehannet@logilab.fr>
# Date 1312532976 -7200
# Fri Aug 05 10:29:36 2011 +0200
# Node ID 1a064c88dda28833edc003db543e1779a8b7d7d3
# Parent 5e695b561d980b93fb6766c0f652cbe675c39c76
lgp: fix sources.list corruption when using lgp shell (closes: #72434)

No more use the OTHERMIRROR env variable due to conflictual combination with
the command line parameter "--othermirror" used in shell command.

New:
- dynamic lgp sources.list entries are now managed by E00lgpmirrors
and write now the entries in the chrooted file:
/etc/apt/sources.list.d/lgp.list
- add explicit message from hook result

Fixes:
- no more override the static mirror list managed by lgp from
/etc/lgp/sources.list.* by using lgp shell
- rewrite shell function merge_other_mirrors() with sed
function is now called merge_lgp_sources_list()
- creation of distribution with non-existent mirror entries is now possible
- drop buggy hooks D90sources and F90sources (replaced by --othermirror option)

Note: a new LGP_OTHERMIRRORS env variable will be visible in chroot shell.

diff --git a/doc/lgp/settings.rst b/doc/lgp/settings.rst
@@ -0,0 +1,39 @@
1 +Settings
2 +========
3 +
4 +Debian and Ubuntu sources.list management
5 +-----------------------------------------
6 +
7 +In some situations, we want to add extra sources.list entries to the chroot.
8 +For example, you want to use acceptance repositories when you build some binary packages.
9 +So, some (build-)dependencies can be found easily.
10 +
11 +Lgp can use sources.list template files with simple scheme to add extra repos.
12 +
13 +/etc/lgp/sources.list.debian:
14 +    for generic Debian distribution
15 +
16 +/etc/lgp/sources.list.debian.${DIST}:
17 +    specific file for the Debian ${DIST} distribution
18 +
19 +/etc/lgp/sources.list.ubuntu:
20 +    for generic Ubuntu distribution
21 +
22 +/etc/lgp/sources.list.ubuntu.${DIST}:
23 +    specific file for the Ubuntu ${DIST} distribution
24 +
25 +Example: you can find the template used at logilab in:
26 +    /usr/share/doc/logilab-packaging/examples/sources.list.debian
27 +
28 +::
29 +
30 +    # Logilab acceptance repository (http://www.logilab.org/card/LogilabDebianRepository)
31 +    #deb http://download.logilab.org/acceptance ${DIST}/
32 +
33 +As you can see, you can use the ${DIST} variable in the template files
34 +instead of a fixed codename. It will be substitued automatically by lgp when
35 +creating or updating an image (chroot).
36 +
37 +The entries found by lgp will be aggregated in the /etc/apt/sources.list/lgp.list
38 +file into the chroot. This action is performed by the `E00lgpmirrors` hook.
39 +
diff --git a/etc/lgp/hooks/D90sources b/etc/lgp/hooks/D90sources
@@ -1,1 +0,0 @@
40 -F90sources
41 \ No newline at end of file
diff --git a/etc/lgp/hooks/E00lgpmirrors b/etc/lgp/hooks/E00lgpmirrors
@@ -0,0 +1,21 @@
42 +#!/bin/bash
43 +
44 +. /tmp/hooks/lgp.functions
45 +
46 +LGP_MIRRORFILE=/etc/apt/sources.list.d/lgp.list
47 +
48 +# exits now if no mirror to add
49 +[ -z "$LGP_OTHERMIRRORS" ] && exit 0
50 +
51 +echo "$LGP_OTHERMIRRORS" > $LGP_MIRRORFILE
52 +
53 +# we have to check immediately that the new mirror list is correct
54 +if apt-get update > /dev/null; then
55 +	print_hook "add new sources.list content in $LGP_MIRRORFILE:\n$LGP_OTHERMIRRORS"
56 +else
57 +	wprint_hook "invalid sources.list content in $LGP_MIRRORFILE:\n$LGP_OTHERMIRRORS"
58 +	echo
59 +	wprint_hook "these entries will be removed."
60 +	rm -f $LGP_MIRRORFILE
61 +fi
62 +
diff --git a/etc/lgp/hooks/F90sources b/etc/lgp/hooks/F90sources
@@ -1,15 +0,0 @@
63 -#!/bin/bash
64 -
65 -. /tmp/hooks/lgp.functions
66 -
67 -echo >> /etc/apt/sources.list
68 -echo "# Uncomment the next line if you want to use your remote repository" >> /etc/apt/sources.list
69 -echo "#deb http://$SUDO_USER:$SUDO_USER@intranet.logilab.fr/~$SUDO_USER/dists $DIST/" >> /etc/apt/sources.list
70 -
71 -if [ -f "$RESULTDIR/Packages.gz" ]; then
72 -	print_hook "Adding a new /etc/apt/sources.list entry for '$RESULTDIR' (used in priority)..."
73 -	echo >> /etc/apt/sources.list.new
74 -	echo "deb file://$(dirname $RESULTDIR) $(basename $RESULTDIR)/" >> /etc/apt/sources.list.new
75 -	cat /etc/apt/sources.list >> /etc/apt/sources.list.new
76 -	mv -f /etc/apt/sources.list.new /etc/apt/sources.list
77 -fi
diff --git a/etc/lgp/pbuilderrc.sh b/etc/lgp/pbuilderrc.sh
@@ -14,18 +14,17 @@
78  	test -f $DEBSCRIPT
79  	test "$(basename $(realpath $DEBSCRIPT))" = "sid"
80  	return $?
81  }
82 
83 -merge_other_mirrors() {
84 -	MIRRORS_FILE=$1
85 -	if [ -f $MIRRORS_FILE.$DIST ]; then
86 -		eval "NEW_MIRRORS=\"$(grep -v '^#\|^$' $MIRRORS_FILE.$DIST | tr '\n' '|')\""
87 -	elif [ -f $MIRRORS_FILE ]; then
88 -		eval "NEW_MIRRORS=\"$(grep -v '^#\|^$' $MIRRORS_FILE | tr '\n' '|')\""
89 +merge_lgp_sources_list() {
90 +	SOURCESLIST=$1
91 +	if [ -f $SOURCESLIST.$DIST ]; then
92 +		sed "s/\${DIST}/${DIST}/g" $SOURCESLIST.$DIST
93 +	elif [ -f $SOURCESLIST ]; then
94 +		sed "s/\${DIST}/${DIST}/g" $SOURCESLIST
95  	fi
96 -	echo "$NEW_MIRRORS"
97  }
98 
99  DEBOOTSTRAP=${DEBOOTSTRAP:-"debootstrap"}
100  : ${DEBOOTSTRAPOPTS:-()}
101  case "${DEBOOTSTRAP}" in
@@ -52,16 +51,16 @@
102  #PBUILDERSATISFYDEPENDSOPT=('--check-key')
103 
104  if is_pure_debian $DIST; then
105  	MIRRORSITE=${DEBIAN_MIRRORSITE}
106  	COMPONENTS=${DEBIAN_COMPONENTS}
107 -	OTHERMIRROR="$OTHERMIRROR|$(merge_other_mirrors $DEBIAN_SOURCESLIST)"
108 +	export LGP_OTHERMIRRORS="$(merge_lgp_sources_list $DEBIAN_SOURCESLIST)"
109  else
110  	MIRRORSITE=${UBUNTU_MIRRORSITE}
111  	COMPONENTS=${UBUNTU_COMPONENTS}
112 -	OTHERMIRROR="$OTHERMIRROR|$(merge_other_mirrors $UBUNTU_SOURCESLIST)"
113  	DEBOOTSTRAPOPTS=("${DEBOOTSTRAPOPTS[@]}" "--keyring=/usr/share/keyrings/ubuntu-archive-keyring.gpg")
114 +	export LGP_OTHERMIRRORS="$(merge_lgp_sources_list $UBUNTU_SOURCESLIST)"
115  fi
116 
117  # Note: don't use DISTRIBUTION directly
118  DISTRIBUTION="${DIST}"
119  #: ${DIST:="unstable"}
diff --git a/etc/lgp/sources.list.debian b/etc/lgp/sources.list.debian
@@ -1,15 +1,10 @@
120  # Debian sources.list template file
121  #
122 -# You have to use the environment variable ${DIST} in this file instead of a
123 -# fixed codename
124 -#
125 -# CAUTION! Be sure the repository exists before creating pbuilder images
126 +# You have to use the environment variable {DIST} in this file instead of a
127 +# fixed codename to permit distribution name substitution
128  #
129 -# The deb lines here are the ones that will appear at the top of the
130 -# sources.list inside the chroot, so this is the place to list your local
131 -# mirror sites; apt will then use them in preference to the ones listed in
132 -# MIRRORSITE
133 -# Internally, lgp set OTHERMIRROR pbuilder variable
134 +# Content of this file will be aggregated to /etc/apt/sources.list.d/lgp.list
135 +# in chroot
136 
137  # Logilab acceptance repository (http://www.logilab.org/card/LogilabDebianRepository)
138  #deb http://download.logilab.org/acceptance ${DIST}/
diff --git a/etc/lgp/sources.list.ubuntu b/etc/lgp/sources.list.ubuntu
@@ -1,17 +1,12 @@
139  # Ubuntu sources.list template file
140  #
141 -# You have to use the environment variable ${DIST} in this file instead of a
142 -# fixed codename
143 -#
144 -# CAUTION! Be sure the repository exists before creating pbuilder images
145 +# You have to use the environment variable {DIST} in this file instead of a
146 +# fixed codename to permit distribution name substitution
147  #
148 -# The deb lines here are the ones that will appear at the top of the
149 -# sources.list inside the chroot, so this is the place to list your local
150 -# mirror sites; apt will then use them in preference to the ones listed in
151 -# MIRRORSITE
152 -# Internally, lgp set OTHERMIRROR pbuilder variable
153 +# Content of this file will be aggregated to /etc/apt/sources.list.d/lgp.list
154 +# in chroot
155 
156  deb http://security.ubuntu.com/ubuntu ${DIST}-security main restricted
157  deb http://archive.ubuntu.com/ubuntu ${DIST}-updates main restricted
158 
159  # Logilab acceptance repository (http://www.logilab.org/card/LogilabDebianRepository)