Bug 45411 - grub-install fails with PXE
grub-install fails with PXE
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UCS Installer
UCS 4.2
Other Linux
: P3 normal (vote)
: UCS 4.2-3-errata
Assigned To: Philipp Hahn
Richard Ulmer
:
Depends on: 46167
Blocks: 41305
  Show dependency treegraph
 
Reported: 2017-09-18 15:49 CEST by Philipp Hahn
Modified: 2018-06-12 17:14 CEST (History)
5 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 6: Setup Problem: Issue for the setup process
Who will be affected by this bug?: 1: Will affect a very few installed domains
How will those affected feel about the bug?: 2: A Pain – users won’t like this once they notice it
User Pain: 0.069
Enterprise Customer affected?: Yes
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number: https://univention.plan.io/issues/9027,2018051421000311
Bug group (optional):
Max CVSS v3 score:
hahn: Patch_Available+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Philipp Hahn univentionstaff 2017-09-18 15:49:25 CEST
patches/grub-installer/4.2-0-0-ucs/1.117+deb8u/0002-Choose-better-default-for-bootdev.patch is buggy:

 | grep -F -v ${cdsrc:+-e "$(device_to_disk "$cdsrc")"} ${hdsrc:+-e "$(device_to_disk "$hdsrc")"} |

fails when neither "cdsrc" not "hdsrc" is given (|grep -F -v|).
This breaks the PXE installer, where than '(hd0)' is used as the default, which fails.

Just adding a "-e THIS_WILL_NEVER_BE_A_VALID_DEVICE_NAME_TO_FILTER_OUT" is enough.
And while at it, add '-x' to make it a '--line-regexp'.

+++ This bug was initially created as a clone of Bug #42198 +++
Comment 1 Philipp Hahn univentionstaff 2017-12-21 18:31:04 CET
This blocks customer 00026 from migrating to UCS-4.2!
It is not enough to to release grub-installer_*.udeb as errata, as it must be put into 4.2/maintained/4.2-x/$ARCH/, so it gets listed in .../debian-installer/binary-$arch/Packages.
Comment 2 Arvid Requate univentionstaff 2017-12-21 19:39:39 CET
Could you add a ticket number and set flags accordingly? Otherwise this might go unnoticed.
Comment 4 Philipp Hahn univentionstaff 2018-01-18 18:04:14 CET
r17966 | Bug #45411 grub-di: Make sure to always pass at least one expression to `grep`.

Cherry picked package grub-installer[80815] version 1.117+deb8u1 from 4.2-0-0[78]/None[0] to 4.2[78]/errata4.2-3[505]

Package: grub-installer
Version: 1.117+deb8u1A~4.2.0.201801161258
Branch: ucs_4.2-0
Scope: errata4.2-3
"grub-installer" is used by the Debian-Installer.
therefore it is not enough to put it into the component/4.2-3-errata/ directory.
Instead it is patched into the old release:

# Install to apt.knut.univention.de:
cd /var/univention/buildsystem2
install -m 0664 -g buildgroup apt/ucs_4.2-0-errata4.2-3/amd64/grub-installer_* test_mirror/ftp/4.2/maintained/4.2-3/amd64/
install -m 0664 -g buildgroup apt/ucs_4.2-0-errata4.2-3/i386/grub-installer_* test_mirror/ftp/4.2/maintained/4.2-3/i386/
install -m 0664 -g buildgroup apt/ucs_4.2-0-errata4.2-3/source/grub-installer_* test_mirror/ftp/4.2/unmaintained/4.2-3/source/
PYTHONPATH=/usr/bin python -c '
import announce_ucs_release as aur
ran = aur.ReleaseAnnouncer("4.2-3", False)  # public=False → "test_mirror/ftp/"
ran.gen_index()
ran.gen_dists()
ran.generate_release_file("/etc/archive-keys/ucs4.0.txt", "6B8BFD3C")
ran.fix_permissions()'

# Synchronize to updates-test.software-univention.de:
cd /var/univention/buildsystem2
(cd test_mirror/ftp && find 4.2/*maintained/4.2-3 \( -name Packages\* -o -name Sources\* -o -name Release\* -o -name grub-installer\* \) -exec cp -v -u -p --reflink=auto {} ../../mirror/testing/{} \;)
sudo update_mirror.sh -v testing/4.2/{un,}maintained/4.2-3

-rw-r--r-- 1 buildgroup 281372 16. Jan 15:19 mirror/testing/4.2/maintained/4.2-3/amd64/grub-installer_1.117+deb8u1A~4.2.0.201801161258_amd64.udeb
-rw-r--r-- 1 buildgroup 281496 16. Jan 15:19 mirror/testing/4.2/maintained/4.2-3/i386/grub-installer_1.117+deb8u1A~4.2.0.201801161258_i386.udeb
-rw-r--r-- 1 buildgroup   1177 17. Jan 09:35 mirror/testing/4.2/unmaintained/4.2-3/source/grub-installer_1.117+deb8u1A~4.2.0.201801161258.dsc
-rw-r--r-- 1 buildgroup 208384 17. Jan 09:35 mirror/testing/4.2/unmaintained/4.2-3/source/grub-installer_1.117+deb8u1A~4.2.0.201801161258.tar.xz
-rw-r--r-- 1 buildgroup 281372 16. Jan 15:18 test_mirror/ftp/4.2/maintained/4.2-3/amd64/grub-installer_1.117+deb8u1A~4.2.0.201801161258_amd64.udeb
-rw-r--r-- 1 buildgroup 281496 16. Jan 15:18 test_mirror/ftp/4.2/maintained/4.2-3/i386/grub-installer_1.117+deb8u1A~4.2.0.201801161258_i386.udeb
-rw-r--r-- 1 buildgroup   1177 17. Jan 09:35 test_mirror/ftp/4.2/unmaintained/4.2-3/source/grub-installer_1.117+deb8u1A~4.2.0.201801161258.dsc
-rw-r--r-- 1 buildgroup 208384 17. Jan 09:35 test_mirror/ftp/4.2/unmaintained/4.2-3/source/grub-installer_1.117+deb8u1A~4.2.0.201801161258.tar.xz


For QA:

OK: You can use ~phahn/misc/Hackaton-2017-08-25/install-pxe to do a fully automatic PXE installation.

OK: I built a new DVD for internal testing to verify, that grub-installer will work with the next DVD:
  cd /var/univention/buildsystem2/
  repo_admin.py --cherrypick -r 4.2 -s ucs4.2-3 --releasedest 4.2 --dest errata4.2-3 -p debian-installer  # because of new kernel in that scope
  build-cd-ucs4 -r 4.2-3 -s errata4.2-3 -i /var/univention/buildsystem2/apt/ucs_4.2-0-errata4.2-3/ --no-link --no-content-list-update
  isoinfo -R -i ./isotests/ucs_4.2-3-20180118-171740-dvd-amd64.iso -f | grep grub-installer # /amd64/grub-installer_1.117+deb8u1A~4.2.0.201801161258_amd64.udeb
  DVD=./isotests/ucs_4.2-3-20180118-152158-dvd-amd64.iso ~phahn/misc/Hackaton-2017-08-25/install-dvd

FYI: As there is nothing to release as errata, there is no YAML file.

TODO: After QA the files must me copied to mirror/ftp/ as well:
# Install to univention-repository.knut.univention.de:
cd /var/univention/buildsystem2
install -m 0664 -g buildgroup apt/ucs_4.2-0-errata4.2-3/amd64/grub-installer_* mirror/ftp/4.2/maintained/4.2-3/amd64/
install -m 0664 -g buildgroup apt/ucs_4.2-0-errata4.2-3/i386/grub-installer_* mirror/ftp/4.2/maintained/4.2-3/i386/
install -m 0664 -g buildgroup apt/ucs_4.2-0-errata4.2-3/source/grub-installer_* mirror/ftp/4.2/unmaintained/4.2-3/source/
PYTHONPATH=/usr/bin python -c '
import announce_ucs_release as aur
ran = aur.ReleaseAnnouncer("4.2-3", True)  # public=True → "mirror/ftp/"
ran.gen_index()
ran.gen_dists()
ran.generate_release_file("/etc/archive-keys/ucs4.0.txt", "6B8BFD3C")
ran.fix_permissions()'
# Synchronize to updates.software-univention.de
sudo update_mirror.sh -v ftp/4.2/{un,}maintained/4.2-3
Comment 5 Richard Ulmer univentionstaff 2018-05-04 11:56:47 CEST
OK: The changes in the grep command look sensible
OK: The PXE installation works with the updated grub-installer package and doesn't without
-> VERIFIED

By the way: The same changes to the grep command are already in use in UCS 4.3-0.
Comment 6 Philipp Hahn univentionstaff 2018-06-06 17:30:19 CEST
This erratum was never releases: As a .udeb it cannot be published using the normal errata process, but must be build in (or copied to) the next apt/ UCS release scope to be picked up by the DVD build. As the .udeb was never announced as an erratum it was never copied to mirror/ftp/ and never copied back to that release scope during the preparation phase of UCS-4.2-4; as such the .udeb is not on the UCS-4.2-4 DVD, which still contains the old broken version ….201704031545.

Setting up a local repository with the UCS-4.2-4 DVD for a PXE installation is thus currently broken.

The issue was manually fixed at the customer side by doing the following:

 UDEB='amd64/grub-installer_1.117+deb8u1A~4.2.0.201801161258_amd64.udeb'
 cd /var/lib/univention-repository/mirror/4.2/maintained/4.2-3/amd64
 wget http://updates-test.software-univention.de/4.2/maintained/4.2-3/$UDEB
 cd ../..
 apt-ftparchive packages 4.2-4/$UDEB >> 4.2-4/amd64/Packages
 gzip -9 < 4.2-4/amd64/Packages > 4.2-4/amd64/Packages.gz
 python -c 'import univention.updater.mirror as m;m.UniventionMirror(False).update_dists_files()'

As this would breaks as soon as `univention-repository-update net` is invoked (which would pull our Packages files), I also releases the .udeb manually to mirror/ftp/ and re-created the Packages/Release files:

 UDEB='grub-installer_1.117+deb8u1A~4.2.0.201801161258'
 cd $BS2/apt/ucs_4.2-0-ucs4.2-4/
 repo-copy-dsc -vvvv ../ucs_4.2-0-ucs4.2-3/source/${UDEB}.dsc
 repo-apt-ftparchive --release ucs_4.2-0-ucs4.2-4
 TESTFTP='test_mirror/ftp' TESTING='mirror/testing' FTP='mirror/ftp'
 for site in $FTP $TESTFTP
 do
  cd $BS2/$site
  cp -vt 4.2/unmaintained/4.2-4/source/ $OLDPWD/source/${UDEB}.*
  cp -vt 4.2/maintained/4.2-4/amd64/ $OLDPWD/amd64/${UDEB}_amd64.udeb
  cp -vt 4.2/maintained/4.2-4/i386/ $OLDPWD/i386/${UDEB}_i386.udeb
 done
 $TS/REPO/repo-renew-ftp 4.2-4
 cd $BS2
 for part in maintained unmaintained
 do
  rsync --verbose --recursive --links --chmod=D2775,F664 --times --owner --chown=:buildgroup --delete-after $TESTFTP/4.2/$part/4.2-4/ $TESTING/4.2/$part/4.2-4
 done
 ~phahn/misc/REPO/binpkg.py *mirror/*/4.2/*maintained/4.2-4

QA:
<http://xen1.knut.univention.de:8000/packages/source/grub-installer/?since=4.2-0>

$ find *mirror/*/4.2/maintained/4.2-4 \( -name Packages -o -name Sources \) -exec grep-dctrl -l -s Filename \( -w -F Package grub-installer -a -F Version 1.117+deb8u1A~4.2.0.201801161258 \) {} +
mirror/ftp/4.2/maintained/4.2-4/i386/Packages
mirror/ftp/4.2/maintained/4.2-4/amd64/Packages
mirror/ftp/4.2/maintained/4.2-4/dists/ucs424/main/debian-installer/binary-amd64/Packages
mirror/ftp/4.2/maintained/4.2-4/dists/ucs424/main/debian-installer/binary-i386/Packages
mirror/testing/4.2/maintained/4.2-4/i386/Packages
mirror/testing/4.2/maintained/4.2-4/amd64/Packages
mirror/testing/4.2/maintained/4.2-4/dists/ucs424/main/debian-installer/binary-amd64/Packages
mirror/testing/4.2/maintained/4.2-4/dists/ucs424/main/debian-installer/binary-i386/Packages
test_mirror/ftp/4.2/maintained/4.2-4/i386/Packages
test_mirror/ftp/4.2/maintained/4.2-4/amd64/Packages
test_mirror/ftp/4.2/maintained/4.2-4/dists/ucs424/main/debian-installer/binary-amd64/Packages
test_mirror/ftp/4.2/maintained/4.2-4/dists/ucs424/main/debian-installer/binary-i386/Packages

FAIL: currently ftp/{mirror,testing}/4.0/maintained/4.?-?/dists/ucs4??/main/**/Packages are inconsistent and need manual fixing:
for site in $TESTFTP $TESTING $FTP
do
 for part in maintained unmaintained
 do
  ( cd $BS2/$site/4.2/$part/4.2-4 && repo-check-Packages-valid && repo-check-Relese-valid )
 done
done
Comment 7 Philipp Hahn univentionstaff 2018-06-12 17:14:08 CEST
The fixed grub-installer package was also copied to the UCS-4.2-3 release on all sites.

QA: I successfully did a PXE installation with UCS-4.2-[34] from $TESTFTP and $FTP

(In reply to Philipp Hahn from comment #6)
> FAIL: currently
> ftp/{mirror,testing}/4.0/maintained/4.?-?/dists/ucs4??/main/**/Packages are
> inconsistent and need manual fixing:

This is fixed now: Bug #46167

for site in $TESTFTP $TESTING $FTP
do
  for part in maintained unmaintained
  do
   ( cd $BS2/$site/4.2/$part/4.2-4 && repo-check-Packages-valid . && repo-check-Release-valid . )
  done
done