Bug 41305 - Use updates.software-univention.de for PXE installation
Use updates.software-univention.de for PXE installation
Status: CLOSED FIXED
Product: UCS extended documentation
Classification: Unclassified
Component: Installation
unspecified
Other Linux
: P5 normal with 2 votes (vote)
: UCS 4.3
Assigned To: Philipp Hahn
Jannik Ahlers
:
Depends on: 38507 43837 45185 45411
Blocks: 45400 48123
  Show dependency treegraph
 
Reported: 2016-05-20 09:56 CEST by Philipp Hahn
Modified: 2019-02-14 12:13 CET (History)
5 users (show)

See Also:
What kind of report is it?: Development Internal
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?: Yes
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Philipp Hahn univentionstaff 2016-05-20 09:56:59 CEST
Our public repository <https://updates.software-univention.de/> is missing several files, which prevents it from being used as the site for a PXE network installation:
- check for missing .udeb (Bug #38507 fixed that but they're missing for all previous releases; as such most .udebs are still missing as they haven't been re-built and re-announced)
- check for dists/binary-$arch/Packages*
- check for dists/debian-installer/binary-$arch/Packages
- what else?

Currently a local repository must be created, which cost time and space and prevents a fast install of demo instances.
Comment 1 Philipp Hahn univentionstaff 2016-08-29 06:03:40 CEST
Missed once again during UCS Technical training 2016-08 Task #4773

(In reply to Philipp Hahn from comment #0)
> - what else?

- Put kernel and initramfs with installer somewhere - only latest or all UCS releases
- Optional: nice iPXE boot menu to select version
Comment 2 Philipp Hahn univentionstaff 2016-11-20 21:44:50 CET
r74489 | Bug #41305 DVD: Put USS onto DVD, not into InitRamFS
r74488 | Bug #41305: Register key for (PXE) installation
r16882 | Bug #41305 d-i: PXE installation
r74491 | Bug #41305 DVD: Copyright
r16883 | Bug #41305 pxe: Use univention-archive-key
r74521 | Bug #41305: Register key for (PXE) installation

Package: univention-dvd
Version: 2.0.0-4A~4.2.0.201611161523
Branch: ucs_4.2-0

Package: univention-archive-key
Version: 7.0.0-1A~4.2.0.201611161527
Version: 7.0.0-2A~4.2.0.201611161532
Version: 7.0.0-3A~4.2.0.201611161535
Branch: ucs_4.2-0

Package: base-installer
Version: 1.154A~4.2.0.201611161546
Branch: ucs_4.2-0

r74615 | Bug #41305 repo: Move preseed.cfg from InitRamFS to /cdrom/
r74614 | Bug #41305 repo: Always set suite to use for d-i
r74613 | Bug #41305 repo: Add option -d to make URLs relative


1. You need local DHCP (and TFTP) servers
   univention-install isc-dhcp-server tftpd-hpa ipxe
2. If your client are too old and do not support downloading via http://, configure chain-loading a newer version of the iPXE client as described in <http://ipxe.org/howto/dhcpd#pxe_chainloading>
  cat >> /etc/dhcp/local.conf <<__CONF__
if exists user-class and option user-class = "iPXE" {
      filename "http://updates.software-univention.de/pxe/4.1-4/amd64/pxelinux.0";
  } else {
      filename "undionly.kpxe";
      nextserver IP-OF-YOUR-TFTP-SERVER;
  }
__CONF__
3. cp
Comment 3 Philipp Hahn univentionstaff 2017-09-05 18:33:31 CEST
(In reply to Philipp Hahn from comment #2)
> 3. cp

ssh omar
cd /var/univention/buildsystem2
for arch in amd64 i386
do
 install -d mirror/ftp/pxe/4.2-2/$arch
 tar xfC apt/ucs_4.2-0-ucs4.2-2/dists/ucs422/main/installer-$arch/current/images/netboot/netboot.tar.gz  mirror/ftp/pxe/4.2-2/$arch/
done
sudo update_mirror.sh -v ftp/pxe/4.2-2

r82495 | Bug #41305 repo: Preserve hard-links
r82494 | Bug #41305 repo: Generate Release files for PXE

<https://hutten.knut.univention.de/mediawiki/index.php/Philipp_memo/Release#PXE-Installation_aktualisieren>

Preseed via "preseed/url" is sometimes fails in Debian-Jessie, as 1.66 does include <https://anonscm.debian.org/cgit/d-i/preseed.git/commit/?id=f2e7bae02caec5a66e25d5af7aab90eb7648deca>, but not <https://anonscm.debian.org/cgit/d-i/preseed.git/commit/debian-installer-startup.d/S60auto-install?id=51ecf03340777b71caaebe00dd05ec17d0c729cc>.
Comment 4 Philipp Hahn univentionstaff 2017-09-18 15:49:47 CEST
FYI: Since Bug #38507 'announce_ucs_release' announces all *.udeb from apt/ to ftp/. They are
- included in /X.Y/[un]maintained/X.Y-Z/$ARCH/Packages (BAD, not required)
- included in /X.Y/[un]maintained/X.Y-Z/dists/ucsXYZ/main/binary-$ARCH/Packages (BAD, not required)
- included in /X.Y/[un]maintained/X.Y-Z/dists/ucsXYZ/main/debian-installer/binary-$ARCH/Packages (GOOD, required)

In contrast to apt/, there they are
- included in ucs_X.Y-0[ucsX.Y-Z]/$ARCH/Packages (OKAY, as only used internally)
- excluded from ucs_X.Y-0[ucsX.Y-Z]/dists/ucsXYZ/main/binary-$ARCH/Packages (GOOD, not required)
- included in ucs-X.Y-0[ucsX.Y-Z]/dists/ucsXYZ/main/debian-installer/binary-$ARCH/Packages (GOOD, required)

(.udeb are stripped down .deb packages, which are missing several files to make them smaller. They are NOT supposed to be installed by hand, only by the Debian-Installer)

Currently 'create_dists_packages' uses the first directory, to create the later two, filtering in/out .debs accordingly.
- For apt/ this works by design, as there all .udeb are in the Packages files.
- For ftp/ this currently only works, as we don't (yet) filter out .deb there (but we should).
Comment 6 Jannik Ahlers univentionstaff 2019-02-04 11:29:04 CET
The Debian Installer uses a DHCP request to query the preseed file, even if a path to a preseed is provided by "d-i preseed/url".
UCS answers this request with the bootloader once again, which results in an error in the debian installer.
Comment 7 Jannik Ahlers univentionstaff 2019-02-04 11:49:35 CET
This is only a problem if you try to deliver the bootloader via http. The deb installer assumes a preseed file only if the dhcp request returns an url.

udm policies/dhcp_boot modify \
	--dn "cn=default-settings,cn=boot,cn=dhcp,cn=policies,$LDAP" \
	--set boot_filename="$HTTP" \ <----- setting this to an url causes the error
	--set boot_server=
Comment 9 Philipp Hahn univentionstaff 2019-02-04 13:12:54 CET
(In reply to Jannik Ahlers from comment #6)
> The Debian Installer uses a DHCP request to query the preseed file, even if
> a path to a preseed is provided by "d-i preseed/url".
> UCS answers this request with the bootloader once again, which results in an
> error in the debian installer.

debian-installer/packages/preseed/preseed.sh:173
173 dhcp_preseed_url () {
...
177 »···»···»···if echo "$FN" | grep -q "://" ; then

So the script treats any "DHCP option bootfile-name" containing a "://" as a valid URL to a "preseed.cfg" file :-(
See <https://www.debian.org/releases/stretch/amd64/apbs02.html.en#preseed-dhcp>

An extra step need to be performed: <http://jenkins.knut.univention.de:8080/view/Doku/job/BuildDocBookBranch/53/artifact/webroot/installation-4.3.html#profile:netinstaller:public>

[phahn/41305-pxe] 0705c64923 squash! Bug #41305: Document PXE installation
 doc/extended-docs/installation-4.3.xml | 31 ++++++++++++++++++++++++++-----
 1 file changed, 26 insertions(+), 5 deletions(-)
Comment 10 Jannik Ahlers univentionstaff 2019-02-05 12:26:24 CET
I fixed some smaller mistakes and typos:
7a38b2d Bug #41305: fix typos in PXE documentation

- The TEMPLATE file is zipped by default, this is not mentioned anywhere in the documentation, maybe we should give a hint
- In the template, the value
    'd-i mirror/http/directory string /univention-repository/4.3/maintained/4.3-3'
  is wrong, it should be:
    'd-i mirror/http/directory string /4.3/maintained/4.3-3'
- The last two paragraphs are almost identical, one of them should be removed (the ones talking about the reinstall option)

Other than that, everything works as expected.
Comment 11 Philipp Hahn univentionstaff 2019-02-06 08:27:05 CET
(In reply to Jannik Ahlers from comment #10)
> I fixed some smaller mistakes and typos:
> 7a38b2d Bug #41305: fix typos in PXE documentation

Thanks.

> - The TEMPLATE file is zipped by default, this is not mentioned anywhere in
> the documentation, maybe we should give a hint

This is an unfortunate side effect of `dh_installexamples` and more specifically `dh_compress`, which by default compresses files in /usr/share/doc/$package/. This needs to be fixed in the package "services/univention-net-installer", which is out-of-scope for this bug. I added a footnote.

> - In the template, the value
>     'd-i mirror/http/directory string
> /univention-repository/4.3/maintained/4.3-3'
>   is wrong, it should be:
>     'd-i mirror/http/directory string /4.3/maintained/4.3-3'

Yes: the template was for the use with a "local" repository; I've added a paragraph to the documentation that this setting MUST be changed.

> - The last two paragraphs are almost identical, one of them should be
> removed (the ones talking about the reinstall option)

I merged them into one.


[phahn/41305-pxe] d2923a6974 Bug #41305 pxe: QA fixes
 doc/extended-docs/installation-4.3.xml | 26 +++++++++++++++++---------
 1 file changed, 17 insertions(+), 9 deletions(-)

<http://jenkins.knut.univention.de:8080/view/Doku/job/BuildDocBookBranch/54/artifact/webroot/installation-4.3.html#profile:netinstaller:public>


> Other than that, everything works as expected.

Thank you for testing.
After successful QA please REOPEN so I can merge the change into 4.3-3 and 4.4-0.
Comment 12 Jannik Ahlers univentionstaff 2019-02-06 09:20:46 CET
Ok, everything looks good now.
Comment 14 Philipp Hahn univentionstaff 2019-02-14 12:13:03 CET
[master] dd16110 Bug #46874,Bug #32277,Bug #36733,Bug #39556,Bug #40162,Bug #41684,Bug #34726,Bug #41305,Bug #36869: PUBLISH