Bug 30547 - Switch to the debian installer
Switch to the debian installer
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UCS Installer
UCS 3.1
Other Linux
: P5 enhancement (vote)
: UCS 4.0
Assigned To: Sönke Schwardt-Krummrich
Stefan Gohmann
: interim-1
: 22488 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2013-02-22 08:42 CET by Stefan Gohmann
Modified: 2014-11-26 06:54 CET (History)
4 users (show)

See Also:
What kind of report is it?: ---
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional): Release Goal, Roadmap discussion
Max CVSS v3 score:


Attachments
PreSeeding example for using Debian installer for UCS (5.22 KB, application/x-cpio)
2013-03-24 20:57 CET, Philipp Hahn
Details
univention-debootstrap cleanup (145.91 KB, patch)
2013-07-15 09:48 CEST, Philipp Hahn
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Stefan Gohmann univentionstaff 2013-02-22 08:42:19 CET
The base installation (partition, network, root password, base installation) should be done by the debian installer. After the base installation is ready the UCS appliance mode should be started.
Comment 1 Philipp Hahn univentionstaff 2013-03-24 20:40:50 CET
The Debian Installer internally uses debootstrap to install the base system, which is not working for UCS-3.x. See Attachment 5143 [details] from Bug #29858 for a working debootstrap-script for UCS-3.0.


Additional issues:

1. The Debian installer expects the repository the be of the "deb $MIRROR $SUITE $COMPONETs..." layout, from where the installer automatically extracts the available suites. For a temporary work-around is sufficient to put two files on the server:

dists/ucs-3.0/Release
 Suite: ucs-3.0
 Codename: Horn-Lehe

dists/ucs-3.0/main/binary-amd64/Release
 Codename: 3.0-0/amd64
 Origin: Univention
 Label: Univention
 Version: 3.0-0
 Architecture: amd64


2. base-installer/library.sh # configure_apt() 
Overwrites the /etc/apt/sources.list file with the Debian layout.
As a work-around the sources.list is corrected as early as possible by a post-run script:

usr/lib/post-base-installer.d/00-recover-apt.sh:
 #!/bin/sh

 . /usr/share/debconf/confmodule
 db_get mirror/protocol
 PROTOCOL="$RET"
 db_get mirror/$PROTOCOL/hostname
 HOSTNAME="$RET"
 db_get mirror/$PROTOCOL/directory
 DIRECTORY="$RET"
 case "${HOSTNAME}${DIRECTORY}" in *:9999/debian) DIRECTORY=/univention ;; esac # FIXME: PHahn private
 db_get mirror/suite
 SUITE="$RET"

 cat  > /target/etc/apt/sources.list <<EOF
 deb ${PROTOCOL}://${HOSTNAME}${DIRECTORY}/${SUITE#ucs-}/maintained ${SUITE#ucs-}-0/\$(ARCH)/
 deb ${PROTOCOL}://${HOSTNAME}${DIRECTORY}/${SUITE#ucs-}/maintained ${SUITE#ucs-}-0/all/
 EOF
 chroot /target apt-get update


3. base-installer/library.sh # kernel_update_list()
Only packages matching "^(linux|kernel)-image" minus "linux-image-2.6" are selectable, which either match all our packages or not univention-kerne-image.
As a work-around the kernel is manually installed:

 #d-i base-installer/kernel/image string univention-kernel-image-2.6.32
 d-i pkgsel/include string univention-kernel-image-2.6.32
 d-i base-installer/kernel/skip-install boolean true


4. UCR templates
UCR is installed without being configured properly.

usr/lib/base-installer.d/40netcfg.ucs:
 #!/bin/sh -e
 mkdir -p /target/etc/univention
 cat >>/target/etc/univention/base.conf <<EOF
 # univention_ base.conf

 $(sed -ne 's/^domain */domainname: /p' /etc/resolv.conf)
 $(sed -ne 's/^nameserver */nameserber1: /p' /etc/resolv.conf)
 hostname: $(hostname)
 interfaces/eth0/type: dhcp
 EOF



5. apt-set/generator/50mirror
kills /etc/apt/sources.list again and fails miserable because the repository is then unreachable.
Fix sources again and replace that script!

usr/lib/apt-setup/generators/49ucs:
  #!/bin/sh

 . /usr/share/debconf/confmodule
 db_get mirror/protocol
 PROTOCOL="$RET"
 db_get mirror/$PROTOCOL/hostname
 HOSTNAME="$RET"
 db_get mirror/$PROTOCOL/directory
 DIRECTORY="$RET"
 case "${HOSTNAME}${DIRECTORY}" in *:9999/debian) DIRECTORY=/univention ;; esac # FIXME: PHahn private
 db_get mirror/suite
 SUITE="$RET"

 cat  >"$1" <<EOF
 deb ${PROTOCOL}://${HOSTNAME}${DIRECTORY}/${SUITE#ucs-}/maintained ${SUITE#ucs-}-0/\$(ARCH)/
 deb ${PROTOCOL}://${HOSTNAME}${DIRECTORY}/${SUITE#ucs-}/maintained ${SUITE#ucs-}-0/all/
 EOF

 rm /usr/lib/apt-setup/generators/50mirror
 ln -s /bin/true /usr/lib/apt-setup/generators/50mirror


6, /etc/default/grub
The initrd argument from the installer is stored as the default command line argument.
  d-i grub2/linux_cmdline string verbose


With these modifications I'm now able to fully automatically install a UCS system using the Debian Installer from Debian 7.0 (Wheezy), Installer build: 20130211


After the installation the VM failed to boot into UCS with the following error:
  Unaligned pointer 0x91fc0002
This looks like to be a bug in our version of grub2 or with QEMU, when using QXL:
 <https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/717445>
 <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=616487>
After changing QEMU to use a Cirrus-VGA, the VM boots fine.
Comment 2 Philipp Hahn univentionstaff 2013-03-24 20:57:03 CET
Created attachment 5144 [details]
PreSeeding example for using Debian installer for UCS

0. Save this attachment to TFTP directory as preseed.cfg.cpio
1. Download <http://ftp.nl.debian.org/debian/dists/testing/main/installer-amd64/current/images/netboot/netboot.tar.gz> into TFTP directory
2. Modify pxelinux.cfg/default to
 prompt 0
 timeout 0
 label ucs
        menu label ^Automated install
        kernel debian-installer/amd64/linux
        append auto=true priority=critical vga=788 -- quiet 
        initrd debian-installer/amd64/initrd.gz,preseed.cfg.cpio
 default ucs
3. Configure DHCP and TFTP server (using libvirt):
 <network>
  <name>default</name>
  <forward mode='nat'/>
  <bridge name='virbr0' stp='off' delay='0' />
  <mac address='52:54:00:50:29:71'/>
  <ip address='192.168.100.1' netmask='255.255.255.0'>
    <tftp root='/srv/debian/installer' />
    <dhcp>
      <range start='192.168.100.10' end='192.168.100.254' />
      <bootp file='pxelinux.0' />
    </dhcp>
  </ip>
 </network>
Comment 3 Stefan Gohmann univentionstaff 2013-04-03 05:55:04 CEST
*** Bug 22488 has been marked as a duplicate of this bug. ***
Comment 4 Stefan Gohmann univentionstaff 2013-04-15 07:51:21 CEST
See also Bug #31005.
Comment 5 Stefan Gohmann univentionstaff 2013-06-11 10:20:23 CEST
We decided to do the switch at a later point, for example UCS 3.3 or UCS 4.0
Comment 6 Stefan Gohmann univentionstaff 2013-07-12 15:42:49 CEST
(In reply to Philipp Hahn from comment #1)
> The Debian Installer internally uses debootstrap to install the base system,
> which is not working for UCS-3.x. See Attachment 5143 [details] from Bug
> #29858 for a working debootstrap-script for UCS-3.0.

Does univention-debootstrap-3 work?

I think the Debian Installer should only do some basics:
 - Network configuration
 - Partitioning / grub
 - root Password
 - Installation of base packages, UCR, UMC
 - prepare the system that UMC system setup can install the role packages

After the Debian Installer has finished the system should boot directly in UMC system setup appliance mode. Maybe the system can start a browser directly if the X version of the Debian Installer was started otherwise the appliance mode can be used remote.
The network configuration and the root password should be skipped in the appliance mode otherwise these settings are asked twice.
Comment 7 Philipp Hahn univentionstaff 2013-07-15 09:44:31 CEST
(In reply to Stefan Gohmann from comment #6)
> (In reply to Philipp Hahn from comment #1)
> Does univention-debootstrap-3 work?

It's used to create the UCS-3.x DVDs (branches/ucs-3.0/internal/repo-ng/build-cd/build-cd-ng-3.1), so somehow it does, but:

1. Why is there a nearly empty "univention-debootstrap" package?
# apt-cache policy univention-debootstrap-3 univention-debootstrap
univention-debootstrap-3:
  Installed: 4.0.3-5.45.201202231555
  Candidate: 4.0.3-5.45.201202231555
  Version table:
 *** 4.0.3-5.45.201202231555 0
        500 http://apt.knut.univention.de/3.0/maintained/ 3.0-1/all/ Packages
        100 /var/lib/dpkg/status
     4.0.3-4.43.201111081035 0
        500 http://apt.knut.univention.de/3.0/maintained/ 3.0-0/all/ Packages
univention-debootstrap:
  Installed: 4.0.0-1.34.201105130727
  Candidate: 4.0.0-1.34.201105130727
  Version table:
 *** 4.0.0-1.34.201105130727 0
        500 http://apt.knut.univention.de/3.0/maintained/ 3.0-0/all/ Packages
        100 /var/lib/dpkg/status


2. It only works with our internal repository (and perhaps the everything-in-one-directory DVD format):
FAIL:
 /usr/sbin/debootstrap-ucs3 ucs3 ./foo http://apt.knut.univention.de/
 /usr/sbin/debootstrap-ucs3 ucs3 ./foo http://apt.knut.univention.de/3.1/
 /usr/sbin/debootstrap-ucs3 ucs3 ./foo http://apt.knut.univention.de/3.1/maintained/
 /usr/sbin/debootstrap-ucs3 ucs3 ./foo http://apt.knut.univention.de/3.1/maintained/amd64/
 I: Retrieving Packages.gz
 I: Retrieving Packages
 I: Resolving dependencies of required packages...
 I: Resolving dependencies of base packages...
 I: Chosen extractor for .deb packages: dpkg-deb
 W: Failure trying to run: chroot /root/foo mount -t proc proc /proc

OK:
 /usr/sbin/debootstrap-ucs3 ucs3 ./foo http://omar.knut.univention.de/build2/ucs_3.1-0

It should either be fixed to work for external users too, or be moved to some internal-only location.

3. There a 3 identical copies of the shell function library:
6bc5ad6c045edc31e9623ea52a875bd5  ./usr/share/debootstrap-ucs3/functions-ucs3
6bc5ad6c045edc31e9623ea52a875bd5  ./usr/share/debootstrap-ucs3/functions
6bc5ad6c045edc31e9623ea52a875bd5  ./usr/share/debootstrap/functions-ucs3
which seem to be identical to the original one from Squeeze deboostrap.

4. debootstrap-ucs3 (1.0.26.26.201105181428) is an exact copy of debootsrap from Debian Squeeze. (UCC has and needs the newer version from Debian Wheezy 1.0.42.29.201210191222).

After fixing the "ucs3" script to not include functions-ucs3, the original program works fine:
/usr/sbin/debootstrap ucs3 ./foo http://omar.knut.univention.de/build2/ucs_3.1-0 /usr/share/debootstrap-ucs3/scripts/ucs3


> I think the Debian Installer should only do some basics:
+1
Comment 8 Philipp Hahn univentionstaff 2013-07-15 09:48:36 CEST
Created attachment 5322 [details]
univention-debootstrap cleanup
Comment 9 Stefan Gohmann univentionstaff 2014-01-14 20:20:12 CET
Quote from 2014010921001095:

> Software raid installation is a lot of manual work.

This will be fixed with the switch to the debian installer.
Comment 10 Moritz Muehlenhoff univentionstaff 2014-06-18 10:26:33 CEST
The debian-installer package was omitted during the Wheezy import since we'll likely start with a more recent version of the installer.
Comment 11 Philipp Hahn univentionstaff 2014-08-20 10:52:34 CEST
r52865 | Bug #30547: Install recommends by default
r52864 | Bug #35378 CD: Install recommends by default
  Install recommends by default

Package: univention-dvd
Version: 0.2.0-7.9.201408201044
Branch: ucs_4.0-0
Comment 12 Sönke Schwardt-Krummrich univentionstaff 2014-08-26 10:22:39 CEST
The new DVD is using the d-i → RESOLVED
Comment 13 Philipp Hahn univentionstaff 2014-08-26 20:40:49 CEST
r53055 | Bug #30547 repo: Fix dpkg-name *.udeb rename
 dpkg-name renames .udeb to .deb - undo that
Comment 14 Stefan Gohmann univentionstaff 2014-08-29 10:07:46 CEST
I've created a new bug for the net installer: Bug #35745

Installation looks good so far. There are several separate bugs tagged for MS2.

Changelog: OK
Comment 15 Stefan Gohmann univentionstaff 2014-11-26 06:54:11 CET
UCS 4.0-0 has been released:
 http://docs.univention.de/release-notes-4.0-0-en.html
 http://docs.univention.de/release-notes-4.0-0-de.html

If this error occurs again, please use "Clone This Bug".