Univention Bugzilla – Bug 45645
[4.1r2] Teachers collect data from themselves, quickly multiplying the folder size
Last modified: 2017-12-21 12:25:13 CET
Merge changes to 4.1r2. -------------------------------------------------------------------- +++ This bug was initially created as a clone of Bug #45572 +++ When a teacher collects a project with a class as member, files will be collected from every user inside that class. This unfortunately includes his own project folder, which contains every previously collected version of this project. If this teacher now collects a project multiple times, the size of the newly collected project versions will multiply, as they include all previously collected project versions inside the teachers project folder. The collecting teacher shouldn't collect his own folder content. ------------------- Simple example, assuming no one touches any files: 1 Class containing 2 Students and 1 Teacher 1 Project with that Class as member and 1 Project File, Size: 1Mb Teacher distributes project: All class members (3) receive one 1Mb file Teacher collects project: Teacher collects two 1Mb files from every member excluding himself Teacher then collects his own folder including one 1Mb file and all folders from other members :Total Teacher size: 1Mb + 2Mb + 3Mb = 6Mb Teacher collects project a second time: Teacher collects two 1Mb files from every member excluding himself Teacher then collects his own folder including one 1Mb file, his first collection of members (2Mb), his second collection of members (2Mb) his first collection of himself (3Mb) and his new collection of himself (8Mb) :Total Teacher size: 1Mb + 2Mb + 2Mb + 3Mb + 8Mb = 16Mb The folder would have a size of 36Mb after a third collection. Graphical Display after second collection: [16.0M] ├── [1.0M] FILE ├── [3.0M] teacher1 │ ├── [1.0M] FILE │ ├── [1.0M] bschwartz │ └── [1.0M] testStudent ├── [8.0M] teacher1 version2 │ ├── [1.0M] FILE │ ├── [3.0M] teacher1 │ │ ├── [1.0M] FILE │ │ ├── [1.0M] bschwartz │ │ └── [1.0M] testStudent │ ├── [1.0M] bschwartz │ ├── [1.0M] bschwartz version2 │ ├── [1.0M] testStudent │ └── [1.0M] testStudent version2 ├── [1.0M] bschwartz ├── [1.0M] bschwartz version2 ├── [1.0M] testStudent └── [1.0M] testStudent version2 --------------------
The teachers home directory is now skipped, when collecting distributed materials. Cherry-picked from 4.2: commit abae615691d927bf0dc1738f17d94ca81a668619 Bug #45645: advisory commit d3641c20febe7ba085c8c8ba190ce0cc11178b90 Bug #45645: advisory commit 3a0a40d74863cbf81e4d9a25f459e8e266198dff Bug #45572: advisory commit f71d7a56bafeb3c063dd129053dda05dddef3cc8 Bug #45645: don't collect teachers own data ucs-school-umc-distribution 14.0.2-4.89.201711031537
If missing, please add a note in the UCS@school teacher manual, that clearly states, that files of users with role teacher are never collected.
[4.1r2 39196da4] Bug #45645: mention that teachers file are not collected Last sentence of chapter 3.7.3: http://jenkins.knut.univention.de:8080/job/UCSschool%204.1/job/UCSschool%204.1%20R2%20Manual/10/artifact/webroot/ucsschool-lehrer-handbuch-4.1r2.pdf
Commit 724a09c1: store and compare school roles instead of DNs Commit 35382053: update advisory
Copied comment from bug 45572: ------------------------------ (In reply to Florian Best from comment #7) > So the behavior now is: "Teacher" and "Teachers and Staff" materials aren't > collected. "Students" and "Staff" materials are collected. Staff users are usually not replicated to the educational UCS@school servers (→ Edu DC Slave). I.e. only files of "student" users are collected. (In reply to Florian Best from comment #8) > The behavior is not consistent anymore. Why are the files distributed to > teachers and teachers+staff but not collected from them? They should not be > distributed at all then. Good question. We initiated a new discussion with ProfServices and Development and we all think, that the current approach does not exactly hit the requirements. The current approach will prevent the collection of files distributed to other teachers. Instead, a new solution has been discussed: --- a/ucs-school-umc-distribution/umc/python/distribution/util.py +++ b/ucs-school-umc-distribution/umc/python/distribution/util.py @@ -54,6 +54,7 @@ _ = Translation('ucs-school-umc-distribution').translate DISTRIBUTION_CMD = '/usr/lib/ucs-school-umc-distribution/umc-distribution' DISTRIBUTION_DATA_PATH = ucr.get('ucsschool/datadistribution/cache', '/var/lib/ucs-school-umc-distribution') +POSTFIX_DATADIR_SENDER_PROJECT_SUFFIX = ucr.get('ucsschool/datadistribution/datadir/sender/project/suffix', '-Ergebnisse') POSTFIX_DATADIR_SENDER = ucr.get('ucsschool/datadistribution/datadir/sender', 'Unterrichtsmaterial') POSTFIX_DATADIR_RECIPIENT = ucr.get('ucsschool/datadistribution/datadir/recipient', 'Unterrichtsmaterial') @@ -264,7 +265,7 @@ class Project(_Dict): def sender_projectdir(self): '''The absolute path of the project directory in the senders home.''' if self.sender and self.sender.homedir: - return os.path.join(self.sender.homedir, POSTFIX_DATADIR_SENDER, self.name) + return os.path.join(self.sender.homedir, POSTFIX_DATADIR_SENDER, '%s%s' % (self.name, POSTFIX_DATADIR_SENDER_PROJECT_SUFFIX)) return None @property → files are distributed to $HOME/Unterrichtsmaterial/${PROJECTNAME}/ → files are collected to $HOME/Unterrichtsmaterial/${PROJECTNAME}-Ergebnisse/ → this should prevent the exponential collection loop → REOPEN → please revert the modifications done so far and apply the patch above A hotfix is currently also possible: Use different directory names in "ucsschool/datadistribution/datadir/sender" and "ucsschool/datadistribution/datadir/recipient". Currently both default to "Unterrichtsmaterial".
The previous commits were reverted: commit 14c57a45f89e8555dab6a437dae91bf266dcb666 Revert "Bug #45645: store and compare school roles instead of DNs" This reverts commit 724a09c181085c47e157ff6691a1c2a264297b5b. commit 9f4ff232225c8beef686d77ac56140589c29f05c Revert "Bug #45645: mention that teachers file are not collected" This reverts commit 39196da4e78e21a837e056b52378c652dce9e410. commit 10429905e71b3b5c51148a96bc3de1e47d332ebc Revert "Bug #45645: don't collect teachers own data" This reverts commit f71d7a56bafeb3c063dd129053dda05dddef3cc8. And the new UCRV ucsschool/datadistribution/datadir/sender/project/suffix introduced (cherry-picked from 4.2). commit 47104296ab90976f777cd29682a3caa7823df625 Bug #45645: separate project distribution and collection directories And explained in the manual (cherry-picked from 4.2). commit 1670dfe35d4d29ba25673aefb070918e29a375f6 Bug #45572: literal and anvar cannot be children of footnote commit 3ab50683518efc47841a871c6ac6a87e4832a4ad Bug #45572: explain UCR in manual Changelog and advisory: commit 61ee9e655e5402df31d035f21d91ad34b765d0cd Bug #45645: advisory update commit 08435b84b3a940110e4d4c5af7bcbd8be001b17b Bug #45645: changelog and advisory ucs-school-umc-distribution 14.0.2-6.91.201711221148
3.7.3. Einsammeln der verteilten Dokumente: http://jenkins.knut.univention.de:8080/job/UCSschool%204.1/job/UCSschool%204.1%20R2%20Manual/20/artifact/webroot/ucsschool-lehrer-handbuch-4.1r2.pdf
OK: merge OK: no UCR variable description (as other UCR variables aren't described either)
UCS@school 4.1 R2 v16 has been released. http://docs.software-univention.de/changelog-ucsschool-4.1R2v16-de.html If this error occurs again, please clone this bug.