Uploading files to app center fails

Apps & App Center App Development
#1

Hi,

for the last days I’ve been trying to upload multiple .deb files to the app center with the CLI tool.
Files are uploaded one after another through a little script.

Unfortunately all attempts fail sooner or later during like this:

Executing upload: univention-appcenter-control upload 4.3/opsi=4.1.1.9-7 /tmp/ucs-app-center-deploy/ucs_4.3/opsi_41/testing/winexe_4.8.0.0-19_amd64.deb
Curling https://provider-portal.software-univention.de/univention/auth
Curling https://provider-portal.software-univention.de/univention/command/appcenter-selfservice/api
Curling https://provider-portal.software-univention.de/univention/command/appcenter-selfservice/query
Unrecoverable result: {u'status': 510, u'message': u'', u'traceback': None, u'location': u'https://provider-portal.software-univention.de/univention/command'}

Sometimes it is another error like this:

Unrecoverable result: {u'status': 591, u'message': u'Internal server error during "appcenter-selfservice/upload".', u'traceback': u'Internal server error during "appcenter-selfservice/upload".\nRequest: appcenter-selfservice/upload\n\nTraceback (most recent call last):\n  File "/usr/lib/pymodules/python2.7/univention/management/console/base.py", line 260, in execute\n    function.__func__(self, request, *args, **kwargs)\n  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/decorators.py", line 609, in _response\n    return function(self, request)\n  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/appcenter-selfservice/__init__.py", line 552, in upload\n    self._run_appcenter_action(\'selfservice-add-file\', app=app, files=files, force=force, clear_packages=clear)\n  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/appcenter-selfservice/__init__.py", line 336, in _run_appcenter_action\n    return action.call(**kwargs)\n  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/__init__.py", line 220, in call\n    return obj.call_with_namespace(namespace)\n  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/__init__.py", line 226, in call_with_namespace\n    result = self.main(namespace)\n  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/selfservice.py", line 639, in main\n    SelfservicePackages.call(app=app, add=packages, force=args.force, clear=args.clear_packages)\n  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/__init__.py", line 220, in call\n    return obj.call_with_namespace(namespace)\n  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/__init__.py", line 226, in call_with_namespace\n    result = self.main(namespace)\n  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/selfservice.py", line 465, in main\n    populate_args = populate._build_namespace(packages=args.add, do_not_delete_duplicates=not args.clear)\n  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/__init__.py", line 194, in _build_namespace\n    self.setup_parser(parser)\n  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/local_appcenter.py", line 360, in setup_parser\n    arch = subprocess.check_output([\'uname\', \'-m\']).strip()\n  File "/usr/lib/python2.7/subprocess.py", line 212, in check_output\n    process = Popen(stdout=PIPE, *popenargs, **kwargs)\n  File "/usr/lib/python2.7/subprocess.py", line 390, in __init__\n    errread, errwrite)\n  File "/usr/lib/python2.7/subprocess.py", line 916, in _execute_child\n    self.pid = os.fork()\nOSError: [Errno 12] Cannot allocate memory', u'location': u'https://provider-portal.software-univention.de/univention/upload'}

What can be done to make the uploads work?

#2

Okay, here is a new error message for you to enjoy:

Unrecoverable result: {u'status': 400, u'message': u'There is not enough free space on disk', u'traceback': None, u'location': u'https://provider-portal.software-univention.de/univention/upload'}

Would be nice if someone would take care of this!

#3

Hello @opsi4ucs,

We took care about this by changing the monitoring limits for the disk space and the storage “architecture” which will allow us to react faster on small disk space. I hope this problem will not occur again in the future and our IT will take care of it before you see such an error.

Best regards,
Nico

#4

Hi Nico,

uploading is still not possible for all the files I want to update. Some files have uploaded without problems but then this came up:

univention-appcenter-control upload 4.3/opsi=4.1.1.9-7 /tmp/ucs-app-center-deploy/ucs_4.3/opsi_41/testing/wimtools_1.9.2-1_amd64.deb
Curling https://provider-portal.software-univention.de/univention/auth
Curling https://provider-portal.software-univention.de/univention/command/appcenter-selfservice/api
Curling https://provider-portal.software-univention.de/univention/command/appcenter-selfservice/query
Uploading /tmp/ucs-app-center-deploy/ucs_4.3/opsi_41/testing/wimtools_1.9.2-1_amd64.deb
Curling https://provider-portal.software-univention.de/univention/upload/appcenter-selfservice/upload
Unrecoverable result: {u'status': 591, u'message': u'Internal server error during "appcenter-selfservice/upload".', u'traceback': u'Internal server error during "appcenter-selfservice/upload".\nRequest: appcenter-selfservice/upload\n\nTraceback (most recent call last):\n  File "/usr/lib/pymodules/python2.7/univention/management/console/base.py", line 260, in execute\n    function.__func__(self, request, *args, **kwargs)\n  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/decorators.py", line 609, in _response\n    return function(self, request)\n  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/appcenter-selfservice/__init__.py", line 552, in upload\n    self._run_appcenter_action(\'selfservice-add-file\', app=app, files=files, force=force, clear_packages=clear)\n  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/appcenter-selfservice/__init__.py", line 336, in _run_appcenter_action\n    return action.call(**kwargs)\n  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/__init__.py", line 220, in call\n    return obj.call_with_namespace(namespace)\n  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/__init__.py", line 226, in call_with_namespace\n    result = self.main(namespace)\n  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/selfservice.py", line 639, in main\n    SelfservicePackages.call(app=app, add=packages, force=args.force, clear=args.clear_packages)\n  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/__init__.py", line 220, in call\n    return obj.call_with_namespace(namespace)\n  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/__init__.py", line 226, in call_with_namespace\n    result = self.main(namespace)\n  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/selfservice.py", line 465, in main\n    populate_args = populate._build_namespace(packages=args.add, do_not_delete_duplicates=not args.clear)\n  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/__init__.py", line 194, in _build_namespace\n    self.setup_parser(parser)\n  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/local_appcenter.py", line 360, in setup_parser\n    arch = subprocess.check_output([\'uname\', \'-m\']).strip()\n  File "/usr/lib/python2.7/subprocess.py", line 212, in check_output\n    process = Popen(stdout=PIPE, *popenargs, **kwargs)\n  File "/usr/lib/python2.7/subprocess.py", line 390, in __init__\n    errread, errwrite)\n  File "/usr/lib/python2.7/subprocess.py", line 916, in _execute_child\n    self.pid = os.fork()\nOSError: [Errno 12] Cannot allocate memory', u'location': u'https://provider-portal.software-univention.de/univention/upload'}

Judging from the message Cannot allocate memory I’d say the server is lacking RAM.

#5

Hi,

every time a new univention-appcenter-control is run, it takes a considerable amount of RAM on the server. It will be freed eventually, but if run too frequently, might ending up eating all the RAM.

The reason the script is called so many times is that we also have an upload limit per request. opsi cannot upload all packages at once.

I put a new version of the script online. You may want to test uploading *.deb again with one command. It will reuse one session and upload as many times as it needs to with a certain upload limit.

I have tested it with my local installation of the Provider Portal and it worked fine. I hope it helps.

Kind regards,
Dirk

#6

Hi Dirk,

I used this new version to upload all *.deb files at once and this worked!
Thank you!

Mastodon