Import user from ucs@school administration

Problem The import of users fails with the following error message:

Internal server error during “schoolimport/dry-run/progress”. Internal server error during “schoolimport/dry-run/progress”.
Request: schoolimport/dry-run/progress

Traceback (most recent call last):
File “/usr/lib/python2.7/dist-packages/univention/management/console/base.py”, line 358, in __error_handling
six.reraise(etype, exc, etraceback)
File “/usr/lib/python2.7/dist-packages/univention/management/console/base.py”, line 261, in execute
function.func(self, request, args, **kwargs)
File “/usr/lib/python2.7/dist-packages/univention/management/console/modules/decorators.py”, line 639, in _decorated
return function(self, request, args, **kwargs)
File “/usr/lib/python2.7/dist-packages/univention/management/console/modules/decorators.py”, line 321, in _response
result = _multi_response(self, request)
File “/usr/lib/python2.7/dist-packages/univention/management/console/modules/decorators.py”, line 181, in _response
return function(self, request)
File “/usr/lib/python2.7/dist-packages/univention/management/console/modules/decorators.py”, line 443, in _response
return list(function(self, iterator, nones))
File “/usr/lib/python2.7/dist-packages/univention/management/console/modules/decorators.py”, line 289, in _fake_func
yield function(self, args)
File “/usr/lib/pymodules/python2.7/univention/management/console/modules/schoolimport/init.py”, line 209, in poll_dry_run
return progress.poll()
File “/usr/lib/python2.7/dist-packages/univention/management/console/modules/mixins.py”, line 102, in poll
six.reraise(self.exc_info[1], None, self.exc_info[2])
File “”, line 2, in reraise
ServerError: Received status_code=500 with reason=‘Internal Server Error’ for requests.post(**files={u’input_file’: (u’1579711422-esportazioni-23-11-2019-pwd-ok.txt’, <closed file ‘’, mode ‘’ at 0x7fc786648db0>, ‘text/plain’)}, url=u’https://pdc.morgagniwifi.it/api/v1/imports/users/’, auth=(u’Administrator’, u’
******’), headers={u’Accept’: u’application/json’}, params=None, data={u’school’: u’https://pdc.morgagniwifi.it/api/v1/schools/LiceoMorgagni/’, u’dryrun’: True, u’user_role’: u’teacher’}).

Please report your UCS and UCS@school versions. Run: univention-app info
Please have a look at the follwing logfiles:

  • /var/lib/ucs-school-import/jobs/2020/*/ucs-school-import.log
  • /var/log/univention/ucs-school-import/workers-*.log
  • /var/log/univention/ucs-school-import/gunicorn_*.log

Greetings
Daniel Tröder

UCS: 4.4-3 errata427
UCS@School 4.4 V4

the 2020 folder is not created and log files are not written and date back to 2019

workers-import.log:
[2020-01-23 20:43:49,214: ERROR/MainProcess] consumer: Cannot connect to amqp://importhttpapi:**@127.0.0.1:5672/importhttpapi: [Errno 111] Connection refused.
Trying again in 32.00 seconds…

workers-dryrun.log:
[2020-01-23 21:01:31,279: ERROR/MainProcess] consumer: Cannot connect to amqp://importhttpapi:**@127.0.0.1:5672/importhttpapi: [Errno 111] Connection refused.
Trying again in 32.00 seconds…

guicorn_error.log:
[2020-01-23 20:17:03 +0000] [3354] [DEBUG] GET /v1/
[2020-01-23 20:18:00 +0000] [877] [CRITICAL] WORKER TIMEOUT (pid:3354)
[2020-01-23 20:18:04 +0000] [3354] [INFO] Worker exiting (pid: 3354)
[2020-01-23 20:18:06 +0000] [28696] [INFO] Booting worker with pid: 28696
[2020-01-23 20:39:31 +0000] [3356] [DEBUG] GET /v1/
[2020-01-23 20:40:12 +0000] [3360] [DEBUG] GET /v1/schools/
[2020-01-23 20:40:39 +0000] [28696] [DEBUG] GET /v1/imports/users/
[2020-01-23 20:40:42 +0000] [3360] [DEBUG] GET /v1/schools/LiceoMorgagni/

guicorn_access.log:
127.0.0.1 - Administrator [23/Jan/2020:20:40:09 +0100] “GET /v1/ HTTP/1.1” 200 177 “-” “python-requests/2.12.4”
127.0.0.1 - Administrator [23/Jan/2020:20:40:39 +0100] “GET /v1/schools/ HTTP/1.1” 200 173 “-” “python-requests/2.12.4”
127.0.0.1 - Administrator [23/Jan/2020:20:40:42 +0100] “GET /v1/imports/users/?ordering=date_created&dryrun=False&limit=20 HTTP/1.1” 200 20099 “-” “python-requests/2.12.4”
127.0.0.1 - Administrator [23/Jan/2020:20:40:42 +0100] “GET /v1/schools/LiceoMorgagni/ HTTP/1.1” 200 282 “-” “python-requests/2.12.4”
127.0.0.1 - Administrator [23/Jan/2020:20:40:43 +0100] “GET /v1/schools/LiceoMorgagni/ HTTP/1.1” 200 282 “-” “python-requests/2.12.4”

if necessary I can also provide the complete files

Thank you very much
Donato Giano

This looks like the RabbitMQ server cannot be contacted. Please check / restart it:

service rabbitmq-server status
service rabbitmq-server restart

Then restart the Import-Job-Queue:

service celery-worker-ucsschool-import status
service celery-worker-ucsschool-import restart

Greetings
Daniel Tröder

Thanks a lot now it works

Thanks
Donated Janus

Hi@all,

sry for posting on an old thread but I have a similar problem and the solution posted here does not solve it :frowning:

**/var/log/univention/management-console-module-schoolimport.log**

22.07.21 11:43:39.915  MODULE      ( PROCESS ) : Interner Server-Fehler in "schoolimport/schools".
22.07.21 11:43:40.024  MODULE      ( PROCESS ) : Der UCS@school Import API HTTP Server ist nicht erreichbar: Received status_code=500 with reason='Internal Server Error' for requests.get(**files=None, url=u'https://******/api/v1/imports/users/', auth=(u'Administrator', u'*********************'), headers={u'Accept': u'application/json'}, params={'ordering': 'date_created', 'limit': 20, 'dryrun': False}, data=None).

When I try to open the user-import module in umc I’m receiving the following traceback

Interner Server-Fehler in "schoolimport/schools".
Request: schoolimport/schools

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/univention/management/console/base.py", line 359, in __error_handling
    six.reraise(etype, exc, etraceback)
  File "/usr/lib/python2.7/dist-packages/univention/management/console/base.py", line 262, in execute
    function.__func__(self, request, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/decorators.py", line 641, in _decorated
    return function(self, request, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/decorators.py", line 321, in _response
    result = _multi_response(self, request)
  File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/decorators.py", line 181, in _response
    return function(self, request)
  File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/decorators.py", line 443, in _response
    return list(function(self, iterator, *nones))
  File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/decorators.py", line 289, in _fake_func
    yield function(self, *args)
  File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/schoolimport/__init__.py", line 104, in schools
    dict(id=school.name, label=school.displayName) for school in self.client.school.list()
  File "/usr/lib/python2.7/dist-packages/ucsschool/http_api/client.py", line 593, in list
    return self._to_python(self._list_resource(**params))
  File "/usr/lib/python2.7/dist-packages/ucsschool/http_api/client.py", line 551, in _list_resource
    return self._resource_from_url(resource_url, **params)
  File "/usr/lib/python2.7/dist-packages/ucsschool/http_api/client.py", line 543, in _resource_from_url
    return self.client.call_api("get", url, params=params)
  File "/usr/lib/python2.7/dist-packages/ucsschool/http_api/client.py", line 524, in call_api
    raise exc(msg, status_code=response.status_code)
ServerError: Received status_code=500 with reason='Internal Server Error' for requests.get(**files=None, url=u'https://******/api/v1/schools/', auth=(u'Administrator', u'*********************'), headers={u'Accept': u'application/json'}, params={}, data=None).

Any hints on whats going wrong here?

br,
Manuel

Although the error looks similar, it is often best to create a new thread instead of posting in an old one.

Regarding the error message: When was the system installed? We had a small time window where a bug caused an incomplete initialization of a background database for the school import, see Bug 53599

Please test if re-running the joinscript fixes the issue:
univention-run-join-scripts --run-scripts 40ucs-school-import-http-api.inst --force

Although the error looks similar, it is often best to create a new thread instead of posting in an old one.

sry for that :worried:

Regarding the error message: When was the system installed? We had a small time window where a bug caused an incomplete initialization of a background database for the school import, see Bug 53599

I installed this system 2 weeks ago on July 6th.

Please test if re-running the joinscript fixes the issue:
univention-run-join-scripts --run-scripts 40ucs-school-import-http-api.inst --force

Unfortunately the behaviour stays the same. Tried it with restarting the relevant services as described here Problem: Nach dem Installieren kann das Import Modul nicht aufgerufen werden

br,
Manuel

Hi Manuel,

but you have installed the latest errata updates? The errata updates fixes this issue, but you also have to rerun the joinscript and restart the services.

Hi @scheinig,

actually I didn’t since I set up the system but I have now. It didn’t change the behaviour.

br,
Manuel

Hi Manuel,

and after installing the latest errata updates you also reran the joinscript and restarted the services?

Hi @scheinig,

yes, I did as you describe in your solution I mentioned.

univention-run-join-scripts --run-scripts 40ucs-school-import-http-api.inst --force
systemctl restart celery-worker-ucsschool-import.service 
systemctl restart celeryd.service
systemctl restart ucs-school-import-http-api.service
systemctl restart rabbitmq-server.service

br,
Manuel

Hi Manuel,

it is possible for you to restart the entire Server?
And is the message still the same?
You are using a user with the permission to import?

Hi @scheinig,

I did already restart the whole server just to be sure.
Yes, the message stays the same.
Yes, the user should have the permission to import (group import_all).

I restarted again just to be sure - same error :frowning:

br,
Manuel

Hi Manuel,

the joinscript ran successfully?
univention-check-join-status
shows successful?

This a singleserverschool, not a multi school environment?
Can you please provide the output of

dpkg -l |grep ucs-school-import-http-api

python /usr/lib/python2.7/dist-packages/ucsschool/http_api/manage.py shell -c \
    "from ucsschool.http_api.import_api.models import School; print School.objects.all()"
service apache2 status
service ucs-school-import-http-api status
service postgresql status
service rabbitmq-server status
service celery-worker-ucsschool-import status
univention-ldapsearch -LLL objectClass=ucsschoolOrganizationalUnit ou
Mastodon