During a graphical user import with a csv file the UMC throws a traceback and the error message:
ucsschool.http_api.client.ServerError: Received status_code=500 with
reason='Internal Server Error' for
requests.post(**url='https://primary.schule.intranet/api/v1/imports/users/
', data={'dryrun': True, 'school':
'https://primary.schule.intranet/api/v1/schools/schule/', 'user_role':
'student'}, files={'input_file': 'students.csv'}, params=None,
auth=('Administrator', '********'), headers={'Accept': 'application/json'}).
There are no further logs to be found because of this bug and we just know that the POST request fails with a 500 error, which is a generic “catch-all” response to server issues, indicating that the server cannot find a more appropriate 5XX error to respond with.
Problem
We need to get further information to understand what the real cause is. Usually there are logs to be found inside of /var/lib/ucs-school-import/jobs/<year>/<jobid> and /var/log/univention/ucs-school-import, but in this case there is nothing to be found, because the whole process is abrupting so early.
Background
The web server Apache runs a reverse proxy for the URL path /api/. All access to that path is forwarded to Gunicorn, which is a Python web application server, listening on localhost on port 9898 for HTTP connections.
Gunicorn starts the WSGI application returned by ucsschool.http_api.app.wsgi:application(). That is the entry point for the Django web application framework. Django parses and routes requests to the appropriate view/controller functions.
1. Gather more logging information
As the process is stopping so early we need to look at one of the early steps, which might be the web server Apache, the web application server Gunicorn or the web application framework Django. Setting apache2/loglevel to debug didn’t give us any more helpful information, so Gunicorn and Djanog are the next culprits. Those two are intertwined and we can set the following variable to gather more information from them:
ucr set ucsschool/import/http_api/django_debug=yes
Note: This variable should not be set during the production and only for the limited time of reproducing the import error.
To make sure that the change is working we restart the following services:
systemctl restart ucs-school-import-http-api.service ucs-school-import-celery-worker.service
2. Reproduce the error & deactivate debug logging
After activating the django_debug variable, we reproduce the user import error in the UMC and then deactivate the debug logging again:
ucr unset ucsschool/import/http_api/django_debug && systemctl restart ucs-school-import-http-api.service ucs-school-import-celery-worker.service
3. Checking the log files
We can now check the following Gunicorn log files for more information:
/var/log/univention/ucs-school-import/gunicorn_access.log
/var/log/univention/ucs-school-import/gunicorn_error.log
/var/log/univention/ucs-school-import/workers-dryrun.log
/var/log/univention/ucs-school-import/workers-import.log
In our case we found a Traceback in the /var/log/univention/ucs-school-import/gunicorn_error.log file.
4. Examine the Traceback
At the end of the Traceback that we found in step 3 we find the cause for the abrupt stop of the import:
File "/usr/lib/python3/dist-packages/django/core/files/storage.py", line 57, in save
name = self._save(name, content)
File "/usr/lib/python3/dist-packages/django/core/files/storage.py", line 334, in _save
os.makedirs(directory, self.directory_permissions_mode)
File "/usr/lib/python3.7/os.py", line 221, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/var/spool/ucs-school-import/media/uploads/2024-10-16'
As we can see there seems to be a permission error on a directory.
5. Solution
In the configuration file /etc/gunicorn.d/ucs-school-import you can find the user and group that is used by Gunicorn. Per default that should be uas-import. This is important to know for setting the correct permissions.
We can now check the current permissions:
$ ls -lah /var/spool/ucs-school-import/media/
insgesamt 12K
drwxr-xr-x 3 root root 4,0K Mär 25 2024 .
drwxr-xr-x 3 root root 4,0K Mär 25 2024 ..
drwxr-x--- 6 root root 4,0K Okt 16 12:00 uploads
And now we know that the uploads directory permissions are wrong and need to be changed to the user and group uas-import:
chown uas-import:uas-import /var/spool/ucs-school-import/media/uploads/
After setting the correct permissions the graphical user import is working again and proceeds with the POST request and processes the csv file.