Problem: Internal server error during “schoolimport/dry-run/progress"

Knowledge Base Community
1

Problem:

Internal server error during “schoolimport/dry-run/progress”.
Request: schoolimport/dry-run/progress

RabbitMQ vhost fails to start with not_a_dets_file error

2025-08-12 10:54:29.130 [info] <0.23182.21> accepting AMQP connection <0.23182.21> (127.0.0.1:52162 -> 127.0.0.1:5672)
2025-08-12 10:54:29.159 [error] <0.23182.21> Error on AMQP connection <0.23182.21> (127.0.0.1:52162 -> 127.0.0.1:5672, vhost: 'none', user: 'importhttpapi', state: openin
g), channel 0:
 {handshake_error,opening,
                 {amqp_error,internal_error,
                             "access to vhost 'importhttpapi' refused for user 'importhttpapi': vhost 'importhttpapi' is down",
                             'connection.open'}}
2025-08-12 10:54:29.161 [info] <0.23182.21> closing AMQP connection <0.23182.21

When using RabbitMQ on UCS, a virtual host (vhost) may fail to start. Applications trying to connect to this vhost will see errors like:

[error] Error on AMQP connection (...) vhost 'importhttpapi' is down

The RabbitMQ log shows details similar to:

CRASH REPORT Process <0.24550.21> ... 
reason: no match of right hand value {error,{not_a_dets_file,
"/var/lib/rabbitmq/mnesia/rabbit@server/msg_stores/vhosts/AY5G6ZMFG8M1RI6CZT35GGSPF/recovery.dets"}}
...
Unable to initialize vhost data store for vhost 'importhttpapi'.
The vhost will be stopped for this node.

Cause

RabbitMQ stores persistent message metadata in DETS files (recovery.dets) under the node’s Mnesia data directory.
If this file becomes corrupted (for example due to an unclean shutdown, a full disk, or process termination during writes), the vhost cannot be recovered and is marked as down.

As a result, applications cannot connect to the vhost.

Solution

The corrupted recovery.dets file must be removed. This will delete all unconsumed persistent messages stored in that vhost, but queues and vhost configuration remain intact.

  1. Stop the RabbitMQ application:
rabbitmqctl stop_app
  1. Remove the corrupted recovery file:
rm /var/lib/rabbitmq/mnesia/rabbit@server/msg_stores/vhosts/AY5G6ZMFG8M1RI6CZT35GGSPF/recovery.dets

:warning: Warning: All persistent messages pending in this vhost will be lost.

  1. Start RabbitMQ again:
rabbitmqctl start_app

The vhost should now start normally.
You have additionally to restart the cerlery daemon

service ucs-school-import-celery-worker restart

See also: