Samba-tool drs showrepl is trying to sync with a non existing Neighbour WERR_FILE_NOT_FOUND INBOUND and OUTBOUND

On a setup with master, backup and 2 slaves (bl-dcsz, bl-dchb)
samba-tool drs showrepl is telling me that an old entry for bl-dcsz cant be reached. (there was an issue with bl-dcsz in the past where the system was rejoined)

so, bl-dcsz itself is a real and working system and only the master is shwong me that error, on all other system the sync is fine

This seems to be an old record “f2a7b9b1-a542-4c29-b73b-ab9ed509f3ce” as you can see in this replication status

root@master:~# samba-tool drs showrepl
Default-First-Site-Name\MASTER
DSA Options: 0x00000001
DSA object GUID: 8f32029d-1f23-4fea-b52d-75aa32e6da6a
DSA invocationId: 91faf128-5655-420e-bbc1-7b637cd473fb

==== INBOUND NEIGHBORS ====

CN=Schema,CN=Configuration,DC=bbc,DC=local
	Default-First-Site-Name\BACKUP via RPC
		DSA object GUID: 97bf73d4-6800-4f2e-b0a8-e140b713abf0
		Last attempt @ Mon May 17 10:15:28 2021 CEST was successful
		0 consecutive failure(s).
		Last success @ Mon May 17 10:15:28 2021 CEST

CN=Schema,CN=Configuration,DC=bbc,DC=local
	Default-First-Site-Name\BL-DCSZ via RPC
		DSA object GUID: f2a7b9b1-a542-4c29-b73b-ab9ed509f3ce
		Last attempt @ Mon May 17 10:15:28 2021 CEST failed, result 2 (WERR_FILE_NOT_FOUND)
		32450 consecutive failure(s).
		Last success @ Wed Jan  6 15:49:18 2021 CET

CN=Schema,CN=Configuration,DC=bbc,DC=local
	Default-First-Site-Name\BL-DCHB via RPC
		DSA object GUID: 56f17b42-44b3-4fb9-b4c8-f2e9b8280725
		Last attempt @ Mon May 17 10:15:28 2021 CEST was successful
		0 consecutive failure(s).
		Last success @ Mon May 17 10:15:28 2021 CEST

DC=bbc,DC=local
	Default-First-Site-Name\BACKUP via RPC
		DSA object GUID: 97bf73d4-6800-4f2e-b0a8-e140b713abf0
		Last attempt @ Mon May 17 10:15:28 2021 CEST was successful
		0 consecutive failure(s).
		Last success @ Mon May 17 10:15:28 2021 CEST

DC=bbc,DC=local
	Default-First-Site-Name\BL-DCSZ via RPC
		DSA object GUID: f2a7b9b1-a542-4c29-b73b-ab9ed509f3ce
		Last attempt @ Mon May 17 10:17:00 2021 CEST failed, result 2 (WERR_FILE_NOT_FOUND)
		32813 consecutive failure(s).
		Last success @ Wed Jan  6 15:49:19 2021 CET

DC=bbc,DC=local
	Default-First-Site-Name\BL-DCHB via RPC
		DSA object GUID: 56f17b42-44b3-4fb9-b4c8-f2e9b8280725
		Last attempt @ Mon May 17 10:17:00 2021 CEST was successful
		0 consecutive failure(s).
		Last success @ Mon May 17 10:17:00 2021 CEST

CN=Configuration,DC=bbc,DC=local
	Default-First-Site-Name\BACKUP via RPC
		DSA object GUID: 97bf73d4-6800-4f2e-b0a8-e140b713abf0
		Last attempt @ Mon May 17 10:15:28 2021 CEST was successful
		0 consecutive failure(s).
		Last success @ Mon May 17 10:15:28 2021 CEST

CN=Configuration,DC=bbc,DC=local
	Default-First-Site-Name\BL-DCSZ via RPC
		DSA object GUID: f2a7b9b1-a542-4c29-b73b-ab9ed509f3ce
		Last attempt @ Mon May 17 10:15:28 2021 CEST failed, result 2 (WERR_FILE_NOT_FOUND)
		32453 consecutive failure(s).
		Last success @ Wed Jan  6 15:49:19 2021 CET

CN=Configuration,DC=bbc,DC=local
	Default-First-Site-Name\BL-DCHB via RPC
		DSA object GUID: 56f17b42-44b3-4fb9-b4c8-f2e9b8280725
		Last attempt @ Mon May 17 10:15:28 2021 CEST was successful
		0 consecutive failure(s).
		Last success @ Mon May 17 10:15:28 2021 CEST

DC=DomainDnsZones,DC=bbc,DC=local
	Default-First-Site-Name\BACKUP via RPC
		DSA object GUID: 97bf73d4-6800-4f2e-b0a8-e140b713abf0
		Last attempt @ Mon May 17 10:15:28 2021 CEST was successful
		0 consecutive failure(s).
		Last success @ Mon May 17 10:15:28 2021 CEST

DC=DomainDnsZones,DC=bbc,DC=local
	Default-First-Site-Name\BL-DCSZ via RPC
		DSA object GUID: f2a7b9b1-a542-4c29-b73b-ab9ed509f3ce
		Last attempt @ Mon May 17 10:15:28 2021 CEST failed, result 2 (WERR_FILE_NOT_FOUND)
		32821 consecutive failure(s).
		Last success @ Wed Jan  6 15:49:18 2021 CET

DC=DomainDnsZones,DC=bbc,DC=local
	Default-First-Site-Name\BL-DCHB via RPC
		DSA object GUID: 56f17b42-44b3-4fb9-b4c8-f2e9b8280725
		Last attempt @ Mon May 17 10:15:28 2021 CEST was successful
		0 consecutive failure(s).
		Last success @ Mon May 17 10:15:28 2021 CEST

DC=ForestDnsZones,DC=bbc,DC=local
	Default-First-Site-Name\BACKUP via RPC
		DSA object GUID: 97bf73d4-6800-4f2e-b0a8-e140b713abf0
		Last attempt @ Mon May 17 10:15:28 2021 CEST was successful
		0 consecutive failure(s).
		Last success @ Mon May 17 10:15:28 2021 CEST

DC=ForestDnsZones,DC=bbc,DC=local
	Default-First-Site-Name\BL-DCSZ via RPC
		DSA object GUID: f2a7b9b1-a542-4c29-b73b-ab9ed509f3ce
		Last attempt @ Mon May 17 10:15:28 2021 CEST failed, result 2 (WERR_FILE_NOT_FOUND)
		32527 consecutive failure(s).
		Last success @ Wed Jan  6 15:49:18 2021 CET

DC=ForestDnsZones,DC=bbc,DC=local
	Default-First-Site-Name\BL-DCHB via RPC
		DSA object GUID: 56f17b42-44b3-4fb9-b4c8-f2e9b8280725
		Last attempt @ Mon May 17 10:15:28 2021 CEST was successful
		0 consecutive failure(s).
		Last success @ Mon May 17 10:15:28 2021 CEST

==== OUTBOUND NEIGHBORS ====

CN=Schema,CN=Configuration,DC=bbc,DC=local
	Default-First-Site-Name\BL-DCSZ via RPC
		DSA object GUID: f2a7b9b1-a542-4c29-b73b-ab9ed509f3ce
		Last attempt @ Mon May 17 10:16:58 2021 CEST failed, result 2 (WERR_FILE_NOT_FOUND)
		57 consecutive failure(s).
		Last success @ NTTIME(0)

CN=Schema,CN=Configuration,DC=bbc,DC=local
	Default-First-Site-Name\BL-DCHB via RPC
		DSA object GUID: 56f17b42-44b3-4fb9-b4c8-f2e9b8280725
		Last attempt @ NTTIME(0) was successful
		0 consecutive failure(s).
		Last success @ NTTIME(0)

CN=Schema,CN=Configuration,DC=bbc,DC=local
	Default-First-Site-Name\BACKUP via RPC
		DSA object GUID: 97bf73d4-6800-4f2e-b0a8-e140b713abf0
		Last attempt @ NTTIME(0) was successful
		0 consecutive failure(s).
		Last success @ NTTIME(0)

DC=bbc,DC=local
	Default-First-Site-Name\BL-DCSZ via RPC
		DSA object GUID: f2a7b9b1-a542-4c29-b73b-ab9ed509f3ce
		Last attempt @ Mon May 17 10:16:58 2021 CEST failed, result 2 (WERR_FILE_NOT_FOUND)
		57 consecutive failure(s).
		Last success @ NTTIME(0)

DC=bbc,DC=local
	Default-First-Site-Name\BL-DCHB via RPC
		DSA object GUID: 56f17b42-44b3-4fb9-b4c8-f2e9b8280725
		Last attempt @ NTTIME(0) was successful
		0 consecutive failure(s).
		Last success @ NTTIME(0)

DC=bbc,DC=local
	Default-First-Site-Name\BACKUP via RPC
		DSA object GUID: 97bf73d4-6800-4f2e-b0a8-e140b713abf0
		Last attempt @ NTTIME(0) was successful
		0 consecutive failure(s).
		Last success @ NTTIME(0)

CN=Configuration,DC=bbc,DC=local
	Default-First-Site-Name\BL-DCSZ via RPC
		DSA object GUID: f2a7b9b1-a542-4c29-b73b-ab9ed509f3ce
		Last attempt @ Mon May 17 10:16:58 2021 CEST failed, result 2 (WERR_FILE_NOT_FOUND)
		57 consecutive failure(s).
		Last success @ NTTIME(0)

CN=Configuration,DC=bbc,DC=local
	Default-First-Site-Name\BL-DCHB via RPC
		DSA object GUID: 56f17b42-44b3-4fb9-b4c8-f2e9b8280725
		Last attempt @ NTTIME(0) was successful
		0 consecutive failure(s).
		Last success @ NTTIME(0)

CN=Configuration,DC=bbc,DC=local
	Default-First-Site-Name\BACKUP via RPC
		DSA object GUID: 97bf73d4-6800-4f2e-b0a8-e140b713abf0
		Last attempt @ NTTIME(0) was successful
		0 consecutive failure(s).
		Last success @ NTTIME(0)

DC=DomainDnsZones,DC=bbc,DC=local
	Default-First-Site-Name\BL-DCSZ via RPC
		DSA object GUID: f2a7b9b1-a542-4c29-b73b-ab9ed509f3ce
		Last attempt @ Mon May 17 10:16:58 2021 CEST failed, result 2 (WERR_FILE_NOT_FOUND)
		116 consecutive failure(s).
		Last success @ NTTIME(0)

DC=DomainDnsZones,DC=bbc,DC=local
	Default-First-Site-Name\BL-DCHB via RPC
		DSA object GUID: 56f17b42-44b3-4fb9-b4c8-f2e9b8280725
		Last attempt @ NTTIME(0) was successful
		0 consecutive failure(s).
		Last success @ NTTIME(0)

DC=DomainDnsZones,DC=bbc,DC=local
	Default-First-Site-Name\BACKUP via RPC
		DSA object GUID: 97bf73d4-6800-4f2e-b0a8-e140b713abf0
		Last attempt @ NTTIME(0) was successful
		0 consecutive failure(s).
		Last success @ NTTIME(0)

DC=ForestDnsZones,DC=bbc,DC=local
	Default-First-Site-Name\BL-DCSZ via RPC
		DSA object GUID: f2a7b9b1-a542-4c29-b73b-ab9ed509f3ce
		Last attempt @ Mon May 17 10:16:58 2021 CEST failed, result 2 (WERR_FILE_NOT_FOUND)
		57 consecutive failure(s).
		Last success @ NTTIME(0)

DC=ForestDnsZones,DC=bbc,DC=local
	Default-First-Site-Name\BL-DCHB via RPC
		DSA object GUID: 56f17b42-44b3-4fb9-b4c8-f2e9b8280725
		Last attempt @ NTTIME(0) was successful
		0 consecutive failure(s).
		Last success @ NTTIME(0)

DC=ForestDnsZones,DC=bbc,DC=local
	Default-First-Site-Name\BACKUP via RPC
		DSA object GUID: 97bf73d4-6800-4f2e-b0a8-e140b713abf0
		Last attempt @ NTTIME(0) was successful
		0 consecutive failure(s).
		Last success @ NTTIME(0)

==== KCC CONNECTION OBJECTS ====

Connection --
	Connection name: 78febdf6-55e3-4791-bbc9-31aaeacb72b2
	Enabled        : TRUE
	Server DNS name : backup.bbc.local
	Server DN name  : CN=NTDS Settings,CN=BACKUP,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=bbc,DC=local
		TransportType: RPC
		options: 0x00000001
Warning: No NC replicated for Connection!
Connection --
	Connection name: 961bcdc1-7a44-4730-b165-12c95445cd58
	Enabled        : TRUE
	Server DNS name : bl-dcsz.bbc.local
	Server DN name  : CN=NTDS Settings,CN=BL-DCSZ,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=bbc,DC=local
		TransportType: RPC
		options: 0x00000001
Warning: No NC replicated for Connection!
Connection --
	Connection name: e02ef902-799a-4838-8cc2-84db42d26519
	Enabled        : TRUE
	Server DNS name : bl-dchb.bbc.local
	Server DN name  : CN=NTDS Settings,CN=BL-DCHB,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=bbc,DC=local
		TransportType: RPC
		options: 0x00000001
Warning: No NC replicated for Connection!

i dont even see this record samba is trying to reach in the s4search “f2a7b9b1-a542-4c29-b73b-ab9ed509f3ce”

root@master:~# univention-s4search --cross-ncs cn=bl-dcsz
# record 1
dn: CN=BL-DCSZ,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=bbc,DC=local
objectClass: top
objectClass: server
cn: BL-DCSZ
instanceType: 4
whenCreated: 20201218135708.0Z
uSNCreated: 6260
showInAdvancedViewOnly: TRUE
name: BL-DCSZ
objectGUID: 03685289-157a-467f-969f-f2d6fb1320d1
systemFlags: 1375731712
dNSHostName: bl-dcsz.bbc.local
objectCategory: CN=Server,CN=Schema,CN=Configuration,DC=bbc,DC=local
serverReference: CN=BL-DCSZ,OU=Domain Controllers,DC=bbc,DC=local
whenChanged: 20201218140542.0Z
uSNChanged: 6264
distinguishedName: CN=BL-DCSZ,CN=Servers,CN=Default-First-Site-Name,CN=Sites,C
 N=Configuration,DC=bbc,DC=local

# record 2
dn: CN=bl-dcsz,OU=technische-user,OU=intern,OU=best-blu,DC=bbc,DC=local
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: user
cn: bl-dcsz
sn: dcsz
description: VPN User DC Salzgitter
instanceType: 4
whenCreated: 20201218120033.0Z
displayName: dcsz
uSNCreated: 5704
name: bl-dcsz
objectGUID: 3499054f-ae71-4d56-9788-3f185e550c19
codePage: 0
countryCode: 0
primaryGroupID: 513
objectSid: S-1-5-21-3172901005-1270155125-13220644-1256
sAMAccountName: bl-dcsz
sAMAccountType: 805306368
userPrincipalName: bl-dcsz@BBC.LOCAL
lockoutTime: 0
objectCategory: CN=Person,CN=Schema,CN=Configuration,DC=bbc,DC=local
uidNumber: 2080
gidNumber: 5001
unixHomeDirectory: /home/bl-dcsz
loginShell: /bin/bash
servicePrincipalName: DNS/SLAVE.tech.schein.ig
userAccountControl: 66048
whenChanged: 20201218133443.0Z
uSNChanged: 6242
pwdLastSet: 132527720790000000
accountExpires: 9223372036854775807
distinguishedName: CN=bl-dcsz,OU=technische-user,OU=intern,OU=best-blu,DC=bbc,
 DC=local

# record 3
dn: CN=BL-DCSZ,OU=Domain Controllers,DC=bbc,DC=local
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: user
objectClass: computer
cn: BL-DCSZ
instanceType: 4
whenCreated: 20201218135708.0Z
displayName: BL-DCSZ$
uSNCreated: 6255
name: BL-DCSZ
objectGUID: fd83c4fe-9d40-4301-b543-2a0d24ad6bb5
userAccountControl: 532480
codePage: 0
countryCode: 0
primaryGroupID: 516
objectSid: S-1-5-21-3172901005-1270155125-13220644-1270
accountExpires: 9223372036854775807
sAMAccountName: BL-DCSZ$
sAMAccountType: 805306369
dNSHostName: bl-dcsz.bbc.local
rIDSetReferences: CN=RID Set,CN=BL-DCSZ,OU=Domain Controllers,DC=bbc,DC=local
objectCategory: CN=Computer,CN=Schema,CN=Configuration,DC=bbc,DC=local
isCriticalSystemObject: TRUE
msDS-SupportedEncryptionTypes: 31
serverReferenceBL: CN=BL-DCSZ,CN=Servers,CN=Default-First-Site-Name,CN=Sites,C
 N=Configuration,DC=bbc,DC=local
operatingSystem: Univention Corporate Server
operatingSystemVersion: 4.4-7
pwdLastSet: 132641247173228630
lastLogonTimestamp: 132654578995476070
servicePrincipalName: HOST/BL-DCSZ
servicePrincipalName: HOST/bl-dcsz.bbc.local
servicePrincipalName: GC/bl-dcsz.bbc.local/bbc.local
servicePrincipalName: HOST/bl-dcsz.bbc.local/BBC
servicePrincipalName: ldap/bl-dcsz.bbc.local/BBC
servicePrincipalName: ldap/bl-dcsz.bbc.local
servicePrincipalName: HOST/bl-dcsz.bbc.local/bbc.local
servicePrincipalName: ldap/bl-dcsz.bbc.local/bbc.local
servicePrincipalName: ldap/BL-DCSZ
servicePrincipalName: RestrictedKrbHost/BL-DCSZ
servicePrincipalName: RestrictedKrbHost/bl-dcsz.bbc.local
servicePrincipalName: ldap/bl-dcsz.bbc.local/DomainDnsZones.bbc.local
servicePrincipalName: ldap/bl-dcsz.bbc.local/ForestDnsZones.bbc.local
whenChanged: 20210517082204.0Z
uSNChanged: 8068
distinguishedName: CN=BL-DCSZ,OU=Domain Controllers,DC=bbc,DC=local

# returned 3 records
# 3 entries
# 0 referrals


also here i dont see the entry

root@master:~# univention-s4search --cross-ncs -b CN=BL-DCSZ,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=bbc,DC=local dn
# record 1
dn: CN=701c7293-150c-4f2a-9254-1093f9cd7bce,CN=NTDS Settings,CN=BL-DCSZ,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=bbc,DC=local

# record 2
dn: CN=4c085d7b-9188-4860-9a96-9c763a7c7bd4,CN=NTDS Settings,CN=BL-DCSZ,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=bbc,DC=local

# record 3
dn: CN=3ac1f4d5-706d-4ef6-953c-b812c15622a7,CN=NTDS Settings,CN=BL-DCSZ,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=bbc,DC=local

# record 4
dn: CN=BL-DCSZ,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=bbc,DC=local

# record 5
dn: CN=NTDS Settings,CN=BL-DCSZ,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=bbc,DC=local

# returned 5 records
# 5 entries
# 0 referrals
root@master:~# 

I dont understand where samba is getting that information to contact that inbound/outbound neighbour

Does someone has any ideas where to find the record in AD samba is trying to use in this case?

Mastodon