Symptoms

A customer registered a domain through Odin Business Automation Standard (OBAS) or added a domain that had been registered elsewhere. The correct DNS zone was created in OBAS, which can be seen in the customer's control panel and in the Provider's control panel in Domain Manager.

The domain has the DNS hosting service enabled. However, no DNS zone was created on the name servers.

This article describes steps for troubleshooting the problem with DNS synchronization between OBAS itself and SSH-managed name servers registered in OBAS.

Resolution

In general, when a domain is added in OBAS, it is created in the PowerDNS server that works in OBAS (in earlier versions, this was the BIND server). The domain then automatically is synchronized with the proper nameservers, which are managed by OBAS.

The first step is checking that the domain was added to the PowerDNS server inside OBAS. The simplest way is checking if the domain can be resolved locally. Log in to OBAS by SSH and run the command below:

# dig soa customer.com @localhost

In the example above, we assume we added the domain "customer.com" and that we are trying to troubleshoot DNS synchronization problems with this domain.

If you see the local PowerDNS server answers correctly for the domain "customer.com," then check if the OBAS-managed nameservers are configured correctly for this domain. Repeat the command above for all nameservers that are configured to the host DNS zone for the domain "customer.com":

# dig soa customer.com @DNS_SERVER_IP

(Replace "DNS_SERVER_IP" with the actual IP address of each DNS server you want to check.)

If any DNS servers registered in OBAS that must have the domain zone "customer.com" do not answer to an SOA request, check if OBAS works with this server. Assume the IP address of the problem DNS server is 192.168.1.101. OBAS is working with slave DNS servers (those that are registered in OBAS) using the special user "namedsync," which is created on every DNS server registered in OBAS.

To check if DNS synchronization between OBAS and the slave name server is working correctly, log in to the OBAS server as root and perform the following steps:

# su - apache
$ ssh namedsync@192.168.1.101

As a result, you are logged in to the DNS server 192.168.1.101 without needing to enter the password. Authentication is performed using an SSH public/private keypair, which is configured during DNS server registration in OBAS (alongside the creation of the user "namedsync").

You may face several problems that will not allow you to log in to the DNS server using the command above. Check for them:

  • Check if the server 192.168.1.101 is available from the OBAS server at all -- simply try to ping it from OBAS. If you cannot ping the DNS server, check the routing and firewall rules in your network.
  • Check if the user "namedsync" exists on the DNS server 192.168.1.101 -- log in to this server by SSH as root and check if the user exists.
  • Check if the SSHD server is running on the DNS server. Start it if it is not.
  • Log in to the OBAS control panel. Go to "Top > Service Director > Domain Manager > Name Servers" and choose the name server with the IP 192.168.1.101. Check the parameters Port and SSH protocol version, and then make sure the corresponding port and protocols are configured correctly on the DNS server 192.168.1.101.
  • Check that the SSH RSA and DSA public keys of the user "apache" from the OBAS server (files "/var/www/.ssh/id_dsa.pub" and "/var/www/.ssh/identity.pub") are present in the file "/home/namedsync/.ssh/authorized_keys" on the DNS server 192.168.1.101. These exact keys are used to provide password-less authentication between OBAS and the slave name server. Make sure both keys are added to the file "authorized_keys" on 192.168.1.101 as a single line per key.

You may face a problem with a changed SSH key fingerprint when trying to log in by SSH from the OBAS server to the managed DNS server. In this case, the following output will be shown:

$ ssh namedsync@192.168.1.101
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It also is possible the RSA1 host key has just been changed.
The fingerprint for the RSA1 key sent by the remote host is
94:ae:38:8c:72:66:8d:35:87:cf:63:0e:d3:e1:c4:63.
Please contact your system administrator.
Add correct host key in /var/www/.ssh/known_hosts to get rid of this message.
Offending key in /var/www/.ssh/known_hosts:4
RSA1 host key for 192.168.1.101 has changed and you have requested strict checking.
Host key verification failed.

The latter problem may occur if you reinstalled the DNS server registered in OBAS and, as a result, the host SSH keys were changed on this server (you might also recreate the Virtuozzo Containers container where the DNS server is working).

The simplest way to solve the problem with changed SSH keys is either removing the corresponding entry from the file "/var/www/.ssh/known_hosts" on the OBAS server, or removing the whole file. Of course, you must be sure that SSH keys were not changed as a result of someone's malicious actions.

Imagine you performed all the actions listed above and solved the problem with SSH connectivity, and now OBAS is able to connect to the DNS server 192.168.1.101 without a password. However, the newly added domain "customer.com" still is not available on this DNS server.

OBAS adds all domains on managed nameservers to the file "/var/named/named.zones" ("/var/named/chroot/var/named/named.zones" if BIND on the slave DNS server is working in the chrooted environment. Check the "/etc/sysconfig/named" file, parameter "ROOTDIR"). The file "/var/named/named.zones" must be available for writing for the "namedsync" user. Log in to the name server 192.168.1.101 and check if it is as follows:

# ls -l /var/named/chroot/var/named/named.zones
-rw-r--r--  1 namedsync named 6501 Mar 18 14:23 /var/named/named.zones

If you see the user "namedsync" does not have "write" permissions for this file, you have to correct this problem:

# chown namedsync:named /var/named/named.zones
# chmod 644 /var/named/named.zones

It also makes sense to check other permissions and ownership.

After that, log in to the OBAS control panel and perform full DNS synchronization for the DNS server 192.168.1.101: go to "Top > Service Director > Domain Manager > Name Servers." Check the box to the left of the server with the IP address "192.168.1.101" and click the button "Synchronize."

Alternatively, you can log in to the OBAS server by SSH and perform synchronization using the utility "/usr/sbin/hspc-dns-sync.pl."

# /usr/sbin/hspc-dns-sync.pl -ns NS_ID

(Replace "NS_ID" with the actual ID of the DNS server 192.168.1.101, which is shown in the OBAS control panel in the list of nameservers.)

As a result of DNS synchronization, the domain "customer.com" should appear in the file "/var/named/named.zones" on the server 192.168.1.101. An entry like this should be added:

zone "customer.com" IN {
        type slave;
        masters { 192.168.1.100; };
        file "customer.com.zone";
};

In the example above, the IP address 192.168.1.100 is the address of the OBAS server. This is the master server for all DNS servers registered in OBAS, which are slave servers in turn. All slave servers are to transfer domain zones from the master, using native BIND tools.

If you see that the zone definition for the domain "customer.com" appears in the file "/var/named/named.zones" but the domain still cannot be resolved on the slave name server, then check if zone transfer is working between the OBAS master DNS server and the slave name server 192.168.1.101. To do this, log in to the slave DNS server by SSH and run the utility "dig," which should give you an output like this:

# dig axfr customer.com @192.168.1.100
; <<>> DiG 9.2.3 <<>> axfr provider.com @192.168.1.100
;; global options:  printcmd
customer.com.              1200    IN      SOA     obas.provider.com. root.obas.provider.com. 2010031800 7200 3600 604800 1200
customer.com.              600     IN      NS      ns2.oobas.provider.com.
customer.com.              600     IN      NS      ns1.obas.provider.com.
customer.com.              1200    IN      SOA     obas.provider.com. root.obas.provider.com. 2010031800 7200 3600 604800 1200
;; Query time: 4 msec
;; SERVER: 192.168.1.100#53(192.168.1.100)
;; WHEN: Thu Mar 18 15:20:54 2010
;; XFR size: 4 records

Note: The IP address of the master server (which is the OBAS server) must be used in the command above. In our example, it is 192.168.1.100, while the IP address of the DNS server is 192.168.1.101.

If you see the error message "Transfer failed issuing the command 'dig,'" it might mean that zone transfers are prohibited in the PowerDNS server on OBAS installation. Log in to the OBAS server by SSH as root and check the parameters "allow-axfr-ips" and "disable-axfr" in the file "/etc/powerdns/pdns.conf." All OBAS-managed name servers must be listed in the parameter allow-axfr-ips, or this parameter may even be commented out (which is not recommended for the sake of security).

Assuming the OBAS installation has two nameservers with the IP addresses 192.168.1.101 and 192.168.1.102, the parameter "allow-axfr-ips" must be set to the following value:

allow-axfr-ips=192.168.1.101 192.168.1.102

Also, check that the parameter "disable-axfr" is either commented or set to "no."

If you changed any parameter in the configuration file "/etc/powerdns/pdns.conf," then do not forget to restart the PowerDNS server for the changes to take effect:

# service pdns restart

Verify the OBAS log for __ssh_exec errors and fix them. For example, this error indicates that the configuration file is missing on the slave name servers. For example:

/var/log/hspc/hspc.log
--->8---
[2013/08/30 13:51:35] [DEBUG] [8663] [HSPC::MT::DM::NS::SlaveNS::SSH::__ssh_exec] OUT: ; ERR: grep: /etc/sysconfig/named: No such file or directory
---8<---

Solution: create the file /etc/sysconfig/named on the slave name servers.

or

/var/log/hspc/hspc.log
---8<---
[2014/02/10 16:28:11] [WARN] [17513] [HSPC::MT::DM::NS::SlaveNS::SSH::__scp] Executed local command scp  -o 'Protocol 2' -o 'BatchMode yes'  -o 'IdentityFile /var/www/.ssh/id_dsa'  -o 'IdentityFile /var/www/.ssh/identity'  -P 22 /tmp/S3UG51neAz namedsync@dns2.pbastest.com:/var/named/chroot/var/named/named.zones 2>&1  | grep -v Warning: FAILED (bash: scp: command not found
--->8---

Solution: install package openssh-clients on the slave name servers.

Please also make sure that named.conf files have not been improperly modified on slave DNS servers. For instance, the following string must not be commented out

include "/var/named/named.zones";

See also

DNS records synchronization troubles

DNS zones are not synchronized with a slave server

DNS zone synchronization

Internal content