Symptoms

Task "Collect disk usage statistics on host ng-host-1" fails with the following output:

Destination host 'ng-host-1' (#101), IP '10.20.30.40' : Quota error: Get quota operation has been failed (code: -2)

More information can be found in poa.log on the ng host (standalone NG host or director in Linux NG WebCluster):

DBG [task:72460529:1000306 2818:943825:b12f2b70 pleskd 1605629818]: [ DSC::getQuotaDiskUsageOnFS] ===> ENTRY
DBG [task:72460529:1000306 2818:943825:b12f2b70 pleskd 1605629818]: [ DSC::RpcQuotaManager::do_getQuota] ===> ENTRY
DBG [task:72460529:1000306 2818:943825:b12f2b70 pleskd 1605629818]: [ DSC::RpcQuotaManager::do_getQuota] RPC get quota (10.20.30.123, /srv/ngnfs40.provider.local/ngnfs40); UID: 123456 QTYPE: 0
DBG [task:72460529:1000306 2818:943825:b12f2b70 pleskd 1605629818]: [ DSC::RpcQuotaManager::do_getQuota] <=== EXIT [0.001018]
DBG [task:72460529:1000306 2818:943825:b12f2b70 lib 1605629818]: [ getQuotaDiskUsageOnFS] {module_id="HCL"; code="2500"} Quota error: Get quota operation has been failed (code: -2)
DBG [task:72460529:1000306 2818:943825:b12f2b70 pleskd 1605629818]: [ DSC::getQuotaDiskUsageOnFS] <=== EXIT (by exception) [0.005569]

Use mount command to find affected mount point:

[root@ng-host-1 ~]# mount
10.20.30.123:/srv/ngnfs40.provider.local/ngnfs40 on /var/www/vhosts/10 type nfs (rw,nosuid,nodev,noatime,nodiratime,proto=tcp,intr,soft,nfsvers=3,actimeo=10,wsize=32768,rsize=32768,nocto,lock,sloppy,addr=10.20.30.123)

On target NFS share (10.20.30.123) quotas check gives negative results. Filter data by affected partition from above logs:

# mount | grep ngnfs40
/dev/sdb1 on /srv/ngnfs40.provider.local/ngnfs40 type ext4 (rw,nosuid,nodev,noexec,noatime,nodiratime,discard,quota,usrquota,data=ordered)

# quotaon -p /dev/sdb1
group quota on /srv/ngnfs40.provider.local/ngnfs40 (/dev/sdb1) is off
user quota on /srv/ngnfs40.provider.local/ngnfs40 (/dev/sdb1) is off

These two commands are showing that quotas are supported by filesystem, but not enabled.

Cause

RPC quotas are turned off for the mounted partition.

Resolution

In order to make quota mechanism working, please, enable quotas on NG NFS host shared partition during nearest maintenance window by performing the following actions on NFS node itself as root:

  1. Create user quota files on target file system (please note that quotacheck invocation will try to remount target FS read-only for the time of processing to ensure that usage does not change. That could be overridden by adding option '-m' to quotacheck invocation):

    # quotacheck -cu /srv/ngnfs40.provider.local/ngnfs40
    
  2. Acquire current quota usage from filesystem (again, use -m option to avoid remounting):

    # quotacheck -avu
    

    Positive results will be indicated by the following output:

    quotacheck: Checked xxxxx directories and yyyyy files
    
  3. Enable quota accounting for filesystem:

    # quotaon -u /srv/ngnfs40.provider.local/ngnfs40
    

Then quotas start to work correctly and task shall no longer fail.

External references:

RedHat Storage Administration Guide

Internal content