Item paging stops at HWM if there are no replicas

This is not a recent regression - it appears to have been introduced in 3.0.2 , as the "default" value returned by getMemResidentPer() if there are no items changed from 0 (source) to 100 (source).
As Ephemeral buckets are always 100% resident, they are not affected - store.getActiveResidentRatio() < store.getReplicaResidentRatio() will never be true.

Depending on version, one of the following snippets is present in PagingVisitor::visitBucket

    // skip active vbuckets if active resident ratio is lower than replica

    double current = static_cast<double>(stats.getEstimatedTotalMemoryUsed());

    double lower = static_cast<double>(stats.mem_low_wat);

    double high = static_cast<double>(stats.mem_high_wat);

    if (vb->getState() == vbucket_state_active && current < high &&

        store.getActiveResidentRatio() < store.getReplicaResidentRatio()) {

        return;

    }

        // Non-ephemeral: skip active vbuckets if active resident ratio is

        // lower than replica.

        // (Ephemeral can _only_ evict from active so don't want to skip them!)

        if (!isEphemeral && vb->getState() == vbucket_state_active) {

            double high = store.getPageableMemHighWatermark();

            if (current < high && store.getActiveResidentRatio() <

                                          store.getReplicaResidentRatio()) {

                return;

            }

        }

where version 2 additionally guards against this issue for ephemeral buckets, as ephemeral buckets cannot evict from replicas.

The value returned by store.getReplicaResidentRatio() is originally generated from

size_t getMemResidentPer() {

        if (numItems && numItems >= nonResident) {

            size_t numResident = numItems - nonResident;

            return size_t(numResident * 100.0) / (numItems);

        }

        // Note: access-scanner depends on 100% being returned for this case

        return 100;

    }

Note that this defaults to 100 if there are no items tracked for the selected vbucket state.

If there are no replica vbuckets then store.getActiveResidentRatio() < store.getReplicaResidentRatio() becomes true as soon as any active vbucket has non resident items. In that case, when visiting active buckets the PagingVisitor early exits as soon as current < high is true, skipping the current vbucket.

This does not prevent eviction functioning, but does mean memory usage will only drop slightly below the high watermark, leading to freqent ItemPager runs each recovering a comparatively small amount of data.

Easily demonstrated with cluster run. With no replicas, after loading documents to make the bucket <100% resident, mem_used remains just below the high watermark after eviction, with replicas>=1 it drops below the low watermark.

Changing the low watermark to an dramatically lower percentage (say, 10%) makes this demonstration clearer.