unexecpted swap usage on node while loading data

Loaded 15 Million items(sets only) on a 9 node cluster, causing it to go into dgm with 77% active-resident, 68% replica-resident
No views, no rebalancing.

Bucket Compaction kicks in and some nodes go into swap.

Attaching atop history from node(10.3.3.59) as it has some swap usage, along with diags from master node (59) with screenshot.

output from free on 10.3.3.59:
root@plum-016:~# free
total used free shared buffers cached
Mem: 5958344 5905504 52840 0 28044 1497412
-/+ buffers/cache: 4380048 1578296
Swap: 4194296 236 4194060

Recent excerpt from atop:
ATOP - plum-016 2012/09/11 12:23:32 20 seconds elapsed
PRC | sys 0.98s | user 2.10s | #proc 117 | #zombie 0 | #exit 3 |
CPU | sys 2% | user 5% | irq 0% | idle 304% | wait 88% |
CPU | steal 0% | stl/cpu 0% | | | |
cpu | sys 1% | user 3% | irq 0% | idle 5% | cpu000 w 90% |
cpu | sys 0% | user 1% | irq 0% | idle 98% | cpu001 w 1% |
cpu | sys 0% | user 0% | irq 0% | idle 100% | cpu003 w 0% |
cpu | sys 0% | user 0% | irq 0% | idle 98% | cpu002 w 2% |
CPL | avg1 1.20 | avg5 1.60 | avg15 1.91 | csw 21116 | intr 13087 |
MEM | tot 5.7G | free 41.2M | cache 1.4G | buff 28.2M | slab 73.3M |
SWP | tot 4.0G | free 4.0G | | vmcom 4.4G | vmlim 6.8G |
PAG | scan 8008 | stall 0 | | swin 0 | swout 0 |
DSK | xvdb | busy 49% | read 1341 | write 1320 | avio 7 ms |
DSK | xvda | busy 9% | read 136 | write 308 | avio 8 ms |
NET | transport | tcpi 1632 | tcpo 1610 | udpi 0 | udpo 0 |
NET | network | ipi 1692 | ipo 1610 | ipfrw 0 | deliv 1635 |
NET | lo ---- | pcki 420 | pcko 420 | si 586 Kbps | so 586 Kbps |
NET | eth0 ---- | pcki 1414 | pcko 1191 | si 108 Kbps | so 256 Kbps |

PID SYSCPU USRCPU VGROW RGROW RDDSK WRDSK ST EXC S CPU CMD 1/1
11914 0.57s 1.40s 13380K 5976K 38296K 41856K – - S 5% beam.smp
15723 0.12s 0.27s 0K 0K - - NE 0 E 1% <couch_compac>
15722 0.08s 0.20s 0K 0K - - -E 0 E 1% <couch_compac>
12061 0.02s 0.16s 0K 0K 0K 144K – - S 0% memcached
15726 0.11s 0.01s 18616K 1568K 7552K 5824K N- - D 0% couch_compact
13482 0.02s 0.01s 0K 0K 0K 12K – - R 0% atop
15724 0.00s 0.03s 50604K 2672K 2376K 0K N- - S 0% sshd
398 0.02s 0.00s 0K 0K 0K 132K – - S 0% kjournald
29196 0.01s 0.00s 0K 0K 0K 0K – - S 0% bash
707 0.01s 0.00s 0K 36K 628K 0K – - S 0% sshd
15727 0.00s 0.01s 50604K 1096K 0K 0K N- - S 0% sshd
12074 0.00s 0.01s 0K 0K 0K 0K – - S 0% sigar_port
51 0.01s 0.00s 0K 0K 0K 0K – - S 0% kswapd0
189 0.01s 0.00s 0K 0K 0K 76K – - S 0% kjournald
11949 0.00s 0.00s 0K 0K 0K 0K – - S 0% sh
11950 0.00s 0.00s 0K 0K 0K 0K – - S 0% memsup
15725 0.00s 0.00s 0K 0K - - NE 0 E 0% <df>