Description
As seen in a recent case:
- Due to the high amount of messages the message queue of ns_port_server got backlogged and grew.
- The heap grew to the quite a big size due to all the
{data, Bin}
tuples in the queue.
- The global heap grew even more because all these tuples reference binaries.
- Then the message stream was reduced to a trickle.
- At some point while the queue backlog was clearing, the garbage collection happened, which freed approximately half of the heap, but the heap size was not reduced, because less than 75% of data was freed.
- Now we have quite a big free chunk of the heap and quite slow heap allocation rate, so it could take quite a lot of time for the next garbage collection to happen.
- Therefore we have tons of binaries stuck in the global heap.
the remedy: force garbage collection of ns_port_server periodically.
Attachments
Issue Links
- relates to
-
MB-37141 ns_server should disable memcached console logging
- Open
Gerrit Reviews
For Gerrit Dashboard: MB-37136 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
118905,1 | MB-37136 garbage collect ns_port_server, but not more often than | master | ns_server | Status: NEW | +1 | +1 |