Details
-
Bug
-
Resolution: Done
-
Major
-
None
-
None
-
None
-
1
Description
Testing End to end traceability with current sdk 2.6.0dp1 and vulcan build 5.5.0-2211.
I'am slowing down the network on sdk with less then 500ms still few slow logs are getting generated.
Here is the code snippet:
NodeHost masterNode = cluster.getMasterNode();
try {
System.out.println("Ramping up for 30 seconds");
Thread.sleep(30000);
System.out.println("Introducing delay and running workload for 240 seconds");
RemoteCommands.runCommand(masterNode.getSSH(), "tc qdisc add dev eth0 root netem delay 400ms 200ms loss 10% 25%").get();
Thread.sleep(240000);
System.out.println("Cancelling workload now");
isCancelled = true;
kvQueryWorkload.get();
ftsWorkload.get();
n1qlWorkload.get();
executor.shutdown();
} finally {
System.out.println("Removing simulated delay");
RemoteCommands.runCommand(masterNode.getSSH(), "tc qdisc del dev eth0 root netem").get();
System.out.println("Removed simulated delay");
}
Actual Result:
[559.30 WARN] (ThresholdLogReporter warn:150) Operations over threshold: [{"top":[\{"local_id":"79A583CE4D121060/FFFFFFFF91B78D85","local_address":"192.168.3.193:55321","operation_id":"n1ql:f3ffbd97-4ae7-4ac0-943e-1954a8de5abb","dispatch_us":1281080,"remote_address":"172.23.123.176:8093","total_us":1281177}],"service":"n1ql","count":1}]
number of Get operations so far 4017
number of Get operations so far 4018
number of Get operations so far 4019
number of Get operations so far 4020
number of Get operations so far 4021
number of Get operations so far 4022
number of Get operations so far 4023
number of Get operations so far 4024
number of Get operations so far 4025
number of Get operations so far 4026
number of Get operations so far 4027
[560.74 WARN] (ThresholdLogReporter warn:150) Operations over threshold: [{"top":[\{"server_us":32,"local_id":"79A583CE4D121060/FFFFFFFFD9183AA6","local_address":"192.168.3.193:55268","operation_id":"get:0x4b56","decode_us":11,"dispatch_us":536021,"remote_address":"172.23.123.176:11210","total_us":536169}],"service":"kv","count":1}]
number of Get operations so far 4028
number of Get operations so far 4029
number of Get operations so far 4030
number of Get operations so far 4031
Cancelling workload now
number of Get operations so far 4032
Total number of Get operations 4032
Removing simulated delay
[562.09 WARN] (ThresholdLogReporter warn:150) Operations over threshold: [{"top":[\{"server_us":23,"local_id":"79A583CE4D121060/FFFFFFFFD9183AA6","local_address":"192.168.3.193:55268","operation_id":"get:0x4b60","decode_us":15,"dispatch_us":634274,"remote_address":"172.23.123.176:11210","total_us":634427}],"service":"kv","count":1},{"top":[\{"local_id":"79A583CE4D121060/FFFFFFFF91B78D85","local_address":"192.168.3.193:55321","operation_id":"n1ql:ba343141-5b86-4d49-9f92-e56de08698dd","dispatch_us":1147966,"remote_address":"172.23.123.176:8093","total_us":1148036}],"service":"n1ql","count":1}]
Removed simulated delay
Downloading server logs
[581.33 INFO] (Endpoint info:120) [/172.23.121.38:8094][SearchEndpoint]: Got notified from Channel as inactive, attempting reconnect.
[581.38 INFO] (Endpoint info:120) [/172.23.121.38:8094][SearchEndpoint]: Got notified from Channel as inactive, attempting reconnect.
Finished downloading server logs
Is this expected result or slow logs should not be generated at all ?