Loading...

Details

Type: Bug
Resolution: Duplicate
Priority: Critical
Fix Version/s: 3.0
Affects Version/s: 3.0
Component/s: ns_server
Security Level: Public
Labels:
None
Environment:
centos-6

Is this a Regression?:
Yes

Description

Using UPR, centos-64 bit, Version:: 3.0.0-584

1. Add 5 nodes to cluster
2. Create default bucket with replica=1
3. Add 50000 items
4. Rebalance out 1 node

Cluster Nodes

10.6.2.144
10.6.2.147
10.6.2.148
10.6.2.149
10.6.2.150

Rebalanced-out Node:: 10.6.2.150

Expected UUIDs of buckets is the same after we rebalance out

Observed Difference in UUIDs which are a part of 10.6.2.150

vb_28 :

{'uuid': 'Expected 170724551552615 :: Actual 10627939076719'}

vb_29 :

{'uuid': 'Expected 45359036031519 :: Actual 59640339796757'}

vb_26 :

{'uuid': 'Expected 133471459893767 :: Actual 114252468159585''}

vb_27 :

{'uuid': 'Expected 104126006114437 :: Actual 86374568011280'}

vb_6 :

{'uuid': 'Expected 175130199501286 :: Actual 23856384718339'}

vb_13 :

{'uuid': 'Expected 84680084219325 :: Actual 233007425883197'}

vb_31 :

{'uuid': 'Expected 170201395300887 :: Actual 174005664911105'}

vb_30 :

{'uuid': 'Expected 83362933025898 :: Actual 229555825112404'}

Dump of Bucket, Node, Vbucket Information (uuid, high_seqno, purge_seqno)

Before we rebalanced-out
______________________

----- Bucket default -----

------------~~Node 10.6.2.147~~-----------

for vbucket vb_9

:: for key high_seqno = 1558

:: for key uuid = 224689180220009

:: for key purge_seqno = 0

for vbucket vb_8

:: for key purge_seqno = 0

:: for key uuid = 24173416366515

:: for key high_seqno = 1569

for vbucket vb_1

:: for key purge_seqno = 0

:: for key uuid = 3382665152986

:: for key high_seqno = 1569

for vbucket vb_0

:: for key purge_seqno = 0

:: for key uuid = 252550884809196

:: for key high_seqno = 1551

for vbucket vb_7

:: for key high_seqno = 1551

:: for key uuid = 160369294390021

:: for key purge_seqno = 0

for vbucket vb_11

:: for key purge_seqno = 0

:: for key uuid = 91922888028629

:: for key high_seqno = 1560

for vbucket vb_10

:: for key high_seqno = 1568

:: for key uuid = 98217872738904

:: for key purge_seqno = 0

for vbucket vb_13

:: for key purge_seqno = 0

:: for key uuid = 84680084219325

:: for key high_seqno = 1568

for vbucket vb_12

:: for key high_seqno = 1560

:: for key uuid = 75150401842511

:: for key purge_seqno = 0

for vbucket vb_15

:: for key purge_seqno = 0

:: for key uuid = 21343486345542

:: for key high_seqno = 1569

for vbucket vb_16

:: for key high_seqno = 1560

:: for key uuid = 113002939605931

:: for key purge_seqno = 0

for vbucket vb_21

:: for key purge_seqno = 0

:: for key uuid = 131979757081143

:: for key high_seqno = 1559

for vbucket vb_28

:: for key high_seqno = 1553

:: for key uuid = 170724551552615

:: for key purge_seqno = 0

------------~~Node 10.6.2.144~~-----------

for vbucket vb_8

:: for key purge_seqno = 0

:: for key uuid = 24173416366515

:: for key high_seqno = 1569

for vbucket vb_1

:: for key purge_seqno = 0

:: for key uuid = 3382665152986

:: for key high_seqno = 1569

for vbucket vb_0

:: for key purge_seqno = 0

:: for key uuid = 252550884809196

:: for key high_seqno = 1551

for vbucket vb_3

:: for key high_seqno = 1570

:: for key uuid = 189410128542953

:: for key purge_seqno = 0

for vbucket vb_2

:: for key high_seqno = 1552

:: for key uuid = 202175152188264

:: for key purge_seqno = 0

for vbucket vb_5

:: for key high_seqno = 1552

:: for key uuid = 272505847148341

:: for key purge_seqno = 0

for vbucket vb_4

:: for key purge_seqno = 0

:: for key uuid = 170782289546886

:: for key high_seqno = 1570

for vbucket vb_7

:: for key high_seqno = 1551

:: for key uuid = 160369294390021

:: for key purge_seqno = 0

for vbucket vb_6

:: for key purge_seqno = 0

:: for key uuid = 175130199501286

:: for key high_seqno = 1569

for vbucket vb_14

:: for key purge_seqno = 0

:: for key uuid = 9666779467087

:: for key high_seqno = 1558

for vbucket vb_20

:: for key high_seqno = 1569

:: for key uuid = 226337594275581

:: for key purge_seqno = 0

for vbucket vb_26

:: for key purge_seqno = 0

:: for key uuid = 133471459893767

:: for key high_seqno = 1569

for vbucket vb_27

:: for key purge_seqno = 0

:: for key uuid = 104126006114437

:: for key high_seqno = 1553

------------~~Node 10.6.2.150~~-----------

for vbucket vb_5

:: for key high_seqno = 1552

:: for key uuid = 272505847148341

:: for key purge_seqno = 0

for vbucket vb_6

:: for key purge_seqno = 0

:: for key uuid = 175130199501286

:: for key high_seqno = 1569

for vbucket vb_28

:: for key high_seqno = 1553

:: for key uuid = 170724551552615

:: for key purge_seqno = 0

for vbucket vb_29

:: for key purge_seqno = 0

:: for key uuid = 45359036031519

:: for key high_seqno = 1569

for vbucket vb_13

:: for key purge_seqno = 0

:: for key uuid = 84680084219325

:: for key high_seqno = 1568

for vbucket vb_26

:: for key purge_seqno = 0

:: for key uuid = 133471459893767

:: for key high_seqno = 1569

for vbucket vb_19

:: for key purge_seqno = 0

:: for key uuid = 251498636389290

:: for key high_seqno = 1569

for vbucket vb_27

:: for key purge_seqno = 0

:: for key uuid = 104126006114437

:: for key high_seqno = 1553

for vbucket vb_24

:: for key high_seqno = 1571

:: for key uuid = 184565815683126

:: for key purge_seqno = 0

for vbucket vb_25

:: for key high_seqno = 1552

:: for key uuid = 266765133502323

:: for key purge_seqno = 0

for vbucket vb_31

:: for key purge_seqno = 0

:: for key uuid = 170201395300887

:: for key high_seqno = 1571

for vbucket vb_30

:: for key high_seqno = 1552

:: for key uuid = 83362933025898

:: for key purge_seqno = 0

------------~~Node 10.6.2.148~~-----------

for vbucket vb_9

:: for key high_seqno = 1558

:: for key uuid = 224689180220009

:: for key purge_seqno = 0

for vbucket vb_10

:: for key high_seqno = 1568

:: for key uuid = 98217872738904

:: for key purge_seqno = 0

for vbucket vb_3

:: for key high_seqno = 1570

:: for key uuid = 189410128542953

:: for key purge_seqno = 0

for vbucket vb_2

:: for key high_seqno = 1552

:: for key uuid = 202175152188264

:: for key purge_seqno = 0

for vbucket vb_29

:: for key purge_seqno = 0

:: for key uuid = 45359036031519

:: for key high_seqno = 1569

for vbucket vb_15

:: for key purge_seqno = 0

:: for key uuid = 21343486345542

:: for key high_seqno = 1569

for vbucket vb_14

:: for key purge_seqno = 0

:: for key uuid = 9666779467087

:: for key high_seqno = 1558

for vbucket vb_17

:: for key purge_seqno = 0

:: for key uuid = 50245200692736

:: for key high_seqno = 1570

for vbucket vb_16

:: for key high_seqno = 1560

:: for key uuid = 113002939605931

:: for key purge_seqno = 0

for vbucket vb_19

:: for key purge_seqno = 0

:: for key uuid = 251498636389290

:: for key high_seqno = 1569

for vbucket vb_18

:: for key high_seqno = 1559

:: for key uuid = 5501229932749

:: for key purge_seqno = 0

for vbucket vb_22

:: for key high_seqno = 1570

:: for key uuid = 247498759341926

:: for key purge_seqno = 0

for vbucket vb_23

:: for key purge_seqno = 0

:: for key uuid = 74620381341917

:: for key high_seqno = 1560

------------~~Node 10.6.2.149~~-----------

for vbucket vb_4

:: for key purge_seqno = 0

:: for key uuid = 170782289546886

:: for key high_seqno = 1570

for vbucket vb_11

:: for key purge_seqno = 0

:: for key uuid = 91922888028629

:: for key high_seqno = 1560

for vbucket vb_12

:: for key purge_seqno = 0

:: for key uuid = 75150401842511

:: for key high_seqno = 1560

for vbucket vb_17

:: for key purge_seqno = 0

:: for key uuid = 50245200692736

:: for key high_seqno = 1570

for vbucket vb_18

:: for key high_seqno = 1559

:: for key uuid = 5501229932749

:: for key purge_seqno = 0

for vbucket vb_20

:: for key high_seqno = 1569

:: for key uuid = 226337594275581

:: for key purge_seqno = 0

for vbucket vb_21

:: for key purge_seqno = 0

:: for key uuid = 131979757081143

:: for key high_seqno = 1559

for vbucket vb_22

:: for key high_seqno = 1570

:: for key uuid = 247498759341926

:: for key purge_seqno = 0

for vbucket vb_23

:: for key purge_seqno = 0

:: for key uuid = 74620381341917

:: for key high_seqno = 1560

for vbucket vb_24

:: for key high_seqno = 1571

:: for key uuid = 184565815683126

:: for key purge_seqno = 0

for vbucket vb_25

:: for key high_seqno = 1552

:: for key uuid = 266765133502323

:: for key purge_seqno = 0

for vbucket vb_31

:: for key purge_seqno = 0

:: for key uuid = 170201395300887

:: for key high_seqno = 1571

for vbucket vb_30

:: for key high_seqno = 1552

:: for key uuid = 83362933025898

:: for key purge_seqno = 0

After we rebalanced-out
____________________

----- Bucket default -----

------------~~Node 10.6.2.147~~-----------

for vbucket vb_9

:: for key high_seqno = 2337

:: for key uuid = 224689180220009

:: for key purge_seqno = 0

for vbucket vb_8

:: for key purge_seqno = 0

:: for key uuid = 24173416366515

:: for key high_seqno = 2354

for vbucket vb_1

:: for key purge_seqno = 0

:: for key uuid = 3382665152986

:: for key high_seqno = 2355

for vbucket vb_0

:: for key purge_seqno = 0

:: for key uuid = 252550884809196

:: for key high_seqno = 2326

for vbucket vb_7

:: for key high_seqno = 2326

:: for key uuid = 160369294390021

:: for key purge_seqno = 0

for vbucket vb_6

:: for key purge_seqno = 0

:: for key uuid = 23856384718339

:: for key high_seqno = 2355

for vbucket vb_11

:: for key purge_seqno = 0

:: for key uuid = 91922888028629

:: for key high_seqno = 2339

for vbucket vb_10

:: for key high_seqno = 2353

:: for key uuid = 98217872738904

:: for key purge_seqno = 0

for vbucket vb_12

:: for key high_seqno = 2339

:: for key uuid = 75150401842511

:: for key purge_seqno = 0

for vbucket vb_15

:: for key purge_seqno = 0

:: for key uuid = 21343486345542

:: for key high_seqno = 2354

for vbucket vb_26

:: for key purge_seqno = 0

:: for key uuid = 114252468159585

:: for key high_seqno = 2355

for vbucket vb_16

:: for key high_seqno = 2339

:: for key uuid = 113002939605931

:: for key purge_seqno = 0

for vbucket vb_21

:: for key purge_seqno = 0

:: for key uuid = 131979757081143

:: for key high_seqno = 2338

for vbucket vb_24

:: for key high_seqno = 2357

:: for key uuid = 184565815683126

:: for key purge_seqno = 0

for vbucket vb_31

:: for key purge_seqno = 0

:: for key uuid = 174005664911105

:: for key high_seqno = 2357

for vbucket vb_28

:: for key high_seqno = 2328

:: for key uuid = 10627939076719

:: for key purge_seqno = 0

------------~~Node 10.6.2.144~~-----------

for vbucket vb_8

:: for key purge_seqno = 0

:: for key uuid = 24173416366515

:: for key high_seqno = 2354

for vbucket vb_1

:: for key purge_seqno = 0

:: for key uuid = 3382665152986

:: for key high_seqno = 2355

for vbucket vb_0

:: for key purge_seqno = 0

:: for key uuid = 252550884809196

:: for key high_seqno = 2326

for vbucket vb_3

:: for key high_seqno = 2356

:: for key uuid = 189410128542953

:: for key purge_seqno = 0

for vbucket vb_2

:: for key high_seqno = 2327

:: for key uuid = 202175152188264

:: for key purge_seqno = 0

for vbucket vb_5

:: for key high_seqno = 2327

:: for key uuid = 272505847148341

:: for key purge_seqno = 0

for vbucket vb_4

:: for key purge_seqno = 0

:: for key uuid = 170782289546886

:: for key high_seqno = 2356

for vbucket vb_7

:: for key high_seqno = 2326

:: for key uuid = 160369294390021

:: for key purge_seqno = 0

for vbucket vb_28

:: for key high_seqno = 2328

:: for key uuid = 10627939076719

:: for key purge_seqno = 0

for vbucket vb_29

:: for key purge_seqno = 0

:: for key uuid = 59640339796757

:: for key high_seqno = 2355

for vbucket vb_13

:: for key purge_seqno = 0

:: for key uuid = 233007425883197

:: for key high_seqno = 2353

for vbucket vb_14

:: for key purge_seqno = 0

:: for key uuid = 9666779467087

:: for key high_seqno = 2337

for vbucket vb_20

:: for key high_seqno = 2354

:: for key uuid = 226337594275581

:: for key purge_seqno = 0

for vbucket vb_30

:: for key high_seqno = 2327

:: for key uuid = 229555825112404

:: for key purge_seqno = 0

for vbucket vb_26

:: for key purge_seqno = 0

:: for key uuid = 114252468159585

:: for key high_seqno = 2355

for vbucket vb_27

:: for key purge_seqno = 0

:: for key uuid = 86374568011280

:: for key high_seqno = 2328

------------~~Node 10.6.2.148~~-----------

for vbucket vb_9

:: for key high_seqno = 2337

:: for key uuid = 224689180220009

:: for key purge_seqno = 0

for vbucket vb_10

:: for key high_seqno = 2353

:: for key uuid = 98217872738904

:: for key purge_seqno = 0

for vbucket vb_3

:: for key high_seqno = 2356

:: for key uuid = 189410128542953

:: for key purge_seqno = 0

for vbucket vb_2

:: for key high_seqno = 2327

:: for key uuid = 202175152188264

:: for key purge_seqno = 0

for vbucket vb_29

:: for key purge_seqno = 0

:: for key uuid = 59640339796757

:: for key high_seqno = 2355

for vbucket vb_13

:: for key purge_seqno = 0

:: for key uuid = 233007425883197

:: for key high_seqno = 2353

for vbucket vb_15

:: for key purge_seqno = 0

:: for key uuid = 21343486345542

:: for key high_seqno = 2354

for vbucket vb_14

:: for key purge_seqno = 0

:: for key uuid = 9666779467087

:: for key high_seqno = 2337

for vbucket vb_17

:: for key purge_seqno = 0

:: for key uuid = 50245200692736

:: for key high_seqno = 2355

for vbucket vb_16

:: for key high_seqno = 2339

:: for key uuid = 113002939605931

:: for key purge_seqno = 0

for vbucket vb_19

:: for key purge_seqno = 0

:: for key uuid = 251498636389290

:: for key high_seqno = 2354

for vbucket vb_18

:: for key high_seqno = 2338

:: for key uuid = 5501229932749

:: for key purge_seqno = 0

for vbucket vb_22

:: for key high_seqno = 2355

:: for key uuid = 247498759341926

:: for key purge_seqno = 0

for vbucket vb_23

:: for key purge_seqno = 0

:: for key uuid = 74620381341917

:: for key high_seqno = 2338

for vbucket vb_25

:: for key high_seqno = 2327

:: for key uuid = 266765133502323

:: for key purge_seqno = 0

for vbucket vb_27

:: for key purge_seqno = 0

:: for key uuid = 86374568011280

:: for key high_seqno = 2328

------------~~Node 10.6.2.149~~-----------

for vbucket vb_5

:: for key high_seqno = 2327

:: for key uuid = 272505847148341

:: for key purge_seqno = 0

for vbucket vb_4

:: for key purge_seqno = 0

:: for key uuid = 170782289546886

:: for key high_seqno = 2356

for vbucket vb_6

:: for key purge_seqno = 0

:: for key uuid = 23856384718339

:: for key high_seqno = 2355

for vbucket vb_11

:: for key purge_seqno = 0

:: for key uuid = 91922888028629

:: for key high_seqno = 2339

for vbucket vb_19

:: for key purge_seqno = 0

:: for key uuid = 251498636389290

:: for key high_seqno = 2354

for vbucket vb_12

:: for key purge_seqno = 0

:: for key uuid = 75150401842511

:: for key high_seqno = 2339

for vbucket vb_17

:: for key purge_seqno = 0

:: for key uuid = 50245200692736

:: for key high_seqno = 2355

for vbucket vb_18

:: for key high_seqno = 2338

:: for key uuid = 5501229932749

:: for key purge_seqno = 0

for vbucket vb_20

:: for key high_seqno = 2354

:: for key uuid = 226337594275581

:: for key purge_seqno = 0

for vbucket vb_21

:: for key purge_seqno = 0

:: for key uuid = 131979757081143

:: for key high_seqno = 2338

for vbucket vb_22

:: for key high_seqno = 2355

:: for key uuid = 247498759341926

:: for key purge_seqno = 0

for vbucket vb_23

:: for key purge_seqno = 0

:: for key uuid = 74620381341917

:: for key high_seqno = 2338

for vbucket vb_24

:: for key high_seqno = 2357

:: for key uuid = 184565815683126

:: for key purge_seqno = 0

for vbucket vb_25

:: for key purge_seqno = 0

:: for key uuid = 266765133502323

:: for key high_seqno = 2327

for vbucket vb_31

:: for key purge_seqno = 0

:: for key uuid = 174005664911105

:: for key high_seqno = 2357

for vbucket vb_30

:: for key high_seqno = 2327

:: for key uuid = 229555825112404

:: for key purge_seqno = 0

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending
- Thumbnails
- List
- Download All

rebalance_in_detail
20 kB
18/Apr/14 12:50 AM
rebalance_in.tar.gz
19.50 MB
18/Apr/14 12:50 AM
rebalance_out.tar.gz
1.95 MB
17/Apr/14 6:57 PM

Gerrit Reviews

- Issue Only
- Show All Reviews
- Show Open Reviews
- Show All Issues
- Show Open Issues

No reviews matched the request. Check your Options in the drop-down menu of this sections header.

UPR:: Rebalance-out/in node results in change in vbucket UUID

Details

Description

Attachments

Attachments

Gerrit Reviews

Activity

People

Dates

Gerrit Reviews

PagerDuty